Leading Innovation Through Methodical Experiments

by peter

The Wall Street Journal’s ideas reporter, Pfred Dvorak, wrote a great article describing Best Buy’s experimental forecasting system entitled, “Best Buy Taps ‘Prediction Market’” — an excellent example of how companies are using focused experiments to innovate their internal processes. Anyone interested in innovation should take note.

Essentially, Best Buy faces a massive challenge — how to accurately forecast its annual sales of over $40 billion, throughout over 1,000 stores worldwide, across thousands of SKUs, as well as sales online. This is retail, so customer demand is fickle and market insight is highly dispersed, and small percentage point differences between forecast and actual sales can have a big impact on the bottom line.

In 2004, Jeff Severts, then head of Best Buy’s advertizing division, found that eight merchant teams generated the consolidated forecast, i.e. those merchants who ordered the TVs, computers, video games, etc. As Severts told Gary Hamel, it was not uncommon for those forecasts to be off by 10% fewer than 30 days out. Serverts then heard James Surowiecki speak about his book The Wisdom of the Crowds, and wondered if employees from throughout Best Buy could collectively provide the market insight and information to more accurately predict sales than the merchant teams.

Severts was allowed to start with a low-risk, focused experiment. For a $50 prize, he challenged employees throughout Best Buy’s $1 billion gift card division to estimate monthly gift card sales. He received 192 estimates that, when aggregated, was off actual sales by less than one-half of one percent, compared a 5% miss for the gift-card forecasting team. That successful outcome led to a much larger experiment: estimating Best Buy’s 2005 holiday sales. The results spoke for themselves — the crowd’s estimate was 99.9% accurate four months out, compared to 93% for the merchants.

As yesterday’s WSJ article describes, Best Buy continues to test and refine its prediction systems, but the lessons are simple: leaders that give employees permission run low-risk experiments are much more likely to innovate their products, services, processes, and even their company culture than those who believe that insights come only from small, select groups of people.

Great innovative leaders, such as Proctor & Gamble CEO A.G. Lafley, Brad Anderson, CEO at Best Buy, or the folks at Google surround themselves with diverse points of view and constantly use focused experiments to test new ideas. If you think you have the best insight about a complex problem, you probably don’t, but worse, you’re probably not a good leader.

If you enjoyed this post, make sure you subscribe to my RSS feed!

{ 4 comments… read them below or add one }

AndrewBoldman June 4, 2009 at 12:28 pm

Great post! Just wanted to let you know you have a new subscriber- me!

Reply

online cna classes June 24, 2010 at 7:51 am

found your site on del.icio.us today and really liked it.. i bookmarked it and will be back to check it out some more later

Reply

financial aid for college July 19, 2010 at 5:30 pm

Valuable info. Lucky me I found your site by accident, I bookmarked it.

Reply

Vasu Durgavarjhula June 25, 2011 at 12:50 am

Love your articles. Very insightful and thought provoking. If i understood correctly, the key principle of little bets approach is to focus on “learning” instead of focusing on getting “confirmation” on something. The truth is no one knows if they have the perfect idea. It’s only through experimentation and learning can you take an initial idea and develop it.

Reply

Leave a Comment

Next post: