Written by


Key questions to ask before using experimentation for innovation

In the great pantheon of failed products, there is perhaps a special place for the Amazon Fire Phone. Launched in July 2014 after four years of development, the phone was off the market just 13 months later due to insufficient customer demand. This forced Amazon to write down $170 million.

Why did it fail? The phone’s developers had apparently built the phone for Amazon founder Jeff Bezos, who had involved himself in every aspect of its development, according to Fast Company.  

Critically, they developed the phone in isolation, rather than identifying the main premises around which the product was based and testing these with end users - including the key one: would users be interested in an Amazon-branded high-end smartphone?

Experimentation puts the customer in the foreground from the very beginning, rigorously testing every hypothesis about the business model along the way

So, would a different approach have led to more success for Amazon? One option would have been to adopt experimentation into their research and development process, an approach which has started to gain prominence, as demonstrated by best-selling books, such as The Lean Startup.

Foregrounding the customer at every stage

Experimentation puts the customer in the foreground from the very beginning, rigorously testing every hypothesis about the business model along the way. If any given premise of the product or service doesn’t stick, then it’s back to the drawing board.

Before anything else, the central hypothesis needs to be tested. For example, Zappos launched a bare-bones website to establish whether people would buy shoes online. Its founding team validated this hypothesis by initially buying the shoes in person from competitors and shipping them out.

This approach allows startups to test and tweak the concept based on experiments. They only make a serious investment once they have proven customer demand is there. 

There are potential risks associated with experimentation, one of the greatest being that it may foster a sort of conservatism, favouring incrementalism over bold innovations

Could established firms follow startups in using experimentation? There has already been some uptake, including, eye-catchingly, from Proctor & Gamble. Here, an approach which chief innovation officer Kathy Fish says is about “fall[ing] in love with the problem, not the solution”, has seen research and development costs reduced by 25-50 per cent.

Fish acknowledges, however, that effecting the requisite cultural shift continues to be difficult – as might be expected for a large business with set structures, hierarchies, and processes. 

Experimentation: three questions

How can established firms decide whether and how to adopt experimentation into their innovation processes? In our new paper (which you can read here), we offer a checklist of three initial questions to ask:

  1. How will this work in terms of organisational design? Organisations with change-averse innovation cultures may have to establish independent experimentation divisions, with all the associated challenges that entails. Others, more amenable to change, may be able to leverage experimentation by facilitating closer integration of traditionally sequestered research and development, and customer-facing functions. 
  2. How will experimentation fit with established organisational structures? Changing the approach to research and development will require some thought about how it can fit with existing job roles, established hierarchies and the distribution of rewards. These issues are particularly pronounced if experimentation is seen as simply one step towards adopting a more holistically entrepreneurial mindset.  
  3. How can the effectiveness of experimentation be measured? It’s important to be able to quantify whether experimentation has proven to be successful after launch. Different organisations might adopt different measures. For example, a combination of the following might be considered: reduction in research and development costs, fewer failed product launches and increased revenue from new product launches. 

There are potential risks associated with experimentation, one of the greatest being that it may foster a sort of conservatism, favouring incrementalism over bold innovations.

The danger here is that organisations focus on what consumers want now, and not what they might want in the future. The famously market research-averse Steve Jobs would probably have his reservations. 

Corporate giants like Proctor & Gamble are starting to pursue innovation through experimentation in the hope of preventing failed product launches, like the Amazon Fire Phone. Learning from startups and adopting this approach holds promise.

However, established firms would benefit from first clarifying the key questions around organisational design, structure, and performance. This will help them to boost, rather than inadvertently hamper, their ability to innovate. 

This article draws on findings from the article “Beyond the Lean Start-Up: Experimentation in Corporate Entrepreneurship and Innovation”, which is forthcoming in Innovation: Organization & Management and is authored by Dr Christian Hampel, Professor Markus Perkmann, and Professor Nelson Phillips (who all are at Imperial College Business School).

Written by


Markus Perkmann

About Markus Perkmann

Professor of Innovation and Entrepreneurship and Academic Director, Imperial MBA
Markus Perkmann is Professor of Innovation and Entrepreneurship. He is the academic director of the Imperial MBA and academic director of the Imperial Enterprise Lab. He also curates the Entrepreneurial Journey, our school's experiential startup programme across all on-site MBA programmes.

His research focuses on the business of science, science and technology entrepreneurship and university technology transfer. He is editor of the journal Innovation: Organization & Management, and his research was published in journals including Academy of Management Journal, Organization Science, Research Policy and MIT Sloan Management Review.

Markus has startup experience stemming from working at Greychip, a London-based startup company founded to create an e-learning platform. He holds a PhD in Sociology from Lancaster University.

You can find the author's full profile, including publications, at their Imperial Professional Web Page