While launching and then scaling a pilot is a well-established approach to innovation, there is plenty of evidence that shows that even successful pilot projects often don’t lead to successful scaled implementations. Based on the authors’ many years of consulting on large-scale organizational change, they suggest a better approach to scaling successful pilots. Rather than requiring that new teams replicate the pilot exactly, share with them what you’ve learned from the pilot and then challenge them to find their own solutions that could work as well — or better — in their own contexts.
If you are a leader who wants to introduce major change in your organization — a new technology, process, system, product, partnership, or the like — you’ve probably been advised to construct a pilot project. Its purpose, of course, is to reduce the risk of failure across the entire organization by testing the idea in a small, controlled setting so that you can further refine your solution before you roll it out.
However, while launching and then scaling a pilot sounds like a perfectly logical strategy, there is plenty of evidence that shows that even successful pilot projects often don’t lead to successful scaled implementations. Based on our many years of consulting on large-scale organizational change, and our previous work on why good projects fail anyway, we’ve come to understand the pattern of these failures — and a better approach.
Take a nationwide parts distributor that wanted to increase profitability and thought that a new sales tracking system could help. After extensive research, the company organized a pilot project in one district that gave a few sales representatives a chance to use the software for a quarter. Over that time the reps indeed found that they were better able to sell higher margin products, track their best customers, and utilize data more effectively, all of which helped to boost profitability considerably. Encouraged by these results, the company installed the new system on each salesperson’s devices across the country.
Soon, though, they found that some people resisted using the data when it indicated selling unfamiliar products or prioritizing different customers, others found the technology confusing, and others didn’t understand how to really leverage its capabilities. The result was that many of the sales managers told their people to go back to the old way of working, and overall profitability actually decreased.
What happened? First, there are many reasons that a pilot project can look good. The people chosen to participate are often particularly receptive to trying new things, they often feel they’re “special” for being chosen and therefore work particularly hard (a variation of the Hawthorne effect), extra staff provide training and support, managers are incentivized to make the pilot work, and the usual cultural and administrative barriers to change are temporarily suspended.
But during a wide-scale rollout these conditions no longer exist. Instead, everyone is told to follow specific directions to ensure that the new approach is implemented consistently, training and support are spread thin, there is no relief from other goals, and the change is often viewed as just one more requirement on top of everything else.
The problem runs even deeper, however. Take the parts distributor: The premise of its pilot was to prove that all else being equal, a better sales tracking system would improve profitability. This is probably true — except that in the case of a major rollout, “everything else” is never equal. No two people will use the tool or solution in exactly the same way, nor will they necessarily have the same conditions around them. Motivation, skill, comfort with the old way of working, and a host of other factors become as important in achieving the desired outcome as the tool itself. And in many cases, there are multiple organizations or units, comprised of tens, hundreds, and thousands of people that need to work together in new ways, using the new solution, in order to achieve the desired outcome. No wonder that so many major rollouts fail despite a successful pilot.
But there is an alternative path that leaders can take once they have achieved a successful pilot: As part of scaling the solution, create the conditions that allow individuals and teams to adapt the solution to their unique circumstances and make it their own. Doing so requires encouraging collaboration, behavior-change, and innovation.
More specifically: Rather than telling them exactly what to do, challenge a few front-line teams that represent stakeholders in the system to make highly ambitious progress on key outcomes that you expect the solution to contribute to, in an unreasonably short time frame — 100 days or less. Give the teams some guidance about the extent to which they can modify what came out of the pilot, but otherwise let them loose and see what they can do to generate solutions that they own. As you do this, use the power of peer competition to spark even more innovation. Finally, capture the learnings from each of these 100-day “success experiments” so that teams in the next wave of implementation can start with an even richer menu of possibilities.
Scaling Social Good
To show how this works, let’s look at how the leaders of the 100,000 Homes Campaign overcame the problem when they scaled a successful pilot project to combat homelessness in the United States.
The initial pilot was implemented by a group called Community Solutions in New York’s Times Square to find ways of housing individuals who had been left on the streets the longest. The practices in the pilot included novel ideas such as developing a “by-name list” to organize the outreach team’s efforts so that each homeless person would be known by name and supported as an individual with unique needs, vulnerabilities, and strengths. This pilot program achieved an 87% reduction in street homelessness over four years.
The 100,000 Homes Campaign, started in 2010, aimed to apply these and other successful ideas nationally to house 100,000 of the most vulnerable individuals experiencing chronic homelessness in cities across the country in four years.
By January 2012, two years in, however, things weren’t looking so good. As Campaign Director Becky Kanis-Margiotta said in a New York Times article, “We looked at our numbers and we realized we were on track to be the 30,000 homes campaign.” Kanis-Margiotta and her colleagues had imagined that once communities started using the “by-name list” approach and other proven practices, they would house more people more quickly. This was happening, just not rapidly enough.
In early 2012, Kanis-Margiotta and her team shifted their approach to one that depended more heavily on ideas from the communities themselves. They started by helping communities analyze their own data to understand the local monthly housing placement rates needed to meaningfully reduce homelessness in that area. Then, in May 2012, working with Nadim and his colleagues, the Campaign team created a structure to help enrolled communities accelerate their own progress towards these monthly targets. Each community was challenged by leaders of the various Federal agencies that oversee homelessness programs to create its own 100-day experiment of success, using the proven practices from the Times Square pilot as well as any others that community members were passionate to try.
This modified approach started with a group of four communities and continued over the next two years, with waves of four communities at a time, till more than sixty communities participated in what came to be called 100-Day Challenge Bootcamps.
As these waves unfolded, the menu of practices emerging from these experiments got richer, creating more and more building blocks for subsequent experiments of success. The impact trajectory of the initiative accelerated dramatically, and the goal of enrolled cities housing 100,000 chronically homeless individuals was exceeded in mid-2014. (Of course, the problem of homelessness is far from solved, and many other community-based initiatives were inspired by the campaign and the 100-Day Challenge Bootcamps.)
New Ways to Sell
We’ve also seen this approach of customizing pilots work in corporate settings. For example, a major pharmaceutical company* that Ron worked with wanted to increase sales of older products that were still valuable to patients but were no longer actively marketed. A few pilots proved that when a country or regional sales team focused a concerted marketing effort on a targeted selection of these older products, they could generate additional sales. So now the company wanted to roll out this initiative worldwide.
To do that, a team at headquarters used the experience gleaned from the pilots to analyze which products each country should focus on and what marketing techniques they should use. When they went to the German sales team to get the rollout started, however, they were told that the chosen products and marketing plans wouldn’t work in the country, and that their salespeople had their own ways of working with customers.
But instead of forcing the team to proceed as planned, or giving up on the rollout, the team at headquarters challenged the German team to build on the lessons of the pilots and tailor them to their own situation in the next 100 days. With that encouragement, the German team decided to focus on a few different products that they thought would benefit patients in their markets and developed tailored materials and processes that would work better for them.
This customization turned out to be successful for Germany very quickly. So instead of rolling out a common, HQ-developed approach, each of the other country teams in the company was challenged to develop its own unique 100-day experiment for selling older products.
Scaling the process through waves of these country experiments and building a common knowledge base of how to make them work eventually generated close to a billion dollars in additional sales over the course of two years — while building the capacity of sales teams in each country to sustain the gains into the future.
Taking this more generative, customized approach to scaling a pilot isn’t as straightforward or fast as just telling everyone to implement a specific pilot-tested solution or tool – but it gives you a much greater chance of large-scale, durable impact.
*Editor’s note: Some details have been changed.