The Power of Iteration: Melissajeanperri Highlights GitHub Copilot’s Experimentation Journey

In a recent LinkedIn post, Melissajeanperri discusses the critical role of rapid experimentation and learning from failure in product development, using GitHub Copilot as a prime example. Melissajeanperri highlights insights shared by the CPO at GitHub regarding the initial launch and ongoing success of Copilot, emphasizing that the product’s eventual triumph wasn’t immediately apparent.

Melissajeanperri shares a key takeaway from a compilation episode of the Product Thinking Podcast featuring Mario Rodriguez. According to Melissajeanperri, the CPO revealed that Copilot launched with a 30% acceptance rate, yet it was still considered a significant success by the company.

“The CPO at GitHub told me that Copilot launched with a 30% acceptance rate. He still calls it the best decision they made.”

The Iteration Loop: Embracing Frequent Failure

Melissajeanperri emphasizes that the real magic of Copilot, as explained by Rodriguez, lies not in the underlying model but in the team’s capacity for rapid iteration and learning. Melissajeanperri quotes Rodriguez’s perspective:

“Three experiments a week beats one experiment a week. That was his frame. If you can fail three times, you learn three times.”

This philosophy underscores a product development approach where the frequency of learning, even through failed attempts, is paramount. Melissajeanperri elaborates on how this iterative process allowed the team to explore various user experience modalities. While many of these experiments didn’t yield the desired results, they were instrumental in discovering the winning approach: fill-in-the-middle code suggestions integrated directly into a developer’s workflow.

Avoiding the ‘Build Trap’

Melissajeanperri points out a common pitfall in product development narratives: the tendency to only celebrate the successful outcomes, often overlooking the numerous failed experiments that paved the way. This selective storytelling can lead to what Melissajeanperri terms the ‘build trap’.

The Danger of Stifled Innovation

In Melissajeanperri’s view, the build trap occurs when teams are discouraged or prevented from shipping and learning from versions that do not work. This creates an environment where innovation is stifled, and the organization becomes overly cautious, potentially missing out on crucial learnings that could lead to breakthroughs.

“That is the part most product leaders quietly skip when they tell the origin story of a successful product. We celebrate the version that worked. The build trap is what happens when the team is no longer allowed to ship the versions that don’t.”

Melissajeanperri’s post also mentions that the discussion includes insights from David Bland and Monica Lewis, further enriching the exploration of product thinking and experimentation.

Encouraging a Culture of Experimentation

The core message, as highlighted by Melissajeanperri, is the importance of fostering a culture that not only tolerates but actively encourages experimentation. This means creating psychological safety for teams to try new things, learn from them quickly, and pivot based on data, even if initial attempts are unsuccessful. Melissajeanperri concludes by posing a question to the audience, prompting reflection on their own teams’ experimental practices:

“What is the last experiment your team ran that taught you something even though it did not work?”

This call to action encourages readers to consider the value of learning from failures and to reflect on how their organizations can better embrace a culture of continuous iteration, much like the one that propelled GitHub Copilot to success.