September 17, 2020

How to come up with disruptive ideas for your experimentation program


These are the concepts that your competitors are probably too nervous to test – but they could deliver you a significant competitive advantage if they work.

We have a tendency to think incrementally.

We look at what we have already, and see how we can make it a little better.

It’s hard to throw it out, start over and come up with something that may be better.

Since it’s challenging to think creatively, here are three exercises we use at our office:

Invalidating lore is a good place to start.

“Lore” means the anecdotal knowledge or opinions within a company, which have never been tested. The things you do because “that’s the way they’ve always been done”.

Chamath Palihapitiya started Facebook’s growth team.

He attributed the growth of Facebook to their constant focus on “invalidating lore”. He said, “One of the most important things that we did was just invalidate all of the lore… All we did was disprove all of the random anecdotal nonsense that filtered around the company.”

Remember, this was a company that was only a few years old. But that’s one of the reasons Facebook has been so successful – everything can be challenged, and data > opinion. That is perfect for experimentation strategists – we deal in data, so we should be able to challenge assumptions and show evidence.

So ask yourself, “What’s the lore in my company? What do we do because that’s the way it’s always been done?” Try writing down a narrative describing the customer journey in detail – then question each point.

Domino’s Pizza – never afraid to innovate

Take Domino’s Pizza. They’ve been making pizzas for 60 years – there’s a lot of knowledge and experience in the business, but that brings with it plenty of “lore”: a lot of assumptions and “that’s the way it’s always been done”.

So if you were to write a narrative about the customer journey on Domino’s, you might question its free delivery policy.

Of course, free delivery is a good thing for customers. But now Domino’s faces increasing competition from Deliveroo, Just Eat and Uber Eats – all of whom charge for delivery.

This provides a perfect opportunity to experiment. Is free delivery an advantage for Domino’s, or is it an unnecessary expense that customers take for granted? Could you A/B test a delivery charge and see whether customers would abandon their order? Would they be less likely to come back in the future? Or has it become accepted that you pay for food delivery now?

Seek out the lore in your business, then test it. Forget what you’ve done before or what you thought you knew, and focus on what it could be.

Divergent thinking exercises push creativity to the max.

Next is divergent thinking. This is where you answer a question again and again and again… until you start coming up with weird and wonderful answers. Then you go back, and build on these ideas.

This is the approach that Airbnb founder Brian Chesky used to design their “11-star framework”.

You don’t take one step forward from where you are today. You go to where the ideal becomes impossible and then take one step back.

Chesky and the other founders wanted to create the perfect experience. So they started to brainstorm the equivalent of a hotel’s 5-star experience, and extrapolated from there.

(Do listen to Chesky talking about this himself – it’s hard to do it justice second-hand. [Masters of Scale 10:36–13:23])

Like Chesky says, you design the extreme and then come backward. That way, you create something that’s significantly further ahead of where you’re at today.

You don’t take one step forward from where you are today. You go to where the ideal becomes impossible and then take one step back.

Ask: What would lead to a 2x increase, not 2%?

We need to shift our mindset from incremental growth to exponential growth. It sounds obvious, but the following is a mistake I make all the time…

You’re building an experiment backlog, and you start by looking at what’s already there, what it looks like, how it works…

But often, you anchor your ideas to what’s there already, making small changes rather than throwing things out and starting again.

And that’s totally fine most of the time – especially if you’re deliberately working on low-risk experiments.

But if you want to come up with radical ideas, you need to think differently. So ask yourself: What would increase the conversion rate 2x (not 2%)?

Let’s take an example…

You’re optimising a SaaS website and you want to increase sign-ups. You could…

  • Improve the homepage based on user feedback (iterative)
  • Optimise the form based on best practice (iterative)
  • Emphasise Google or Facebook sign up options (innovative)
  • Change from a single-page form to a chatbot style Q&A approach (innovative)

Or could you remove the registration step altogether?

This is what Posterous did, ten years ago. They wanted to get more people to host content on their blogging platform.

Posterous adoption skyrocketed when they removed the need to register in order to post a blog article.

But rather than forcing people to sign up, they just let people upload the content by sending an email. Then they’d create an account automatically.

This is one of the hardest things to do in experimentation – to look at a problem differently and not just test changes you’d make anyway. Posterous exited to Twitter a few years after starting up, FYI.

Now it’s your turn to take these three approaches and apply them to your customer experience, which craves close examination today. Even better, you can try combining them. Use divergent thinking when brainstorming to discover what could generate a 2x uplift.

Let us know how it goes!

An excerpt from this article by Stephen Pavlovich first appeared on, GO Group Digital’s exclusive partner and service provider for the U.K. and Ireland. Reach out to Conversion’s CEO and founder, Stephen, about how he and his team ideate. Or get in touch!

Got Questions?

Reach Out to Us

About the GO Group

The world’s leading experimentation experts build global experimentation programs and solutions for the GO Group. Contact the Group to learn how its experience and international setup can build or revitalize your experimentation program. The GO Group operates at the intersection of consultancy and conversion, enabling its enterprise clients to unlock business growth and value through the power of experimentation.

Learn more about the GO Group at

Copyright © 2019 GO Group Digital. All rights reserved.


Ready to go evolutionary to epic?

reCAPTCHA is required.