Last week, I left my cozy office in Belgium and went to Amsterdam. I and a bunch of people from ING across the world were given one challenge: becoming "experiment designers" in 4 days.
This quick article will not only take you through the intense journey we just went through but also go through a couple of our learnings on the way.
But wait, what is Experiment Design?
According to the Merriam Webster dictionary, experiment design is "a method of research in which a controlled experimental factor is subjected to special treatment for purposes of comparison with a factor kept constant".
Taking that to the business world (or anywhere else, believe me!), the main objective of running an experiment is to test an idea, feature, or product according to a test and learn approach, and there are many nuances to it: they can be digital, analog, and be run in so many different ways. Experiments are also used to reduce risk in the most conservative of endeavors, as they are based on data and aim for an evidence-based decision-making process.
Running experiments is important because we tend to look at business plans as static, and many organizations choose to implement them without understanding their viability. Who hasn't heard about the famous story of a company that takes years to launch a product, and once they do, nobody buys it?
There are also people who believe that business plans are useless, and the "let's just do it" approach is the correct one. However, this approach can also lead to failure: if you don't know what you are looking for, anything will be okay. You have to know which kind of behavior you are aiming at before starting product development.
The point then is to learn as fast as possible and to move forward with as much certainty as one can get.
According to Eric Ries in The Lean Startup,
"The question is not "Can this product be built?". The more pertirnent questions are: "Should this product be built?" and "Can we build a sustainable business around this set of products and services?". To answer those questions, we need a method for systematically breaking down a business plan into its component parts and testing each part empirically. In other words, we need the scientific method. "
What did we do this week?
Getting to the main point of this article: What did we actually get done throughout these 4 days in Amsterdam? If I tell you, you won’t believe it (it’s okay, we didn’t believe it either)!
We started by being provided with a set of personas, problems and potential solutions that were prepared by Service Designers. Even though we had a starting point, we were told we could also change based on what we learned throughout the experiments.
After that, we have done the following:
Mapping out our assumptions and defining the riskiest ones ("Leap of Faith assumptions);Running (at least) 3 analog experiments (interviews based on paper prototypes, for example);Building and launching a landing page;Running a google ads campaign to drive traffic to the page;Building a digital prototype on Marvel and getting feedback from potential customers;Making a decision based on evidence, and presenting our learnings to different stakeholders on Friday evening.
What were the main learnings?
As you can imagine, doing what was mentioned above wasn't easy at all. However, I must say the power of time-boxing helped us. Here are the main learnings I took out of it:
Learning #1: Follow the rules!
Experimenting has its own set of steps: choose your focus, identify your riskiest assumption, design your experiment/metrics, run your experiment and make an evidence-based decision.
These rules are there for a reason: if you design a prototype without a clear focus, or without knowing which metrics you want to check, your learnings will most likely be all over the place. That means your decision to stop, pivot or iterate will probably also be all over the place.
Learning #2: Solve For speed
We all tend to get picky about details, and delay the start of a test because "this color on my landing page is not good enough", or "I could have more time to improve this function". Sometimes, this is not the point! Over-designing and over-thinking can delay the process of learning, and most of the times it will not be essential for the experiment. Make sure to understand what is it that you want to learn, and what is the fastest way to get there.
Learning #3: Document your learnings
We are all too excited to get out of the building and talk to potential customers. Or maybe we are just too busy to sit down and write whatever happened during an experiment. But in the end, your learnings will most likely be forgotten if you don’t document them.
Imagine if you stop working on this project, get sick, leave for a week, and someone needs to take over. Or maybe a couple of important stakeholders drop to your working space and want to hear more about what you're doing. How are you going to do that without documentation? For this reason, make sure to keep the numbers and learnings of each experiment loop.
Example of the documentation process around experiments. You can use it to explain your journey to different stakeholders and to make sure you keep the learnings for the future.
Closing the loop
This week was full of learning, and these are just some of them. It made me re-think about what I was doing, and now I have a lot of improvement points to be taken forward from Monday onwards.
However, the truth is: I did not become an experiment designer in one week, because there is no such thing as a certificate for an experiment designer.
Everyone can design an experiment at any point in time, as long as we keep an explorer mindset and make sure to follow the steps on how to run experiments. And more than that: we can experiment in any aspect of life.
Experimenting can change the way we see the world and also how we build the world around us, moving from a bunch of useless things that nobody wants to solutions that actually make sense and fit a need. I hope that after this, you are also ready to take this mindset forward!
Thanks for reading this far, and special thanks to everyone who made this week remarkable.
If you want to know more about experiments or if you have any feedback, feel free to reach out to me! I am also adding a couple of resources below.
Enjoy!
---------------------------------------------------------------------------------------------------------------
Resources:
Persona Template:https://www.boardofinnovation.com/tools/persona/Assumption Map:https://www.boardofinnovation.com/tools/assumption-mapper/Experiment Canvas: https://designabetterbusiness.tools/tools/experiment-canvasPrototyping Tools: https://www.boardofinnovation.com/staff_picks/favorite-prototyping-tools/Landing Page Builder: https://unbounce.com, https://instapage.comSurveys: Survey MonkeyWayback Machine (to get some inspiration): The Wayback Machine is a digital archive of the web - you can check all previous versions of a website! https://archive.org/web/How to prepare a customer interview: https://www.crazyegg.com/blog/start-talking/
Commentaires