How to prepare, perform and analyse the results of a user test.
-------------------------------------------------------------------


![](https://cdn-images-1.medium.com/max/2000/1*qfbdSK7kbXab26AcIVujjg.jpeg)

A few weeks ago we shared a first article on how we build design at [Alan](http://alan.eu?utm_source=blog&utm_campaign=user_testing):  
[“4 designer tricks to launch an online insurance in less than 9 months”](https://blog.alan.eu/4-designers-tricks-to-launch-an-online-insurance-in-less-than-9-months-6f4f2942d84f).

One of the important tool described in the post is user testing. I wanted to go deeper with this practical guide on user testing for start ups!

Keep in mind this is the summary of my experience running tests at Orange, Withings and [Alan](http://alan.eu?utm_source=blog&utm_campaign=user_testing). It’s certainly not perfect, but **it will provide a solid base if you have no experience at all**.

Please give us your feedback and comments!

# Why should you perform user tests?

*   **Test prototypes** to check if your assumptions are correct before investing engineering time.
*   **Test final interface** before releasing it to look for glitches, friction points and optimise it.
*   You can also **get feedback** from users **on things such as branding, logo or value propositions.**
*   It’s a good way to **engage future customers** and start building your brand.

**So now: how to prepare, perform and analyse the results of a user tes**t?

# I. Prepare
![unnamed (1)](https://cdn-images-1.medium.com/max/1600/1*zb0Fgq4NCtvZGz0Bmi9KjQ.png)
Caption: Ready to go?

## 1\. Avoid bias 🤥

You are trying to learn how users use your product in real life. But, like in [quantum physics](https://en.wikipedia.org/wiki/Observer_effect_%28physics%29), in user testing, the observation modifies the observed.

**You need to reduce the bias linked to the test as much as possible.**

First, make sure that you are in the right state of mind:

*   Distance yourself from your product. You can’t take feedback personally.
*   Be open minded, some feedback will contradict your ideas or thinking.
*   Refrain from arguing with the user. Try to understand their perspective.

In order to do that, it is crucial to:

*   Put the participant in a state where they’ll be **experiencing** things and not intellectualising
*   Have testers that are **real users or real potential users** (people suck at “emulating” being other people)
*   As much as possible, have them perform the test in **realistic conditions** (using their own device, in the real physical environment)

**Note:** If you’re evaluating an existing product, the best is to observe real users performing real tasks. At Alan, we sometimes go to client companies and just observe employees subscribing to Alan after receiving an invitation email.

## 2\. Test 5 users

Tests are time consuming, you don’t want to do too much of them, but to have meaningful feedback, you need a sample that is large enough.  
So how many tests should you perform?

> **5 is the answer.**

Why ? This is a practical guide; for theory, go [here](https://www.invisionapp.com/blog/ux-usability-research-testing/), or [here](https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/).

**Keep in mind that if you can’t do 5, 1 is always way better than 0.  
**The more tests you perform, the larger number of potential issues you can spot and the better you can assess the importance of each of them. 5 is just a sweet spot; extra tests have diminishing returns.

## 3\. Take the time ⏳

The actual test may take between a few minutes and an hour.  
**Don’t go beyond 60 minutes**. The sweet spot is about 30 minutes of real action (that’s about as long as someone can really pay attention to something), add 15 minutes before and after for intro, questions etc…

For the whole test session, I usually plan for:

*   0.5 to 1 day to **prepare** the test
*   1 day to **perform** the test
*   0.5 day to **analyse** the feedback and communicate it

If you want the user test to be useful, **plan for time to implement the feedback**. At least a few days, and you may need another test session to make sure you fixed your problems.

## 4\. Where do I find users? 🕵️

If you need fresh users that don’t know your product yet:

*   Use your team’s network and yours.
*   Find them in the streets. Go to the nearest Starbucks and offer coffee for questions. People are very welcoming in general. Present yourself, explain your situation.
*   Write a post on [Medium](https://medium.com/@francais)!  
    If you live in France, are CEO or HR of a company, a freelancer, or an independent worker of any kind, please fill [**this very small form**](https://goo.gl/forms/xAbhTPvr3Xl1WUEf1) and become a tester for Alan 😊
*   Post on **Facebook**, **LinkedIn**, **Twitter**, **Instagram**, asking for 20 min from anyone fitting the description. Ask your team and friends to do the same.
*   If you already have a live product, you can recruit among prospects.
*   Use **Ebay**, **Craigslist**, **Leboncoin**, and offer vouchers against time. Filter the candidates using a survey (build it with [Google Form](https://www.google.fr/intl/fr/forms/about/)).  
    Make sure you survey does not ask leading questions.
*   Last resort: recruiters.  
    Some recruiters are specialised in finding people in a certain target, it costs about 100€/user in Paris. I really prefer the other options though.

If you want to test things that require users that already use your product, then you need to recruit among your user base.

This does not cover 100% of situations. Sometimes, you’ll need to be creative to find users. Keep in mind that when you launch your product, you’ll need to make contact with your users anyway, so take the time to figure it out!

**Should you offer a reward?** 💰

As much as possible, try not to. Rewards create a bias in users’ motivation.  
If you offer a reward, it should not be too important, otherwise, you’ll end up with testers that are only here for the money. They are not representative of your target (usually) and won’t give you candid feedback.

The equivalent of **30 to 60€ (in gift cards for example)** is a fair amount.

## 5\. Keep track of your users in a screener 📑

I do it in spreadsheet [like this one](https://docs.google.com/spreadsheets/d/1GCWaSxL69duuCQ4hdcydshOk1W12Pq0GikdoDoFVKro/edit?usp=sharing):

![](https://cdn-images-1.medium.com/max/1600/1*o0aWdZ9F4FwB9IsUdSij9A.png)
Caption: Keep it simple.

Nothing fancy:

*   Add your leads (one by line) and the info you have on them (contact info, anything relevant to your case)
*   Follow the status in the first column. Scheduled means you have a meeting arranged.

Once scheduled, send a proper calendar invitation to participants.

They are human (that’s the point, ain’t it?), they will forget the appointment, so set a reminder as well.

**Prepare an email with all the informations for the test:**

*   Location
*   Time
*   Your contact info
*   List what they need to have with them (their computer, phones,…)

**Include some warnings:**

It’s better if the user doesn’t try to understand your product before the test.  
**So specifically ask them not to look for information, go to your website or try your product in advance.**Best is when they ignore who the test is for or what it’s about.

**If planned in advance, send a reminder at the beginning of the week to make sure they didn’t forget.**

## 6\. Build a narrative

There are a lot of things you may want to test, but keep 2 things in mind:

*   Your entire test should stay **within 30 to 45 minutes** of actual testing
*   You’ll need to come up with a narrative that **encompasses everything you want to test** while **still making sense to the participant**

For example, for our first test session at Alan we wanted to know, for our company product:

*   Did the users understand key elements of our products by reading the landing page?
*   Did the users figure out how to subscribe?
*   Were the users able to subscribe quickly and without friction?
*   How did the users feel during the process?
*   Once subscribed, were the next steps obvious?
*   Did the users successfully invite their employees?
*   How long did it take to complete the process?

Taking those questions, we define a list of **high level tasks** the user will perform with our product:

*   Discover alan through the homepages
*   Subscribe
*   Invite their employees

Our intro narrative was the following:

> You are not happy with the current health insurance of your company. You are planning to change. You are reading a TechCrunch article about a new Health Insurance. Here is the article.

At this point, we created a fake TechCrunch article about Alan that stated realistically what an article of the kind might say about us, with a link to our test landing page. Make sure you don’t give more information to the participants than the level of knowledge you expect an actual user to have at this point.

If the narrative is natural, you should not have to manage the articulation between the different high level tasks too much. Sometimes, you may need to write narrative for transitions.

In our case, once the participant explored the homepage, he would sometimes naturally go to the subscription process. Sometimes, we’d just guide him with a _“Let’s say you are convinced and want to join Alan. What would you do?”_.

## 7\. Formalise your story into a protocol 👨‍🔬

The protocol will serve as a guide for you during the test.

Write on it:

*   **The script** (obviously), screen by screen
*   **Title each screen** with a name. Whatever nomenclature works for what you are testing
*   **Checklist** everything you are supposed to do, before, during, and after the test (setup the wifi, reset a demo account, offer orange juice, alert an engineer that you may need support)
*   **Particular focus** on some parts of the story

## 8\. Start your protocol with this 👨‍🏫

> "Thank you so much for being here and helping us out.

> When we develop products, at some point we need some fresh views on what we do. That is why you are here.

> A few recommendations before we start:

> I will ask a lot of questions, but you are not being tested, the product is. I’ll just be trying to figure out how you see it.

> As much as possible, speak out loud, anything that goes through your mind, hesitations, questions, your understanding of what is happening, how you feel. We know it ain’t natural, I’ll remind you during the test.

> As much as possible, behave like you would normally. Don’t feel obliged to read everything if you usually don’t, don’t rush reading if you usually would take your time, we’ll wait.

> The product won’t break. Do anything you’d do in a normal situation.

> I won’t answer questions during the test. If there is a technical problem, I will intervene. We’ll answer all your questions at the end.

> The test will last about 30 minutes".

We usually add (because it’s true for  us):

> Everything that happens in the test is fake. We won’t record your data, you won’t sign anything that is real.

> As much as possible, use real informations when asked (real ID number, IBAN, etc…).

## 9\. Do a test test (no it’s not lame…) 😅

Take one person in your team or a friend, use her as a tester and perform the test. Make sure:

*   The narrative and story work
*   Your checklists are complete
*   Measure the duration to make sure you meet **your 30 minute goal**.

# II. Perform

![](https://cdn-images-1.medium.com/max/1600/1*0E79mDhOFWS27VNxq3fqtA.png)
Caption: Don’t forget pastries or candies for your users 🍫

Congrats, you’re ready!  
Now let’s do the test. 💫

## 10\. Keep your user relaxed 💆

As a way to limit the bias of your test (and because you are a nice person), you want to keep your user in a normal state of being. Being in a foreign environment, feeling tested or judged goes against that.

*   Be friendly. Start with a smile, say hi, ask casual questions (How is your day going, did you find our office okay, …).
*   Use warm up questions (What do you do for work, where do you live, …);  
    You can actually usefully use those to build a portrait of your user.  
    **Don’t ask leading questions that might bias your test**.
*   Use non-formal language
*   Let the test go off topic a little, don’t try to control too much
*   Try to perform the test in a room that does not look like an office room too much
*   Propose drinks and snacks, have them on the table

## 11\. Recording? 🎥

_Should you record your test session?_

Reasons behind it are multiple:

*   It helps when you analyse your test results (you can get back to the image if you don’t remember clearly)
*   You can have verbatims out of them
*   Most importantly, **they are proof**.

One huge benefit of the user testing is that it gets the debates inside product teams out of the hypothetical. Theories gets crushed, ideas trashed.  
Some people have a hard time accepting it and may not trust the results of the tests. **The solution is either:**

*   Have people watch the test live using a smoke window or combination of camera and screencasting (time consuming for your team, complicated setup)
*   Have recordings of the tests, get interesting short video samples out of them.

**There are two problems with recordings:**

*   Some participants won’t like it, they may accept to be recorded but it may create a bias in the test. If you can, audio only recording is less intimidating.
*   It takes forever to deal with 6 hours of recording. If you don’t have a dedicated person for user testing, it will never be done.

**My advice:**

*   If you don’t absolutely have to, don’t record.
*   Do a first test session with all the team watching.  
    **Witness a test once is usually enough to convince everyone on the method and trust the results**.
*   Recruit your **assistant for the test from within the team** (more on that later), and have them rotate so everyone in the team sometimes witnesses the user tests.

## 12\. Keep it real 😌

*   It’s better to perform tests at the participant’s place. In some cases, the user will want to look at some paper they have in a drawer or ask a question to someone in their office. It’s super interesting to see.
*   It’s better to perform tests on participants devices. They may need information from another app, browse the internet to find something, send a message to a friend.  
    It also avoids all configuration problems like “it does not scroll the same as on my computer”. It reduces bias and keeps the participant relaxed.

## 13\. A two people team is better 🎎

The first team member drives the test, following the protocol

*   You will give the participant the directions.  
    Be careful not to tell them what to do in detail and just give high level tasks to accomplish: Say “_You now want to subscribe to Alan_”, and not “_click on the subscribe button_”.
*   Refrain from answering the participant’s questions, note them for later.
*   Let the participant try things, make mistakes. Ask open-ended questions to understand what is happening in their heads like “Can you explain out loud what you are doing?”
*   If you have a question that may bias the test, note it for later. You’ll come back to it at the end of the test.

The second team member takes notes. 📝

**On the note-taking side**, you can use several tools, a document, a spreadsheet, a Trello board. Use the same nomenclature you used in the protocol. **Watch for:**

*   Hesitations
*   What the user reads or doesn’t read
*   Bugs
*   Blockages
*   Movements of the mouse
*   Out loud questions

## 14\. Finish with a debrief ☑️

*   Ask the participant their overall take on your product. How do they feel, did they have a good experience, would they use it…
*   If you noted questions for the participant or questions from them during the test, now is the time to ask.
*   Thank them for their help !

# III. Analyse

![](https://cdn-images-1.medium.com/max/1600/1*uNdD69tdgdT81u6zDj1Eaw.png)
Caption: Let’s get to work.

## 15\. Rate the feedback 👍

Create a test report file, go through your test notes and gather the feedback, step by step.

List the problems spotted, their recurrence (how many times was the problem spotted), and add a gravity level. For gravity, I use this grid:

*   **Minor**: The user had a hesitation but quickly found a solution
*   **Medium**: The user struggled but got out on their own
*   **Blocking**: You had to help the participant

Also write on the protocol the interesting things participants may have said at this point, and your understanding of their state of mind.

## 16\. Prioritise 💪

Obviously, problems that are blocking are important, problems that have a high recurrence as well, and both blocking and recurrent are your top priority.

Keep in mind that **all problems spotted** don’t need to be fixed.

A problem that happened only once, even if medium or blocking, **may not need to be fixed**. Note that the problem was spotted, try to find other ways to validate that it was just a punctual issue (watch your analytics for example).

If you had a big problem (blocking and recurrent), fix and run a new test session to make sure you solved the issue.

## 17\. Summarize, publish and take actions 👌

At the beginning of the test report, add a summary of your feelings about the test.

*   Did it go well?
*   How many users succeeded / failed to do the tasks without your help?
*   What was the overall feeling of the participant during the test?

Highlight the good and bad stuff that happened, duplicate the top priority items on top.

For every problem that needs fixing, add an action to take (something to implement, a task for the design team, a copy change to make,…)

Make sure **everyone in your team has access to the test report, the protocol, the screener and the raw notes**. Let them know they can ask questions.

# Wrap up

Thanks for reading this, I hope it’ll help you build a strong, user-proof product.

One other challenge we have not talked about in this post is how to organise user tests on a regular basis, how to recruit participants continuously and how to mix it into your product organisation.

We are still figuring this out, and we may talk about our conclusions in a later post!

Drawing

A practical guide to user testing for startups

[Alan is the **first 100% digital health insurance in France**](http://alan.eu), and the first independent insurer to be regulated **in the last 30 years**.  
You can learn more about us on [Techcrunch](https://techcrunch.com/2016/10/24/alan-wants-to-turn-health-insurance-into-software-as-a-service/).

![1*JU0E22FnicubqIWZ2Ly0fw](//images.ctfassets.net/ovjsyiqhqmlf/2CWG0rA6wkSEmaWoiqmcEI/801b09e13d9f95e1517e9dadb0eeaefe/1_JU0E22FnicubqIWZ2Ly0fw.jpeg)

A year ago, I left **Withings** to join **Alan** as the **“first and only” designer** in a team of 9 people.

Looking back on our **first 9 months**, here are four important lessons we learned, that helped us achieve our goal of launching a fully functioning health insurance company in just 9 months:

🤗 **Be in contact with users all along the way**

🏃 **Start with a Design Sprint**

💪 **Take radical decisions**

👨‍**Build only what is needed and nothing more**

# Be in contact with users all along 🤗

Our goal was to build a product with users in mind and **break with the tradition among health insurance companies of designing products around technical and legal constraints**.

Even so we did not have a user research team, we needed to fully understand the users needs.

**How to build customer knowledge with limited resources?**

![1* lsOVimLEUHXs3m23QSXTQ](//images.ctfassets.net/ovjsyiqhqmlf/48dyEDbgOQa0MAUq0mSMqc/90d03a78daa1ba36740e72974c5cd739/1__lsOVimLEUHXs3m23QSXTQ.jpeg)

## **1\. Limit the number of customer types you will be targeting at first**

For us that meant to solely focus on building a health insurance product, and limit our target market as much as possible. We narrowed our scope to small to medium sized companies of up to 200 employees, and mostly startups.

Even on that limited scope, we still had several different types of customers:

*   **CEO**: for small companies, CEOs choose the insurance and manage employees
*   **Head of HR**: they are professionals of the subject, they have technical questions and go into deeper analysis
*   **Employees**: Our final user! They will interact with us on a regular basis, have a highly variable knowledge of the field

## 2\. Find your users

Recruiting users for user research depends a lot of your type of users.  
Doing that recruiting exercise is also a great training step. At some point, you will have to push your product to your target, so you’d better know how to get in touch.

Depending of the type of product you can:

*   Use your team’s network and yours
*   Find them in the streets; Go to the nearest Starbucks and offer coffee for questions; People are very welcoming in general; Present yourself, explain your situation.
*   Post on **Facebook**, **Linkedin**, **Twitter**, **Instagram**, asking for 20 min to anyone fitting the description. Ask your team and friends to do the same.
*   Use **ebay**, **craigslist**, **leboncoin**, and offer vouchers against time.
*   Last resort: recruiters.  
    Some recruiters are specialised in finding people in a certain target, it costs about 100€/user in Paris. I really prefer the first 3 options though.

Once you have launched and have a large user base, use your own integrated systems like [**Intercom**](https://www.intercom.com/) to recruit and get user’s feedback.

At Alan, given our target users, we used mostly **team’s network** and **Facebook**. It worked great. We are now starting to use our user base for user research as well.

## 3\. Talk to them before doing anything else

Once you know your target, and know how to get in touch with a sample of users, start interviewing some of them before working on any part of your product..

🎯 Your goals will be:

*   **Understand their pain points**
*   **Assess their knowledge of your field**
*   **Get pricing insights**
*   **Understand their process in taking decisions**

For us at Alan:

**CEO**

The CEO’s pain points were : Choosing / managing the health insurance is time consuming, heavy on paper work, legally complex.

They ignored most of the rules surrounding health insurance, felt it shouldn’t be their focus (it was distracting them from running the company). They did not know how to choose or compare. Most of them weren’t very price-sensitive, even if they have different opinions.

They wanted a “no brainer”; “many people recommend it, coverage is good, price is ok”.

**HR**

The HR’s pain points were : Spending time dealing with employees’ insurance problems, onboarding / removing employees, integration to the payroll system. They felt it was repetitive and an unnecessary duplication of informations.

And so on…

![1*42Lz5Tslo14UJEg5pzIg7A](//images.ctfassets.net/ovjsyiqhqmlf/2kvVcHxMCsKq4YyuAke0aE/5fe1d7ffb4ab0f554138caa622ea6f77/1_42Lz5Tslo14UJEg5pzIg7A.jpeg)

Interviewing users is a team effort, everyone should pick people they know (in the target), and ask questions. It needs to become a habit.

Doing this will help you understand which features of your product is central and needs to be worked out perfectly, and ensure that you are building a product that has value.

To give you an idea, our CEO, [Jean-Charles Samuelian](https://medium.com/@jcsamuelian), talked to 30 to 40 CEOs / HR specialists before we started working on the product.  
Afterwards, we performed 5 people tests at three different times on the “onboarding for companies” process only.

## 4\. Test your product even in early stages

We’ll explain how to conduct a proper user testing in a dedicated article.

In the meanwhile, just get a few users, ideally 5.  
Have them try your product with just a small introduction.  
Don’t say more than a final user is supposed to know.  
Talk as little as possible, and let them use it.

*   Test using prototypes to **check if your assumptions are correct**. You can use [inVision](https://www.invisionapp.com/) for instance (we even used Keynote).
*   Do tests on the final interface **before releasing any major piece of your product** to look for glitches, friction points and optimise the copy.
*   Always have **a few days available after the test to implement changes**.  
    Otherwise, it’s useless.
*   You can also get **feedback** from users **on things such as branding, logo or value proposition**.
*   Try to **engage with your users during the test** as partners, **they are your future customers**.

User tests are great way to get the discussion started and users involved with your project.

At **Alan** we have always conducted at least a series of 5 to 10 user tests for every release of any major piece of interface. We made critical decisions based on those.

# Start with a Design Sprint 🏃

![1*CKMtIkAXKGoqie4-AbjaFA](//images.ctfassets.net/ovjsyiqhqmlf/qpuigW77TEOOe68I6Y4y2/87de15935fba60ceb4a7e05a85fbb861/1_CKMtIkAXKGoqie4-AbjaFA.jpeg)

The “**Design Sprints**” methodology has been put together by people with decades of experience in ideation (now working at **Google Ventures**) as a way to help companies find the right solutions to the right problems. As they describe it:

_“The sprint is a five-day process for answering critical business questions through design, prototyping, and testing ideas with customers”_

At **Alan**, the sprint helped us do just that.

But it is also great for other reasons, especially at the beginning of a project because:

*   It helps you build a strong **team spirit** and kickstart your project. During those five days, we learned how to interact with each other, how to build together, and how to deliver something that made sense to all of us.
*   It contains all the basic elements for good product thinking; **Identify key problems**, **imagine solutions**, push solutions through **rapid design and prototyping**, and **user test**.
*   A “design sprint” is a **collection of useful tools for team work** that we’ve been using on a regular basis since then for other purposes, like when [we roadmaped our next year](https://blog.alan.eu/strategic-product-planning-as-a-team-part-1-73b322437ba0).

As a result, it will put you on the right track to think your product long term.

If you want to know more about this method, it’s [here](http://www.gv.com/sprint/).

# Take radical decisions 💪

We were considering designing **100% of the discovery of our product / onboarding process as a conversational interface**. Our first round of tests had left us under the impression that it was really powerful from a user standpoint.

!(https://www.youtube.com/watch?v=VqDglUcqQpU)
Caption: Screencast of our conversational discovery / onboarding prototype

It was also shiny, new, fashionable and very different from the competition.

We loved it.

On the second round of tests, **although the feedback was still positive** and we had successfully solved part of the problems, we realised that **it would not drive a significant impact on conversion** (note that on prototypes, it’s rarely possible to have quantitative data, so this is more of a qualitative, gut feeling perception).

The technical difficulty would have **grown exponentially** along with the complexity of the product (more and more possibilities are difficult to manage conversationally), and it raised interaction challenges (conversational is a very new type of UI, without many established patterns to work on).

We decided that although we loved it and it was interesting, **it was not valuable enough to spend months solving design and technical challenges** around it since it would not be a game changer.

We decided instead to go for a more traditional approach and build simple pages that would present our product and a more traditional step-based onboarding. It allowed us to:

*   Move fast, not spending time solving tech / design challenges on that part of the product
*   Iterate fast, since it was a lot easier to iterate on simple pages than a conversational interface
*   Be innovative on the content and on the flow since we were not exploring totally new technologies

Since we were able to launch just in time and did multiple iterations on that part of the product in the first few months, we clearly made the right choice.

> Radical is not good in itself, but it is important as a team to be able to see through personal desires, cut short and be able to trash a few weeks of work if necessary.

# Build only what is needed and nothing more 🔧

Everyone knows the words “Minimum Viable Product” (MVP).

Few are capable of applying it well, because it is so difficult to understand what “minimum” and “viable” mean. Answer: it really depends on your users and the type of products you are building.

## What it does not mean

**It does not mean build something shitty or ugly; it has to be “viable”, remember ?**

“Viable” has to be taken in conjunction with your targeted users and the type of product your are building. It can be summed up like that:

*   If your user is not in the absolute necessity to use your product, or can find another product that fills the same need very easily, **experience is key to your product and having great UX / UI is MVP**.
*   If you are in a space with no / few competitors and offering something of immediate value to your user, having great UX / UI is not MVP; **you just need an good enough experience** (don’t spend time building expensive custom UI, don’t spend money on pricey design firms to build show off UI).

For example, when the **Sony Walkman launched**, it offered a very poor audio quality compared to discs or radio. But it was not a problem; the walkman was the only way to get audio while moving, and “poor quality” was much better than nothing. **Audio quality was not MVP**.

![1*BFCMGg3hjrCPy8gvZBM8yQ](//images.ctfassets.net/ovjsyiqhqmlf/6PJEG1aKmAia6wsw6KMciw/95703a82a7ae8591b14681c64c1ef6af/1_BFCMGg3hjrCPy8gvZBM8yQ.png)
Caption: [The History of the Walkman — The Verge](https://www.theverge.com/2014/7/1/5861062/sony-walkman-at-35)

Later, as nomad audio became the norm and audio quality a differentiator, audio quality became MVP to all nomad audio products at a certain level of price.

## What it means

It means you need to carefully select what “**minimum viable**” means to your product.

**How ?**

Imagine you are in 6 month from now.  
Your product failed.

**Why did it fail?**

*   List the reasons why it failed, based on your understanding of the space from your user interviews
*   Use that to define what you’ll need and when you’ll need it

Then ask yourself how you can test solutions to all those problems using the **minimum possible amount of resources**. Don’t rush doing code or looking for technical solution, sometimes an email or a phone call is the most efficient to test an assumption.

## How we did it at Alan ? We faked as much as possible.

Everything you can avoid to build and test is great.  
Once again, what you can fake depends on the product you are building.

On that regard, insurance has some interesting characteristics:

*   **We wouldn’t face an explosive growth**  
    Choosing an insurance is a process that takes time, making contact and getting your brand out there does not happen overnight. We wouldn’t go from a 100 to 100.000 users over a month.
*   **Few essential use cases**  
    We have a few use cases that would represent 95% of how our users would interact with us at the beginning
*   **Lots of rare use cases**  
    That’s what users will want to do the other 5% of the time.  
    It does not mean it’s not important! (they are sometimes very important)  
    Just that it wouldn’t be performed on a regular basis.
*   **Low contact rate**  
    Health insurance is not an everyday product.  
    If the **essential use cases** are well taken care of in the UI, contact rate will be very low.

We **fully automated the essential use cases**.

For the rare use cases, we **faked automation.**

![1*vumKDVsJoaPNXioWyYbLYA](//images.ctfassets.net/ovjsyiqhqmlf/1suTo3T3nSoIICw0EgIeKY/acf2ae53c86f64b9c15da3ad5ea4c3b5/1_vumKDVsJoaPNXioWyYbLYA.jpeg)
Caption: Alan’s very first beneficiary dashboard version.

Because we were not waiting for explosive growth at the very beginning and were counting on a low contact rate, we could fake in a nice way. There would be buttons in the interface, like “**add an administrator**”, or “**hospital care**”, or “**remove a user**”, but instead of triggering an API call, it would just **open a chat box with a pre-filled message** (we use **Intercom**, an integrated chat accessible from every page in our webapp).

It allowed us to:

*   Cover 100% of use cases overnight from a user’s perspective
*   Prioritise future development based on quantitative data on which rare use cases were most used
*   Measure the time it took us to deal with the demands manually
*   Better understand the user’s needs;  
    Some use cases had complex ramifications, raising lots of questions on the way we should deal with them. We had the opportunity, talking with users, to understand in depth what the most important use cases were and not do hypothetical designs.

Some features will never be automated, because they are so rarely used it will never be worth the engineering time to automate them.

**It doesn’t matter!  
The experience is great, our response time on chat is below a minute.**⏱

## We only built what we needed when we needed it

Building an insurance is building lots of “blocks”.

By building only portions of the product when needed, we were able to launch very early.

🗓 To give you an idea:

*   **Jun.**  
    We launched a “wait list” page. No logo, no branding.
*   **Aug.**  
    Branded “wait list” page.
*   **Oct. 23th**  
    We got authorization from the regulators to be an insurance
*   **Oct. 24th**  
    We launched the commercial website on the following scope; A user can understand what Alan is, can register their company with limited options
*   **Nov. 15th**  
    We updated on the following scope; An admin user have access to a company dashboard, an admin user can invite company employees to join Alan, an employee gets an email and can register but can’t do more.
*   **Dec. 1st  
    **We cover a small number of beneficiaries. Users can add spouse and kids, send us documents, have access to some advice on health care.
*   **Jan. 1st  
    **Implemented billing to bill December bills. We cover a much larger number of customers (solved most of the problems met by our first wave of users to avoid a large contact rate).

We effectively cut down “launch an insurance” in 3 large sequential launches allowing us to meet important deadlines (companies willing to switch their health coverage for ours needed to do so before Oct 31st), test, and improve step by step.

# Conclusion

After that first year, what is to come?

Alan is doing really good and we are growing very fast.

My next challenge as the “one and only” designer in the team is not to be the “one and only” designer any more. Grow a team that manages to keep the practical knowledge and proximity to the users, as well as maintaining team spirit and collaboration at top level.

Maybe more on that later!

Édouard Wautier

A practical guide to user testing for startups

4 designer’s tricks to launch an online insurance in less than 9 months