Review

This book advocated for continuous customer discovery interviews long before Terresa Torres made the idea mainstream. It provides a practical and broad overview of how to do customer research. From identifying assumptions and shaping questions, to recruiting subjects and getting the most out of sessions. This is a great introduction to a subject that all Product Teams should be comfortable with.

You Might Also Like…

Jobs To Be Done

At any moment, I have a short queue of product books I want to read. I try to follow long books with shorter ones, so I don't lose momentum. I naively thought this book would be a quick read, but it isn't. It's insight-dense and concise, making it hard to get through.

andrewclark.co.uk

Book Review and Summary: The Mom Test by Rob Fitzpatrick

Get a comprehensive review and summary of the best-selling book "The Mom Test" by Rob Fitzpatrick, the startup and customer development expert. Learn about the key principles and practices for validating and refining your business ideas and products through customer conversations and feedback.

andrewclark.co.uk

Book Review and Summary: Testing with Humans by Ron Chernow

Get a comprehensive review and summary of the best-selling book "Testing with Humans" by Ron Chernow, the market research and product development expert. Learn about the key principles and practices for conducting effective user testing and using the insights to inform product development.

andrewclark.co.uk

Key Takeaways

The 20% that gave me 80% of the value.

• Every new idea is built upon a stack of assumptions. Challenge your risky assumptions as soon as you can by getting out of the building and talking to users.
• Be a detective. Look for clues that help confirm or deny your assumptions and allow them to influence what you do next. Customer discovery is about gaining deep insight into your customer, or your partners, or your market.
• Do customer discovery continuously. Don’t stop talking directly to customers. Your questions will evolve, but no matter what stage you are in, you’ll usually find that your best insights will come from talking to real people and observing real behaviour
• To learn from your customer - you need to define them first. Creating a product for “everyone” is not actionable or useful. Get more specific. Who has the problem you’re trying to solve?
• Business Assumption Exercise:
1. My target customer will be? How would you describe them?
2. The problem my customer wants to solve is? What do they struggle to do?
3. My customer’s need can be solved with? ‘Your elevator pitch’
4. Why can’t my customer solve this today? What obstacles prevent solving this already?
5. The measurable outcome my customer wants to achieve is?
6. My primary customer acquisition tactic will be? One channel will dominate.
7. My earliest adopter will be?
8. I will make money/ revenue by? One will dominate.
9. My primary competition will be? Think about both direct and indirect competition
10. I will beat my competitors primarily because of? How do you differentiate?
11. My biggest risk to financial viability is?
12. My biggest technical or engineering risk is?
13. What assumptions do we have that, if proven wrong, would cause this business to fail
14. Highlight assumptions that are highly important and uncertain. Not every assumption can be tested effectively through qualitative research.
• Get Stories, Not Speculation. Humans are bad at predicting future behaviour.
• Therefore “Would you buy this product?” is a terrible question.
• Instead → Ask your interview subject to share a story about the past, that way you’ll be anchoring on real behaviour.
• Ask Open-Ended Questions. Don’t talk too much. Get the subject to share openly. DO ask questions that start with ‘who, what, why and how’. DON”T ask questions that start with ‘is, are, would and do you’. Close with ‘What should have I asked you that I didn’t?’
• Willingness to buy is hard to answer with qualitative research.
• Instead try questions like...
• How much do you currently spend to address this problem?
• What budget do you have allocated to this, and who controls it?
• How much would you pay to make this problem go away? (Don’t take answers too literally)
• OR Set up a situation where the subject thinks they are actually buying something.
• Get the most from feedback on Prototypes by separating prototypes from questions about their behaviour. Ask your questions about behaviour and challenges first. You don’t want discussion about product features to poison the conversation.
• Design “Pass/Fail” Tests. Set goals for key questions and track results. Do so ahead of time. Targets can be an educated guesses. Setting a goal forces you to track what is happening. It helps you carefully think through what you are hoping to see, and will make decisions and judgment calls easier when reviewing data.
• Once you know what you want to learn, think about how you might gather insight through observation (not just direct interviews)
• If making a cold approach keep it concise, convenient, drop a name, follow up and practice your if you might have to leave a voicemail
• Get Creative. Where are you customers? When are they likely to have time to talk? How can you convince them to talk?
• Find the Moment of Pain. Connecting with people at the moment of their theoretical pain, can be very illuminating. Thinking about the moment of pain you want to address will lead you to find your consumers and you’ll be able to gather valuable observational research.
• Make Referrals Happen. Use referrals to your advantage. If you want to talk to a doctor, use one to refer others.  Set a goal of walking out of every interview with 2 or 3 new candidates.
• How to ensure an effective session
• Do interviews in person if possible
• Talk to one person at a time
• Add a note taker
• Warm up and keep it human
• Get them to tell a story
• Don’t talk, listen.
• Find out if they’ve hacked a solution
• Parrot back to confirm understanding
• The Truth Curve. You don’t know the truth about your product until people are using and paying for it. BUT don’t jump straight to a live product, that’s a very expensive and slow way to iterate on your idea. Start talking to customers and testing assumptions right away. As you build confidence increase the fidelity of your tests.

It is the customer’s job to explain their behaviour, goals, and challenges. It is the product designer’s job to come up with the best solution.

Deep Summary

Longer form notes, typically condensed, reworded and de-duplicated.

Why Talk to Humans?

Every new idea is built upon a stack of assumptions. Your job is to find the balance between vision and reality. Challenge your risky assumptions as soon as you can. To do that, you’ll need to get “Get out of the building”.

Get out of the building! Steve Blank

This book is one of two. “Testing with Humans” covers experimentation and this book “Talking with Humans” covers the critical elements of customer discovery. How to find interview candidates, structure and conduct effective interviews and synthesise your findings.

Product managers tend to over-obsess about their product to the neglect of other business risks. Get out of your head, get out of the building, and go talk to real human beings. It is far better to discover bad assumptions now, before you have invested a lot.

The Customer Discovery Mindset

• Your role is that of the detective. You’re looking for clues that help confirm or deny your assumptions, and patterns that will help you make better decisions. Those decisions should influence what you do next. Customer discovery is about gaining much deeper insight into your customer, or your partners, or your market.
• Don’t ask people to design your product for you
• Don’t get hung up on statistical significance
• Don’t sell or pitch
• Your goal is to learn before investing in building or scaling a solution
• BUT you should be doing customer discovery continuously. Don’t stop talking directly to customers. Your questions will likely evolve, but no matter what stage you are in, you’ll usually find that your best insights will come from talking to real people and observing real behaviour

1. Who do you want to learn from?

Know your customer

To validate the market you need to have an opinion about who your customer is. Try to identify...

• Your mainstream customer
• Your early adopter (start here, don’t obsess over the mainstream too early). Early adopters feel the pain point more and are more likely to try new things
• Your critical partners (think distribution, fulfilment, etc)

Creating a product for “everyone” is not an actionable or useful. Get more specific .

Who has the problem you are interested in solving?

• Think about their job, state of mind, location, age etc.
• Standard demographics might be useful, or they might be irrelevant. What are the commonalities across your customer base?
• You also want to think about your early adopters.  Why do they matter? Most new products fit alongside a “technology adoption curve,” as illustrated below.

2. What do you want to learn?

Go in prepared

Have a list of questions to hand in your interview guide. Doing so helps you stay organised and appear more professional. Get your most important questions in early. The most important questions relate to your most important risks and assumptions.

Business Assumptions Exercise:

• Try to make your assumptions as concise and specific as possible. You want to be able to confirm you hypothesis through qualitative research and experimentation.
1. My target customer will be? How would you describe them?
2. The problem my customer wants to solve is? What do they struggle to do?
3. My customer’s need can be solved with? ‘Your elevator pitch’
4. Why can’t my customer solve this today? What obstacles prevent solving this already?
5. The measurable outcome my customer wants to achieve is?
6. My primary customer acquisition tactic will be? One channel will dominate.
7. My earliest adopter will be?
8. I will make money/ revenue by? One will dominate.
9. My primary competition will be? Think about both direct and indirect competition
10. I will beat my competitors primarily because of? How do you differentiate?
11. My biggest risk to financial viability is?
12. My biggest technical or engineering risk is?
13. What assumptions do we have that, if proven wrong, would cause this business to fail?  (inc. market size)
• Highlight assumptions that are highly important and uncertain. Not every assumption can be tested effectively through qualitative research.

How to Learn

• Get Stories, Not Speculation. Humans are spectacularly bad at predicting their future behaviour.
• These questions make really bad interview questions as you have to treat those answers with a great deal of skepticism.
• “Would you like this idea?”
• “Would you buy this product?”
• It is more effective to ask your interview subject to share a story about the past, that way you’ll be anchoring on real behaviour.
‣
Example Interview Questions and Format
If you were investing an on-call vet service, your interview questions might look like this...
• Warm up: concise intro on the purpose of the conversation
• Warm up: basic questions about person and pet (name, age, picture)
• Who is your current vet?
• Can you tell me about how you found and chose him/her?
• Please describe the last time you had to take your pet to the vet for a checkup
• Walk me through the process of scheduling a time to visit the vet
• What was frustrating about that experience?
• What did you like about that experience?
• Have you ever had an emergency visit to a vet? if yes, can you describe that experience for me?
• Have you ever thought about changing vets? why / why not?
• Ask Open-Ended Questions. You shouldn’t be talking too much. You want to get the subject to share openly. DO ask questions that start with ‘who, what, why and how’. DON”T ask questions that start with ‘is, are, would and do you’. Close with ‘What should have I asked you that I didn’t?’
• Willingness to buy is hard to answer with qualitative research.
• These are two of the hardest questions to ask in qualitative research
• Will people pay?
• How much will they pay?
• Instead try questions like...
• How much do you currently spend to address this problem?
• What budget do you have allocated to this, and who controls it?
• How much would you pay to make this problem go away? (Don’t take answers too literally)
• You’re better off setting up a situation where the subject thinks they are actually buying something.
• E.g. KickStarter or ask for a letter of intent to buy.  People don’t honestly think about willingness to pay unless they feel like it is a real transaction.
• Getting the most from feedback on Prototypes. You can learn a lot by putting mockups or prototypes in front of people. However, interpret reactions with a degree of skepticism. Separate prototypes from questions about their behaviour. Ask your questions about behaviour and challenges first. You don’t want discussion about product features to poison the conversation.

It is the customer’s job to explain their behaviour, goals, and challenges. It is the product designer’s job to come up with the best solution.

• Design “Pass/Fail” Tests. Customer discovery is largely qualitative, but it helps to take a quantitative mindset. Set goals for key questions and track results. Do so ahead of time. Targets can be an educated guesses. Setting a goal forces you to track what is happening. It helps you carefully think through what you are hoping to see, and will make decisions and judgment calls easier when reviewing data.
• An interview guide is not a script. Follow interesting conversation. Let the guide evolve as you learn more. Always plan, prioritise and prep your questions.
• Observation can be as powerful as questions Once you know what you want to learn, think about how you might gather insight through observation (not just direct interviews). Watching uninfluenced behaviour is powerful and can lead to great insights. How they purchase something, or solve a problem.

3. How will you get to them?

Approaching complete strangers can seem intimidating; but people are often willing to help out. Especially if you are working on a topic that interests them and you approach them nicely and professionally. 

3 Rules for Recruiting Subjects

• Try to get one degree of separation away (avoid friends and family)
• Be creative (and don’t expect people to come to you)
• Fish where the fish are (and not where they are not)

How to find your customer

• Get Creative. Where are you customers? When are they likely to have time to talk? How can you convince them to talk?
• Find the Moment of Pain. Connecting with people at the moment of their theoretical pain, can be very illuminating. Thinking about the moment of pain you want to address will lead you to find your consumers and you’ll be able to gather valuable observational research.
• Make Referrals Happen. Use referrals to your advantage. If you want to talk to a doctor, use one to refer others.  Use referrals as much as possible. Set a goal of walking out of every interview with 2 or 3 new candidates. Ask the person being interviewed if they know others who face the problem you are trying to solve. If they feel like you have respected their time, they will often be willing to introduce you to others.
• Conferences & Events can be an amazing recruiting ground, they bring a group of people with shared interests into one place. Ask people for their time, but for later, after the conference or event. Get their details and follow up shortly after the conference. Send a short email to remind them where you met, and ask for a conversation. 
• Enterprise Customers are harder to find. You need laser-like targeting. In addition to conferences, LinkedIn can be extremely useful.  If you have hypotheses on the titles of the people you are seeking, run searches on LinkedIn.  You might be able to get to them through a referral over LinkedIn, or you might need to cold call them through their company’s main phone number.
• Advice vs Selling. Asking for advice will help you gain access. “My name is x and [dropped name] told me you were one of the smartest people in the industry and you had really valuable advice to offer. I’m not trying to sell you anything, but was hoping to get 20 minutes of your time.”
• or create a blog focused on your problem and ask if you can interview people for an article.
• Benefitting from Gatekeepers Try calling the CEO’s office. They might give you a contact, who’ll likely follow up if you mention the CEO’s office.
• Students and Researchers ...if you are a student or researcher, say so. As an extra incentive, you might also offer to share the results of your research with your interview subjects.
• Fish in the Sea. When I say fish where the fish are, it is really important to remember the flip side to that statement: don’t fish where the fish are not.  If a method isn’t working, try something new.
• Finding bored people stuck in line is a tremendous hack.
• Online Forms & Landing Pages. Can help build up a list of people to contact. 3 step funnels. Call to action → Price Choice → Request for an email. We tracked the conversion metrics carefully and used the emails to schedule interviews.
• Guidelines for making a cold approach:
1. Keep things concise
2. Keep things convenient
3. Name drop when you can
4. Follow up if you do’t hear an answer, but don’t be annoying
5. If leaving a voicemail, practice first

4. How can you ensure an effective session?

People find the qualitative part of customer discovery hard for two reasons. Talking to strangers is intimidating. Our instincts on how to do it are often wrong. These tips can help.

• Do Your Interviews In Person In person is best. Reading body language and building rapport is easier. Much of human communication is non-verbal. Video conferencing is second best, at least you can still read facial expressions. Phone calls should be your method of last resort. Avoid using email or anything text based.
• Talk to One Person at a Time. It is useful to have a second person on your side quietly taking notes. Avoid focus groups: To avoid group think and so you can go deeper and focus on one person’s stories, and drill into areas of interest (that’s really hard when you are juggling multiple people.
• Adding a Note Taker lets you to stay in the moment. You’ll be able to focus on the topics, the body language, and where to take the conversation. If you can’t have a note taker ask if you can record the session. Else write your notes up ASAP afterwards.
• Start With a Warm Up & Keep It Human. Concisely explain why you are there, and thank them for the time. Launch into things with one or two easy warm up questions. E.g. where are you from? how long have you worked here? Have a list of questions but be in the moment, and make the subject feel like you’re listening to them.
• Disarm Your Own Biases We have an amazing ability to hear what we want to (confirmation bias). Be prepared to hear things that you might not want to hear.
• Get Them to Tell a Story Humans are terrible at predicting their own behaviour. Get people telling stories about how they experienced a problem area in the past.  Find out if they have tried to solve the problem. 
• What triggered their search for a solution?
• How did they look for a solution?
• What did they think the solution would do, before they tried it?
• How did that particular solution work out?

And if they are struggling to remember specifics, help them set the scene of their story:

• What part of the year or time of day?
• Were you with anyone?
• If asking about a purchase experience: “Imagine you are filming the documentary of your life. Pretend you are filming the scene, watching the actor playing you. At this moment, what is their emotion, what are they feeling?
• Look for Solution Hacks A strong indicator that the market needs a better solution is when people are not just accepting their frustration with a particular problem, but they are trying to solve it.
• Understanding Priority To try a new product, pain needs to be acute enough that they will change their behaviour, take a risk, and even pay for it. If you see evidence that someone actually has a problem, it is worth asking where it ranks in their list of things to solve. Is it their #1 pain, or something too low in priority to warrant attention or budget
• Listen, Don’t Talk. Don’t talk much. Keep questions short and unbiased (don’t lead). Don’t rush to fill the “space” when they customer pauses, because they might be thinking or have more to say. Make sure you are learning, not selling!
• Follow Your Nose and Drill Down. Ask follow up questions and clarifications. Ask for the “why” behind the “what.”
• Parrot Back or Misrepresent to Confirm Try repeating back what the person said. They might correct you or hearing their own thoughts might prompt a second more sophisticated answer. You can misrepresent what they and see if they respond (use this sparingly).
• Do a dry run. If you’re new to customer discover do a dry first. See how your questions feel coming out of your mouth. Get a sense of what it is like to listen carefully and improvise.
• Getting Feedback on Your Product?
1. Separate product feedback from the storytelling part. Dig into stories first, gather feedback second.
2. Disarm their politeness training. Make them feel safe. Ask for brutal honesty. Explain that’s how they can best help.
3. Don’t trust positive feedback about your product. Put people through an actual experience, watch their behaviour and try to get them to open their wallet.
• Know your audience. Mockups in a fashion industry may need to be more polished.

5. How do you make sense of what you learn?

Your goal is not to learn for learning’s sake. Your goal is to make better decisions that increase the odds of success. So how do you translate your observations into decisions?

• Take Good Notes
• At the start of every session take down basic details like: name of subject, name of interviewer, date and time, Type (e.g. in person),Take a photo
• At the start of your notes include a basic descriptive information of the interview subject.
• If tracking many quantitative measures consider using a spreadsheet. Turning your observations into quantifiable metrics is both useful and tricky. Calculating actual metrics helps fight confirmation bias, but also be wary of your statistics becoming facts.
• Dump and Sort. 10 minutes for everyone to write patterns and observations they saw don post its. Sort them into groups. Discuss the patterns as a team and re-review your assumptions or business and decide what to do next.
• Look for Patterns and Apply Judgement. Interviews won’t get you statistically significant data, but you will get insights from patterns. Interpret carefully, because what people say is not always what they do. Use judgement to read between the lines. Read body language, try to understand context and agendas. Trust that you can use human judgement on human connections to make the output useful.
• Don’t abdicate your role as product designer. It is not the job of the customer to design your product. It is yours. You gather information and make decisions. Be an idea filter, not an order-taker.
• Expect False Positives. People want to be helpful and nice, and your brain will want to hear nice things. As you are weighing what you have learned, just keep this in mind.
• The Truth Curve. You don’t know the truth about your product until people are using and paying for it. But that does not mean you should jump straight to a live product, because that is a very expensive and slow way to iterate on your idea. Start talking to customers and testing assumptions right away. As you build confidence increase the fidelity of your tests.
• How many people to talk to? Consumer business (50 to 100 people). B2B can be less. Never stop talking to potential customers, but evolve what you seek to learn. If you’re bored it’s time to change things up and examine different assumptions and risks.
• Lead with vision. These techniques don’t work unless you have a clear idea about how you want to improve the world

6. Conclusion

Qualitative research is a critical tool. Don’t wait until you’ve built or scaled to validate your assumptions. Start now.