by Bill Aulet
Secondary Research: Material you get from sources other than the potential customer. It is indirect. It is generally not specific to your business. These are the industry and government reports that you can find on the Internet or buy from a source that has already done them. I will refer to this as “top-down” market information.
Qualitative vs. Quantitative Research: Qualitative research is exploratory research that helps you generally understand a topic, and is often conducted through a small number of open-ended in-person interviews. Quantitative research uses a much more structured approach and focuses on gathering specific data to prove or disprove the hypotheses created in the qualitative phase.
Ethnographic Research: Technically defined as the systematic study of people and cultures. In primary market research, it turns out that customer psyche plays an important role in decision making. Sometimes, it is even more important than a rational or economic perspective. However, you can’t ask people directly why they do what they do, because people are bad at explaining their habits. And yet it is essential for you to understand this behavior. Anthropologists have been doing ethnographic research for a long time. They embed themselves in the cultures they are studying and observe closely to gain this information.
The Goal of Primary Market Research
The goal of good primary market research is to understand your customer in all dimensions: rationally, emotionally, economically, socially, culturally, and more. Not only have you walked a mile in your customer’s shoes, but you understand them better than they understand themselves. Good primary market research is not just numbers, and it does not start with numbers, but rather that holistic understanding of the customer—the person. Only then can you solve significant problems or otherwise motivate and provide value to the customer.
It is not the customer’s job to design your new product. That is your job. As Elaine Chen says, “The customer is the expert of their problem, and you are the expert in finding a solution to that problem.” It is your responsibility to systematically get enough unbiased information to design and then refine a new product that will create significant value for the customer, and many others just like him or her.
Good primary market research is a continuous process of generating new insights and then converting them into testable hypotheses. You will then continue to use primary market research to validate or invalidate these hypotheses. You may go through the 24 Steps to launch your product based off primary market research, but in reality, the process never ends. Customers and markets change over time, so what is true one year may be invalid the next. Primary market research is an essential skill set for a great entrepreneurial team to have, and it makes being an entrepreneur a lot more fun, too.
The Five Biggest Obstacles to Good Primary Market Research
The fundamental concept of primary market research is quite simple—go out and observe, listen, and interact with potential customers. That can be very hard for some people (i.e., engineers) and very easy for others. Even if it sounds easy, it is not. Here are some of the pitfalls that you have to be aware of before you start.
Lack of Structured Process: While there needs to be flexibility in how you obtain information, you have to have a process and know the process—and know that it is a process! I will step you through that process here, and there are many other resources available to you that I will reference at the end of this chapter.
Not Properly Executing the Designed Process: No process will work if it is not used, and this is no exception. You cannot come up with the right answers without interacting with others. You must get out and talk, observe, and test hypotheses with real potential customers, which often results in answers you would not have gotten on your own. You must dig and get good initial sources to talk to and then grow your sources. You have to learn the process and execute it with good technique. Good technique has a lot to do with avoiding the biases I cover in the next three points.
Confirmation Bias: Extremely common for all people during research, confirmation bias is when you only see the information that confirms your worldview. You ignore or block out any information that runs counter to your hypothesis. A good way to counter this tendency is to set up criteria beforehand that will confirm or disprove your hypothesis so that you don’t change the metrics for success once you start. Even then, bias can still creep in based on how you structure your questions and surveys, inadvertently prompting the customer to provide the answers you want to hear. You must be neutral and not “lead the witness.”
Selection Bias: The people you interview, whether during qualitative or quantitative research, may not be a good representation of the opinions of the group as a whole. Think about Internet polls that allow everyone to vote without controlling for demographics. The type of people who vote do so in percentages that far exceed their true representation of the broader population. As such they produce wildly inaccurate projections. This is why political candidates like U.S. presidential aspirant Ron Paul, with a very passionate base, have done so well on Internet polls, despite not gaining traction in general elections. You need to really understand what your sample should look like to produce meaningful results and control for variations between people that will affect your results. Sometimes, selection bias can be exacerbated by not clearly defining who your customer is. If your end user is “women” and you are at the mall interviewing or observing potential end users, will a 15-year-old affluent urban female have similar opinions as a 40-year-old middle-class suburban mother of three kids? Think about whether your definition of end user is too broad to achieve reliable research results.1
Social Acceptability Bias: If you engage with family or friends, they will most likely not give you accurate answers because they don’t want to offend you by not liking your ideas. They will be polite rather than brutally honest because they want to keep their social relationship strong. You need brutally honest feedback from unbiased people, so it is best to interact with people who do not have a social connection with you. Similarly, on sensitive topics like race or sexuality, you may have trouble getting honest answers, particularly with in-person or phone interviews as compared to anonymous surveys.
There are other biases such as the IKEA effect (once you build a product, your confirmation bias gets much stronger), giving the more entertaining interview more weight, giving the last interview more consideration, etc. It is good to realize your biases and always be on the lookout for them. Being data-driven is good, but the data can be made to mislead as well, so that is not a sufficient answer. A great way to minimize biases is to have multiple people conducting the research who have been trained on the biases inherent in research, so that multiple perspectives will keep things honest.
Process and Techniques
This is the streamlined process I have used and encouraged others to use. It is not as comprehensive as those used in big companies, especially those market research powerhouses like Procter and Gamble, but you don’t have the resources or time to be able to do it that way. You have to get the biggest bang for the buck, so this is an efficient, simplified process guide.
Make a Plan: Many of you will do what I did the first time I learned about primary market research, which was just going out and start talking to potential customers without a plan. This is not disciplined. Develop a plan. This may sound obvious, but it was not to me at first, and it may not be to you. Without a plan, your potential customers will get frustrated after they talk to you once when you come back with more questions that could have been addressed up front. This customer, a very valuable and precious source of information, will then write you off as disorganized and wasting their time. Please save yourself a lot of suffering and losing some potential great sources by having a thoughtful plan. A general plan will look much like the following from Elaine Chen’s Primary Market Research Primer Guide2:
Write out the goals and objectives of the research, and a detailed description of the research technique.
Define recruitme
nt criteria for interviewees.
Develop a recruitment questionnaire.
Develop supporting content (e.g., discussion guide, landing page, online survey).
Recruit subjects.
Run the research program.
Digest results, next steps.
Of these items, the supporting content is often the most overlooked for its value, particularly the discussion guide. In any in-person or phone interview, a discussion guide is crucial so that you know the key questions you should ask each person, in part for consistency but also to make sure each interview yields all the information you need. It will allow you to have a structured dialogue with the person you are engaging with while maintaining flexibility. Likely, you will constantly update this discussion guide at the beginning, but once you have done sufficient interviews, the changes become much less frequent.
Start with Secondary Research . . . But Not Too Much: Before you go out and start to do primary market research, do some homework so you know at least the basics of the industry. Don’t do so much that you think you are an expert of any kind, but don’t be totally naïve. You will be wasting the time of those you engage with if you don’t know the basics. On the other hand, don’t assume all your secondary research is accurate, because sometimes the most interesting opportunities are where conventional wisdom hasn’t kept up with, or never matched at all, the reality in the marketplace.
Start Qualitative Before Quantitative: The most common mistake is to start with a survey. You have to figure out the right questions to ask in a survey, so surveys are useful later but not up front. And while you’ll start by asking specific questions to make sure their demographic (age, gender, etc.) and psychographic (personality, opinions, lifestyles) characteristics match your target customer group, your research questions should be very open-ended, such as, “Please describe a good day for you. Now describe a bad day.” Another question that some experienced primary market research experts prefer is, “Tell me about the last time you did X.” Always follow up with, “Why?,” “Why not?,” or “You said X. Can you say more?” Get them to tell you real specific examples. These examples or stories are generally very powerful and easy for the customer to tell you about. Don’t add friction by asking for data or too many facts right now. You are probing for those big pain points, the biggest fears and motivators, in their lives. While your interview will be loosely scripted, let the customer take the conversation wherever they want it to go.
Source Candidates to Engage With, and Use Them to Find More Candidates: The best places to find candidates are at the “watering holes” for your target customer. Watering holes are those places where your target customer congregates physically or digitally. For many religions, that is their place of worship. For fitness enthusiasts, it is probably their high-end gym. For women about to get married, it is Pinterest. For others, it might well be an industry group or conference. For baby-boomer health-conscious consumers, it might well be Whole Foods. Find out what the watering holes are for your target customer because that will be a highly efficient place to begin engaging them. The sourcing process requires some creativity, as do all the parts of the process, but here are some ideas to help you source leads and make initial contact: Physical watering holes: While digital watering holes can be convenient, you need to be comfortable physically getting out of your workplace and interact directly with your target customer.
LinkedIn or Facebook groups: You can usually find a group that is tied to your target customer and may even be arranged around the problem you are looking to solve. Today this is most often LinkedIn and Facebook, but for your target customer it might be Instagram, Snapchat, or Twitter. This will also change over time as new virtual discussion forums become popular and the existing ones fall out of favor.
Blogs or online discussions: Similar to LinkedIn/Facebook groups. These digital meeting places are very efficient ways of finding candidates, but respect the rules of the blog or forum or you might get banned from the website.
Industry groups and membership lists: A form of watering hole. Depending on the list, you may get a lower response rate than with some of the above methods.
Ads: You can advertise on Google or Facebook, targeted at specific demographics, for as little as $20 to get the names of interested people and then follow up. And if no one responds, that’s important information as well. To be most effective, you should carefully craft the wording of your ads, or you may spend a lot of money getting a lot of responses from people who are not truly in your target customer group.
Read Publicly Available Information: When you do your secondary research, you will likely run across some names linked to your market. Write those names down and find contact information for them. There are various tools to help you find someone’s e-mail address (one I know of is emailhunter.co, but there are many others as well). When you contact them, you should be ready for rejection or, more likely, no response, but it only takes one to respond and then you are in. Like the camel’s nose in the tent, you can expand from there. You may also want to try engaging with key people on social media like Twitter and build up a relationship that way.
The Last Question: Once you get going, the best way to get candidates is from others in the target customer group. As such, your last question at the end of each interview should be, “Who else do you know that has a similar challenge or opportunity to you that I should speak to? Would you be willing to make an introduction?” Once you find one lead, your goal is to have your list grow by getting multiple new leads from every interview.
Initial Contact: Now that you have your initial list of people to contact (a list that should continually be growing as you do your research), things get real. You have to persuade your contact to give you at least 15 minutes of their time, depending on how many questions you have, and that is not easy if you are working with a “cold lead” where you are making first contact. When you initially contact the candidate, it is good to reference someone they know. That is easier than ever with LinkedIn, which shows you mutual connections for each person you look up. Start with short e-mails or dialogue to build credibility and rapport. Explain that you are doing research on their industry to try to help make it better, including their job. Be completely in inquiry mode and not one bit in sales or advocacy mode at this point. You are not really a company yet, nor do you have a product. If you think you are a company, then that is a problem. Be empathetic to making their job better, because that will show in your interactions. Still, be steeled for lots of rejection. There is copious research about little tips and techniques for interacting with cold leads. For instance, Tina Seelig at Stanford found that if you are physically approaching strangers, it is better to have a female than a male make the interaction.3 At first, you will probably feel as if making these initial contacts will never work, but you have to stay at it and then it will get much better. It is never easy, but it gets much easier the more you do it.
Act Like a Great Journalist: I have not been a journalist but have been so impressed when I have interacted with and observed great journalists in action. They have great active listening techniques to get people to talk. The great ones listen with 150 percent of their attention. They have a positive voice and physical stance when engaging. They are incredibly interested and empathetic. They shake their head and say “yes” a lot and are nonjudgmental. They lean forward and listen intently, and visibly so. They make the person being interviewed feel like the most important person in the world. Oh yes, and make sure to smile and be enthusiastic. Humor can help, but sarcasm is not good. The person must like you before they will open up. A couple of ways to verbally engage are using the person’s name a lot and repeating back to the person the words they’ve said.
If You Can, Have Two People Conduct the Interview: Having two people conduct the interview allows one person to concentrate completely on engaging the interviewee while the others is taking detailed notes and observing the interviewee’s nonverbal signals from another angle.
Make sure to write down not just what the person says, but the nonverbal reactions as well. Having two people in the room also allows you more perspectives when reviewing notes afterward.
Constantly Make Sure You Are Interviewing the Right People: If you interview people who are not in your target customer group, you will get spurious information that will not help you understand your target customer, and in fact may confuse you or lead you astray. Make sure you put together a short recruitment questionnaire (it doesn’t have to be longer than five questions) as part of your research plan that screens out people not in your target customer group. This helps you avoid the aforementioned and ever-present selection bias.
As You Get to Forming Hypotheses, Small “n” Might Well Be Better Than Big “n”: After you have done some qualitative research, you will begin to form hypotheses about ways you might be able to benefit the target customer. But you don’t want to spend forever on qualitative research to test it out, because you don’t have the time. MIT Professor Drazen Prelec expresses it as small “n” (number of candidates) and big “n,” where small “n” is a small enough group size that you can go deep in each interview to understand what is causing the phenomenon you feel is important. A bigger “n” gives you more candidates but less in-depth understanding. In the qualitative stage the smaller “n” is better; then, in the quantitative stage, the larger “n” becomes more relevant to test it out more broadly once you understand the hypothesis in a robust manner.
You Should Be Surprised: As Catherine Tucker points out, the goal of qualitative primary market research is to develop new hypotheses; so if you are not surprised, you are probably not learning anything. That defeats the purpose of the process. It would also be a pretty clear sign of confirmation bias if you never learn anything, and that is not a positive thing.
