Lesson: Design Research with Erika Hall

Step: #7 Customer Research: How do you perform useful customer research?

I've taken a pretty strong stance against focus groups, and this has generated some feedback from people, especially who work in market research, who say, "Hey, I've used focus groups, and they've been successful." I'll say that's great, because I would love for people to prove me wrong, because what I want everybody to do is conduct effective research and make better products and have better, stronger businesses. That's my goal.

My goal isn't just to say, "Focus groups are terrible." But in general, focus groups are terrible, because they're often conflated with research. When people think of qualitative research or ethnographic research, their mind immediately goes to a room with a two-way mirror and some M&Ms and a leader, asking people how they feel about Pepsi or something like that.

But focus groups teach you nothing about how people behave in the real world, because that's where your product or service has to succeed, out there with one person in front of a computer or in front of a kiosk or in front of a rack of products, and making that decision to do something that helps your business. They're not going to be sitting around a conference table, in an unfamiliar office, talking to people, saying, "Oh, hey. What's the best option here?"

Focus groups are a study in the group dynamic. That's what's going on there. What people say often has a performative aspect, and it goes back to ego and insecurity and all of those things that really affect design and business decisions.

Anything a participant says in a focus group, they're saying because they know that other people in the group are listening, that the moderator is listening, and that there are probably people on the other side of that glass.

So they're going to say something that makes them seem smarter or makes them seem nicer. It's like the movies that people put in their Netflix queue. You put the foreign films. You put the dramas that you think you should watch. You put the documentaries about the environment, and then you rent the stupid comedies. You actually rent the stupid comedies.

But if you ask somebody, "Oh, what movie do you want to watch next," in front of strangers that they want to impress, they'll probably say, "Oh, I'm really interested in Croatian documentaries about pollution." Those are the kind of answers you'll get. That's why I'm so against focus groups.

Even the New York Times, when they write about research, they talk about it in terms of doing focus groups. I'm taking this strong stand because I want to clarify the issues for people, to get people to really think about what they say when they're talking about focus groups.

Focus groups were originally developed by a very brilliant American sociologist named Robert K. Merton, years ago. Even he came to criticize how they were being used, because as he put it, "They're a means to uncovering questions and themes to pursue in research." They are not, themselves, research.

So if you get a group of people together, talking about how they take photos or share photos at Thanksgiving or something like that, you won't get data from that, but you might get, "Oh, somebody said something interesting. So let's do some research around how they and their families looked at photos together after Thanksgiving dinner," but nothing that people say in a focus group I would take as representative of real-world behavior. People will say that they have different attitudes, but attitudes and behavior aren't a straightforward match like that.