Usability Tests vs Focus Groups

It’s amazing how often people confuse Focus Groups with Usability Testing. Let’s set the record straight. What’s the difference?

Focus Groups

Focus groups are group discussions. You recruit several targeted users, get them in a room together with a moderator, and ask them questions.

Focus groups are wonderful for:

• Learning how customers feel about your products, including their past experiences with them.
• Generating ideas for new products or features (or getting reactions to your ideas).

For example, let’s say Facebook wanted to identify new features their users might like. Holding a series of focus groups would be a great way not only to get feedback on ideas you’re thinking of implementing, but in generating ideas you’d never even thought of.

Focus Groups are best used early in the process, i.e. before development (or re-development).

Pitfalls of focus groups include:

• You only get subjects’ stated opinions. Sometimes, what people say is very different from what they’ll do. (Famously, the Edsel tested very well in focus groups.)
• Participants may influence each other. Too often, one or two opinionated individuals will dominate the group. Other participants – who may have better insights – fear to differ.

It might sound like I’m dissing focus groups, but that isn’t my intent. Focus groups can lead to valuable insights. But they’re often used inappropriately. They are not, for example, the best way to find usability issues on a website.

Usability Tests

Usability tests are totally different. First, they aren’t groups at all; they’re conducted one-on-one.

And you don’t simply ask user’s opinions on your site. Rather, you to observe how people actually use and react to your site.

In a typical test session, a moderator sits beside the user and assigns a series of realistic tasks. The subject is asked to “think out loud” as he completes the tasks. The moderator carefully observes what the user is doing, and can ask follow-up questions, etc. (This is critical, as it helps uncover why users are stumbling, how they feel about the site, etc.)

Obviously, you don’t just do this once. You’d typically run 5 – 10 such tests, and you’d have several rounds at different stages of development.

Advantages of Usability Testing Include:

• It can (and should) be done at any stage of development: from wireframes through to live sites.
• It is the perfect complement to analytics: Analytics tells you what is happening on a website; user testing tells you why.
• Recorded sessions are great for convincing doubtful or reluctant members of the development team that the site has problems that need to be fixed.

Sometimes other methods are more appropriate: Surveys, interviews, card sorts, eye tracking studies, remote testing, automated online tools… But one-on-one usability testing will always be the usability practitioner’s most powerful tool. There’s simply nothing better than actually watching someone use your website, and being able to ask live follow-up questions.

So Why the Confusion?

It’s really funny how often people mix the two things up… even though they are so dramatically different:

• Focus Group:  Get a bunch of people together and ask their opinions on stuff.
• Usability Test:  Give a user something to do and watch him do it.

The confusion is puzzling. But it is so widespread, Steve Krug produced a wonderfully funny little video about it. Check it out: