Why do we still have Focus Groups?

[Protoman200X]

From television shows (live action & animated) to electronics, the idea of focus groups seems logical in theory: a company hires a group of randomized citizens (of varying ages and demographics) and gauges their opinions on a product, with said companies want to improve their changes of success by finding out what the average conscious thinks.

But in practice (from my experience being on both sides of the fence from 19 different FG screenings), I've discovered most people show up because they need the cash & rarely express their honest opinions as consumers. I once saw this one guy lazily pressing Option 'A' for every question presented to him, mumbling to himself in boredom when he will get his check. Another example of a nervous young woman who was influenced to follow along what her sister was doing so it would look like she was comfortable with all the decisions she was making. The old idiom of "Everyone has opinions, but they are not all equally valuable" should apply here, as one person asked why the wheels of the prototype sports car were round. (No, I'm not joking about that one. That actually happened.)

I remember talking to Giancarlo Volpe (one of the creators/show-runners and writers on 'Green Lantern: The Animated Series') about his experience when he & Jim Krieg were watching a focus group screening of an episode of GL (with kids ranging from 6-11), & despite the positive response and the kids overall enthusiasm to know more, the moderators were not gentle with giving them unreliable feedback.

So with that in mind, why are Focus Groups even a thing when it's obvious it's a unreliable avenue to gather genuine feedback on what people truly desire, and instead it's only there to appeal to decision makers validation?

 

[Nome]

I run focus groups all the time. We do our best to screen out stinkers through targeted recruiting (Facebook has been a godsend). But it's obvious that not all participants are equal, and we don't weigh low-effort responses equally. As a whole, focus groups are still invaluable in my particular field, since we do them at a point when we can still make major design changes.

 

[Protoman200X]

I run focus groups all the time. We do our best to screen out stinkers through targeted recruiting (Facebook has been a godsend). But it's obvious that not all participants are equal, and we don't weigh low-effort responses equally. As a whole, focus groups are still invaluable in my particular field, since we do them at a point when we can still make major design changes.

Click to shrink...

As a general question, what happens if there's a conflict when someone sticks to their guns on a decision and refuses to take constructive feedback? In my field, we don't always have much leeway to fight some of the more questionable choices from the higher ups, and the decisions that DO stay make the final product suffer in the end.

 

[Nome]

As a general question, what happens if there's a conflict when someone sticks to their guns on a decision and refuses to take constructive feedback? In my field, we don't always have much leeway to fight some of the more questionable choices from the higher ups, and the decisions that DO stay make the final product suffer in the end.

Click to shrink...

For my situation in particular, it's usually me who has the decision power, but I am also the one commissioning the focus groups and running them to get feedback on my own work, and I'd like to think I'm self-aware enough to avoid falling victim to confirmation bias. We generally use focus groups as a way to validate design thinking, but the reality is that if we don't catch issues in the design phase, live data will reveal issues post-release and that's much more costly (and also reflects poorly on us).

 

[Deleted member 46489]

I'm a writer who's interacted with a lot of beta readers for my work. While I know that's not the same thing as focus groups, I'd like to share my experience.

The MOST important thing when using outside feedback is to KNOW your work. And by that I mean you should have a very deep, intimate understanding of what your work is, what effect/emotions/reactions do you intend to produce in the audience and so on. Every story has a secret heart. You need to know the secret heart of your story.

Why's this important? Because it allows you to process feedback. Any feedback that takes you closer to the secret heart of your story is good feedback. Any feedback that takes you away from that isn't good feedback.

Similarly, I'd imagine that focus groups are only helpful if a creator knows the secret heart or core of their creation.

We have tons of examples of creators who don't understand the heart of their creation and follow any and all feedback in a desperate attempt to improve the creation. At the end of the day, the public experience suffers. Dota Underlords, Gwent and other media are stark examples.

There are also creators who reject all critical feedback and refuse to change their work at all in response. Richard Garfield's behaviour with Artifact comes to mind.

And then you have creators who understand how to process feedback and use it well. They get the heart of their work. Like Icefrog with Dota.

 

[DiipuSurotu]

Because it works

 

[Meg Cherry]

You only ever heard about focus groups when they do something wrong (or are blamed for some bad decision that somebody else doesn't want to take the fall for). But when they work properly, focus groups give relevant perspective & insight into how a mass market is going to react to a product.

 

[Protoman200X]

You only ever heard about focus groups when they do something wrong (or are blamed for some bad decision that somebody else doesn't want to take the fall for). But when they work properly, focus groups give relevant perspective & insight into how a mass market is going to react to a product.

Click to shrink...

Such as the case of why the first wave of boomboxes were matt black?

I'm a writer who's interacted with a lot of beta readers for my work. While I know that's not the same thing as focus groups, I'd like to share my experience.

The MOST important thing when using outside feedback is to KNOW your work. And by that I mean you should have a very deep, intimate understanding of what your work is, what effect/emotions/reactions do you intend to produce in the audience and so on. Every story has a secret heart. You need to know the secret heart of your story.

Why's this important? Because it allows you to process feedback. Any feedback that takes you closer to the secret heart of your story is good feedback. Any feedback that takes you away from that isn't good feedback.

Similarly, I'd imagine that focus groups are only helpful if a creator knows the secret heart or core of their creation.

We have tons of examples of creators who don't understand the heart of their creation and follow any and all feedback in a desperate attempt to improve the creation. At the end of the day, the public experience suffers. Dota Underlords, Gwent and other media are stark examples.

There are also creators who reject all critical feedback and refuse to change their work at all in response. Richard Garfield's behaviour with Artifact comes to mind.

And then you have creators who understand how to process feedback and use it well. They get the heart of their work. Like Icefrog with Dota.

Click to shrink...

I'll keep this in mind. Thank you for your constructive input. :3