-
Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
-
We have made minor adjustments to how the search bar works on ResetEra. You can read about the changes here.
Naughty Dog filmed, livestreamed and analyzed the facial reactions of TLOU2 playtesters (with consent) to fine-tune the narrative
- Thread starter MadLaughter
- Start date
People here thinking they basically went and wrote the whole script based on what focus testers told them lol. At least that what you'd think by some of the reactions here. Focus testing is fairly common, more common than you'd probably think.
That's not what's happening here. Bad research is asking people to tell you what to make. Good research is seeing if your hypothesis works and if it doesn't, knowing why it didn't and how to proceed. It informs the decision making process with data from people who aren't too close to the project to see the whole picture.
Druckmann in this case knew what he wanted the game to express, but did the game actually accomplish that? Did the writing fall flat? Did an animation look bad and make people laugh at something otherwise serious? Was the narrative confusing because the writers wrote the story knowing what would happen next and with knowledge the audience wouldn't possess yet?
All of these are questions research can help answer and explain why without compromising artistic vision.
Additionally a lot of these researchers work on the design teams themselves. The myth that research is somehow a bad thing for the design process is not reflective of modern reality.
Yeah I remember a game commercial showing the reaction of their play testers. I think it was Dead Space at least. I figured the "with consent" was added so people wouldn't jump to conclusions with it being ND and TLOU2.
Something like this, but not this, I couldn't find the video I was looking for. It was a game they wanted to get more of a reaction out of people so they tested on people and tweaked things.
Terrified by 'The Evil Within' (Clean Edit)
How terrifying is The Evil Within? See actual playtest reactions in our new trailer. The Evil Within will now terrify players right in the thick of Halloween...
youtu.be
This game is going to be amazing, isn't it?
I mean...who else does this? Is this a common practice in the gaming industry?
I mean...who else does this? Is this a common practice in the gaming industry?
Did people actually read the article? They're trying to see if the game evokes the emotions they want it to, not changing things to suit everyone and make sure it doesn't offend, or disgust, or whatever too much.
Here's a (likely outdated) list of companies that would be doing this.
It's more common in tech than it is in games, but, the practice is growing. Many games companies will bring in outside agencies to conduct research rather than hire people full time, but we're starting to see more companies like the large publishers and console makers build these teams.
I can't speak for others, but I think there's a difference between attempting to attain feedback (which every good artist does) and trying to perfectly finetune the audience's facial reactions. Trying to study the audience's micro-expressions to precisely calibrate your story is, to me, a poor way to go about making art. For one thing, it's ridiculous (the meaning of many facial expressions are ambiguous, with much of the research on them dubious at best), and second it distracts from what makes art interesting. You don't need to do anything interesting to make a person feel sad, or scared, or angry, or laugh. Instead, interesting art gives people complex emotional and philosophical reactions that are never gonna be revealed in the "clench of a jaw" or the "widening of the eyes." The only point in conducting this level of audience analysis is if you're trying to answer the question "Do we need to punch the viewer harder?" It's for creating art that, like getting punched in the face, is memorable and certainly emotion-producing, but not complex or interesting.
Reading the responses in this thread like
Media is rarely created in a void, because it's not the norm to create the perfect piece of art on your first try. You get feedback, you work on fixing whatever isn't landing the way you intended. What ND are doing sounds extremely useful, especially since games are such a unique medium with player interactivity. I mean, the Grounded: Making of The Last of Us documentary shows how much work and iterations it took in all areas of the first game to reach the excellence of the final product.
I don't know if pandering is the right word here, it's just refinement via feedback. Does that emotional beat you want landing? Is the pacing right? Is there enough tension in this section of the game? Does something need to be added/tightened/removed to get the effect you're wanting?
Before a book is published, many people will read it and give feedback. A script will also through the hands of many people. The feedback for a film, television show, or game will go through even more people, even if the final decision on changes will come down to a select few. What ND is doing sounds like the usual process with making a piece of media, just with very precise technology.
Yes to all of the above.
Media is rarely created in a void, because it's not the norm to create the perfect piece of art on your first try. You get feedback, you work on fixing whatever isn't landing the way you intended. What ND are doing sounds extremely useful, especially since games are such a unique medium with player interactivity. I mean, the Grounded: Making of The Last of Us documentary shows how much work and iterations it took in all areas of the first game to reach the excellence of the final product.
I don't know if pandering is the right word here, it's just refinement via feedback. Does that emotional beat you want landing? Is the pacing right? Is there enough tension in this section of the game? Does something need to be added/tightened/removed to get the effect you're wanting?
Before a book is published, many people will read it and give feedback. A script will also through the hands of many people. The feedback for a film, television show, or game will go through even more people, even if the final decision on changes will come down to a select few. What ND is doing sounds like the usual process with making a piece of media, just with very precise technology.
Yes to all of the above.
'The Last of Us Part II' and Its Crisis-Strewn Path to Release
Videogame developer Naughty Dog was racing to finish the sequel to its blockbuster set in a post-pandemic dystopia. Then it was hit with trolls, hackers—and a real pandemic.www.wired.com
Later in the article:
Maybe this is a common practice in AAA studios, but it's the first I've heard of it. I guess it makes sense to cover all your bases by looking for genuine reactions in addition to written and spoken stuff, which could be embellished or missing info or whatever.
This is super common. EA (as an example) has a few labs setup this way. So do other AAA companies.
When you do user testing with facial capture you can get everything from UI improvements (where are people looking on the screen at what time) to instant reactions (that they wouldn't necessarily tell you in a feedback form).
There is a decent amount of science behind it.
The negative reactions to this are crazy not just because of how extraordinarily common it is, but because people are using it to disparage the artistic value of a Neil Druckmann Naughty Dog game, of all things. Naive at best, bad faith at worst.
How dare they
Also, this seems totally normal and like something a company trying to make a narrative focused game would do?
How? It was with their consent. Seems like a good way to find out if your gameplay and story events are hitting the audience the way you intend them to.
That still happens play testers disliked the ending of TLOU and Neil said fuck it and went with it any way .
Still your going to need feed back on story and gameplay .
As I said, "naive at best, bad faith at worst." Acting like art is all first drafts and impulses -- that refinement and feedback somehow invalidate artistic merit -- is ignorant and reductive. You don't have to like Naughty Dog games to understand that Neil Druckmann isn't trying to create thoughtless people pleasers. So even if he and the team are more informed on what is or isn't landing with large groups of people, it doesn't dictate what the content ultimately is.
See:
100%^^^^
When you seek feedback, you aren't asking someone to tell you how to do it. You're seeing what does or doesn't resonate with certain people, personally assessing why that might be for that specific individual, then doing with that information what you will. Whether you commit to your original choice or work to clarify your message/intentions -- however you choose to do so! -- is entirely up to you.
Last edited:
Good to know. Although I'm not sure they needed to focus test the ending.
At the same time, ME3 would have benefited from that I guess.
I mean, that's kinda industry-standard. Even back then for testing Ori and the Blind Forest, we used Microsofts UR Department. They invited playtesters who played the game while there was a stream of the gameplay, they themselves were filmed and we also got a little controller graphic that'd show us at all times what buttons they were pressing. It's actually really important to film the people, so that you'd see how they'd react to this or that.
This sounds more like Naughty Dog has been behind the times and finally invested a bit into doing proper UR.
I mean, how do you folks think things get tested with a test audience? You watch them play or watch something and you study their reactions and then ask them a bunch of questions. Microsoft even has people with a psychology background in there that'd help you pick the right questions, so you'd get exactly the kinda information you'd want as a developer.
This sounds more like Naughty Dog has been behind the times and finally invested a bit into doing proper UR.
I mean, how do you folks think things get tested with a test audience? You watch them play or watch something and you study their reactions and then ask them a bunch of questions. Microsoft even has people with a psychology background in there that'd help you pick the right questions, so you'd get exactly the kinda information you'd want as a developer.
I'm sorry, I'm confused by your post. Where in the article is said that this is something new for them? They mentioned that they started to apply this to TLOU2 in 2017, not that they haven't done that for any of their previous games.
Doing what you want is how you get the best AND the worst possible results.
Very interesting. I wonder if Japanese studios do the same
Looking at Nintendo's recent games, I'd be insanely surprised if they didn't. I'm sure they do heavy User Research for all their games.