• Light/Dark

Naughty Dog filmed, livestreamed and analyzed the facial reactions of TLOU2 playtesters (with consent) to fine-tune the narrative

MadLaughter

MadLaughter

Avenger
Oct 25, 2017
13,197
www.wired.com

'The Last of Us Part II' and Its Crisis-Strewn Path to Release

Videogame developer Naughty Dog was racing to finish the sequel to its blockbuster set in a post-pandemic dystopia. Then it was hit with trolls, hackers—and a real pandemic.
www.wired.com www.wired.com

rEDU652.png


Later in the article:

BBNeX2V.png


Maybe this is a common practice in AAA studios, but it's the first I've heard of it. I guess it makes sense to cover all your bases by looking for genuine reactions in addition to written and spoken stuff, which could be embellished or missing info or whatever.
 
reclaimerzzzzz

reclaimerzzzzz

Member
Oct 27, 2017
5,147
Love that you added 'with consent' assuming Neil was creeping on playtesters. I'm pretty sure this is common practice; I think I saw this mentioned in a documentary for one of the early entries in the God of war trilogy, I also know they did the same for the lost legacy; from a GDC talk that featured how they designed the elephant ride and its iterations.
Pretty sure they also did the same for Until Dawn.
 
OP
OP
MadLaughter

MadLaughter

Avenger
Oct 25, 2017
13,197
reclaimerzzzzz said:
Love that you added 'with consent' assuming Neil was creeping on playtesters. I'm pretty sure this is common practice; I think I saw this mentioned in a documentary for one of the early entries in the God of war trilogy, I also know they did the same for the lost legacy; from a GDC talk that featured how they designed the elephant ride and its iterations.
Pretty sure they also did the same for Until Dawn.
Click to expand...
Click to shrink...

I added it because without it the title felt a little sensationalist and I just didn't want people to assume Naughty Dog was secretly spying on their testers haha

It's interesting to know that other games have also done the same thing, it feels like something you'd need a pretty considerable budget to be able to do properly, in terms of allocating staff and stuff?
 
Rover_

Rover_

Member
Jun 2, 2020
5,212
common practice, but it goes to show why the game took this long to make as they were willing to fine tune the whole game.
 

FlyStarJay

Member
Jan 7, 2018
429
I was an intern for a start up test company, and they do that. Also they use this biometric thing on their hand to know engagement, if they are bored and all that. And the heart beat
 
sibarraz

sibarraz

Prophet of Regret - One Winged Slayer
Avenger
Oct 27, 2017
18,190
Would be cool if everyone was happy when they saw the Vita, leading to the development of the Vita 2
 
DaleCooper

DaleCooper

Member
Oct 27, 2017
2,853
It's very interesting how technology can shape storytelling to produce the exact desired emotional responses.

It's almost like cheat codes.
 
Minamu

Minamu

Member
Nov 18, 2017
1,905
Sweden
This was first brought into the limelight with Left 4 Dead previews back in the day, but Valve went quite a bit further I believe.
 
Fat4all

Fat4all

Woke up, got a money tag, swears a lot
Member
Oct 25, 2017
94,879
here
is 'without consent' those "YOUR MOM HATES DEAD SPACE 2" recordings 🤔
 
McFly

McFly

Member
Nov 26, 2017
2,745
MadLaughter said:
I added it because without it the title felt a little sensationalist and I just didn't want people to assume Naughty Dog was secretly spying on their testers haha

It's interesting to know that other games have also done the same thing, it feels like something you'd need a pretty considerable budget to be able to do properly, in terms of allocating staff and stuff?
Click to expand...
Click to shrink...
Less sensationalist title would be "Naughty dog playtest their games" because that is what it is. I have taken part in a game test from home where they require you to have a camera to record you playing the game. All companies who make consumer products do this to a certain degree. Drug tests, food testing, movie screening etc.

They gave a talk about it at GDC
 

Deleted member 2595

Account closed at user request
Banned
Oct 25, 2017
5,475
It's just how you craft something special at this point. The bar is so fucking high for everything, you need to drive deeper for detail.
 
Midgarian

Midgarian

Alt Account
Banned
Apr 16, 2020
2,619
Midgar
Square did this with FFVIIR didn't they.

That would explain all of the "THEY KNEW EXACTLY WHAT I WAS THINKING!" moments.
 
jett

jett

Community Resettler
Member
Oct 25, 2017
44,714
That is so bizarre. That even QA would determine the nature of your story, writing and characters. It's test screenings taken to a new level.
 

Nuclear Muffin

Banned
Oct 29, 2017
4,721
This is standard practice at Sony WWS. They focus test the ever living shit out of all of their games.
Every single facet of game and story design in all of their games is driven primarly by focus testing and feedback. It's a science at this point.
 
Deluxera

Deluxera

Member
Mar 13, 2020
2,664
In the God of War documentary they showed exactly that.
It seems to be a standard practice among some studios.
 
ghibli99

ghibli99

Member
Oct 27, 2017
18,053
I think one of the most famous of these usability tests was back in the days of Halo 1. Memory's a bit fuzzy, but I think the story went that one of the players went up to what was a static metal texture that was a wall (but looked like a door), and they sat there trying to figure out how to open it. They got so frustrated that they stopped playing. The wall texture was changed, but it was an eye-opener to the team that it would even remotely be a progression blocker. Usability testing blew up after that since it's meant to focus on the player experience at home vs. a moderated group setting.
 
Tempy

Tempy

Member
Oct 25, 2017
3,335
Deluxera said:
In the God of War documentary they showed exactly that.
It seems to be a standard practice among some studios.
Click to expand...
Click to shrink...

Sony has some testing studios set up for this. Sometimes you can get some great reaction moments. Personally I find the eyetracking tech very useful to see if level design/environment art/lighting/UX are working well together to guide the player through the game.
 
gundamkyoukai

gundamkyoukai

Member
Oct 25, 2017
21,434
jett said:
That is so bizarre. That even QA would determine the nature of your story, writing and characters. It's test screenings taken to a new level.
Click to expand...
Click to shrink...

It's normal at this point .
Playtester have become fans \ editors in a way for games .
For movies it harder to change things but you know that sort of thing happens with TV shows , manga etc etc .
Sometimes you change stuff and other times you don't.
 
StrayDog

StrayDog

Avenger
Jul 14, 2018
2,633
jett said:
That is so bizarre. That even QA would determine the nature of your story, writing and characters. It's test screenings taken to a new level.
Click to expand...
Click to shrink...
If you don't care about the opinion of your consumers you could just do the way you want but if you want to be popular you probable will ask for feedback...
 
Outrun

Outrun

Member
Oct 30, 2017
5,786
I am initially torn on this.

Is it art, if the creator panders to their audience?

I got to think a bit more on this.
 

dep9000

Banned
Mar 31, 2020
5,401
I hate these things. These reactions are not real. They know they're being recorded. They're hamming it up.
 
OP
OP
MadLaughter

MadLaughter

Avenger
Oct 25, 2017
13,197
Outrun said:
I am initially torn on this.

Is it art, if the creator panders to their audience?

I got to think a bit more on this.
Click to expand...
Click to shrink...

My hot take on this is that it'd be a little worse if they were just trying to make everyone -happy-, but the rest of the article talks about how they are trying to bring out emotions of discomfort because that fits the themes they're going for.

Also, some art is high risk, in that there is a huge investment riding on that art, and high-risks come with more caution, I guess.
 
Dalcop

Dalcop

Member
Nov 28, 2017
347
Am I the only one this does nothing for? If you do it, fine, that's your process, but it hardly seems like something to brag about. Not once have I ever heard of changes done due to simple focus audience preferences and thought it was necessary or even for the better. I have issues with the design sensibilities of actual developers, let alone random playtesters.
 
Ominym

Ominym

Member
Oct 25, 2017
1,071
dep9000 said:
I hate these things. These reactions are not real. They know they're being recorded. They're hamming it up.
Click to expand...
Click to shrink...
Any Researcher that knows what they're doing knows about The Hawthorne Effect and ways to work to mitigate it.

Dalcop said:
Am I the only one this does nothing for? If you do it, fine, that's your process, but it hardly seems like something to brag about. Not once have I ever heard of changes done due to simple focus audience preferences and thought it was necessary or even for the better. I have issues with the design sensibilities of actual developers, let alone random playtesters.
Click to expand...
Click to shrink...
This happens a lot more than you think that it does at mature design teams and it definitely positively impacts design decisions. If you think research just consists of "focus groups" I'd encourage you to jump into the other resources I've posted in this thread.
 
Last edited:
TheXbox

TheXbox

Prophet of Truth
Member
Oct 29, 2017
6,607
Sounds like playtesting, although monitoring reactions to story and performance strikes me as incredibly lame.
 
BrassDragon

BrassDragon

Member
Oct 26, 2017
3,154
The Netherlands
When a comedian repeatedly does a set or an acting troupe stages a play night after night, they're constantly finetuning their delivery, lines, blocking and choices based on audience reactions until the bit is pitch perfect

This is just the gaming equivalent of that.
 
Turnabout Sisters

Turnabout Sisters

The Fallen
Oct 25, 2017
2,382
I can't approve of this at all for anything artful. Many of the things I love were originally stone cold boring but I only much later came to realize the genius in it. I'm sure scores of incredible ideas were discarded using this technique, in favor of ones that invoke a more immediate response.

Then again, nothing ND has ever made has ever been artful or thought provoking, so I don't know what I expected. Summer blockbusters have their place too.
 
unholyFarmer

unholyFarmer

Member
Jan 22, 2019
1,375
TheGift said:
I know a lot of movies and games probably do this, but I don't like it. Seems like making a game by formula rather than by artistic direction.
Click to expand...
Click to shrink...
That's the feeling I have with Naughty Dog games since Uncharted 1. After Uncharted 2 and TLOU I decided they are just not for me, but seems their strategy is paying off as there are so many people who love these series and are crazy hyped for TLOU Part II.
 
Dalcop

Dalcop

Member
Nov 28, 2017
347
Ominym said:
Any Researcher that knows what they're doing knows about mature design teams and it definitely positively impacts design decisions. If you think research just consists of "focus groups" I'd encourage you to jump into the other resources I've posted in this thread.
Click to expand...
Click to shrink...
More than nowhere, sure, but there are plenty of studios who don't even evaluate internal feedback to guide design if it's not from the right people, and they certainly don't get to the point where they scrutinize video footage of reactions from external playtesters. It's a difference in approach, but the existence of other teams also using this method doesn't legitimize it as better or discredit it as worse. It's just not something I'm comfortable with when it comes to an artistic medium, at least as it's presented here in this instance. Ideas and design are already diluted enough before we even get to that point. But it sure makes for a widely accepted blockbuster, which is what they were aiming for - so in that sense, it's certainly a success.
 
Saray

Saray

Member
Nov 26, 2018
652
Seems like a strange way to make "art"... kind of boring.
Just imagine Vincent van Gogh altering his paints until people liked it. Not a fan of the idea.
 

FPX

One Winged Slayer
Avenger
Oct 25, 2017
2,276
Ubisoft does the same thing. I went to playtest Watch Dogs Legion a few months ago, even though I wasn't chosen in the end, the paper I signed said there would be a camera on my face as I played to capture my reactions.

I did find it funny that playtesters for a game codenamed "Orwell" would have a camera in their faces.
 
You must log in or register to reply here.