GDC Talk: How Steam users buy games

[Kyougar]

I recently saw this video mentioned in a thread here and thought a separate thread should be in order.

Empathizing with Steam: How People Shop for Your Game

In this 2020 GDC Virtual Talk, Chris Zukowski explains why he spent months watching people browse Steam shares the results of his research with you.Join the ...

www.youtube.com

The video is a month old but I think the conclusions and viewpoints are worth talking about.
He contacted 15 gamers (6 identified as female) with more than 50 games on Steam (well, he probably contacted more but got 15 participants he included in his study/experiment)

Sure, not everyone shops like those 15 people, this is just a window into the shopping habits of people who buy many games on Steam and use the Steam tools to find those games.
I, myself don't go looking for games to buy on Steam, I am one of the few who gets influenced by Streamers/Yters and have a very niche taste and also know for 95% of the time if I buy that game when I open the storepage. Other will only buy AAA or AA releases and won't ever use the Steam tools to decide if they ever want to buy that game.

About the findings in the Video:

It is in line with several talking points I had for years.

• know your Genre/Audience
• show gameplay in trailers, show UI in screenshots
• customers look for negative reviews first, are an important buying decision
• they look for similarities and not unique experiences (something they can't classify doesn't get their attention, something that is familiar will get looked at
• "this is just like xyz!" isn't a negative, they actively search for games that are like the games they already liked.

he shows how users normally shop when they wishlist or buy a game, what they are looking at, what decision and why they are making it, etc.

some conclusions:

• 0, ZERO people bought games full price (in the last 3-month study where he only sent a questionnaire instead of watching people navigating the steam store live)
• no-one bought game because of twitter mentions/recommendations, nearly no-one bought games because of Discord or Reddit recommendations
• nearly no-one wishlisted games because of twitter, discord or Reddit.
• nearly no-one bought/wishlisted games because of streaming/lets players
• the Reddit stats are especially eye-opening because he recruited all participants on Reddit

the biggest decision-makers?
- Friends and Bundles are the main drivers of wishlisting and buying games

He also talks about how to make wishlisted games into sales and mentions the abandoned-cart problem.
And talks about the phenomenon of forgetting games you wishlisted.
Games that are less frequent on sales are forgotten more often "why did I wishlist the game" was a phrase he often got. Games that are on sale more frequently are remembered longer.

His ending solution/recommendation of sales and bundles isn't great but that's the reality for indie developers.
He also doesn't dive into genre-defining games and no-discount indies, but I understand that he speaks to those tiny indies who sell 500 games and not the Factorio's and FTL'ers.

 

[Soulflarz]

Ig im not normal because I definitely wishlist and buy on recs.

 

[AshenOne]

Yooo i recently watched the whole video last night while in bed.. some kind Era user linked me to it in a thread!

Very nice video explaining what goes on in the mind of the customer when he/she is checking out a steam page.

 

[pswii60]

I've bought full price games and I've wishlisted and gone on to buy a ton of games recommended on this very forum. But I wasn't in the 15.

 

[Feep]

I basically don't give a shit about anything with a sample size n = 15. Not that I'm saying any of his conclusions are necessarily wrong, but there's no reason for me to treat any of this as valid data for an entire market.

 

[spam musubi]

Yeah my Steam usage doesn't match up with this in several ways either. How many people did his "research" entail? 15? If he recruited everyone from reddit, did he really try to correct for his biased sample? Steam has over a hundred million users, many of which probably don't even use reddit. They come from different countries with different economic situations. Reddit tends to have a really particular type of gamer.

 

[QuinchoOsito]

I think people are generally too quick to discount studies with small sample sizes ("only 100 people? that means nothing!"), but I am definitely skeptical of any conclusions drawn from 15 randos from Reddit. I think the "talking points" you listed are probably good points of discussion, but I don't feel that way about the "conclusions". Just my two cents.

 

[eXistor]

15 people is nothing. You can't get any decent info from that small a group. 15k? Now that might tell us something.

 

[Kyougar]

He explains in the video why a sample size of 15 is still viable.

 

[low-G]

I basically don't give a shit about anything with a sample size n = 15. Not that I'm saying any of his conclusions are necessarily wrong, but there's no reason for me to treat any of this as valid data for an entire market.

Click to shrink...

This. Plus self-reported data only. Plus the sampling is probably in question.

I don't think I'm probably a normal buyer, but I'm not sure there's much here worthwhile.

He explains in the video why a sample size of 15 is still viable.

Click to shrink...

Just because it's qualitative variables doesn't make it a valid statistic. It's good (like others said) for discovering some factors for further studies.

 

[spam musubi]

I think people are generally too quick to discount studies with small sample sizes ("only 100 people? that means nothing!"), but I am definitely skeptical of any conclusions drawn from 15 randos from Reddit. I think the "talking points" you listed are probably good points of discussion, but I don't feel that way about the "conclusions". Just my two cents.

Click to shrink...

Indeed. People really don't understand statistics and sample sizes, and sometimes you hear embarrassing responses to seemingly small sample sizes that can be valid. But in this case, the embarrassment should be on part of the presenter. If my sample size was like that, I would just focus on my anectodal observations and not even talk about the statistics.

 

[Rosebud]

I wishlist every game I have a bit of interest, and only look at them again when it's at least -75%.

There are rare exceptions like Disco Elysium, that I got at full price.

 

[Mass Effect]

15 people is nothing. You can't get any decent info from that small a group. 15k? Now that might tell us something.

Click to shrink...

While I agree with you, 15K is a bit excessive. You don't really need a sample size that large. A couple of hundred would be plenty good enough.

 

[Bradford]

it's telling that I saw that the cutoff was "more than 50 games" and my first thought was "that is too small a number".

for context I have over 1800 games on steam alone. I would say I probably have bought around 30% of them at full price, but rarely at launch.

 

[eXistor]

While I agree with you, 15K is a bit excessive. You don't really need a sample size that large. A couple of hundred would be plenty good enough.

Click to shrink...

I was just being arbitrary with that number, but I agree. 1500 is probably more than enough.

 

[Rosebud]

it's telling that I saw that the cutoff was "more than 50 games" and my first thought was "that is too small a number".

for context I have over 1800 games on steam alone. I would say I probably have bought around 30% of them at full price, but rarely at launch.

Click to shrink...

Yeah, I own 1000+. That's why I rarely buy stuff at full price lol

 

[spam musubi]

He explains in the video why a sample size of 15 is still viable.

Click to shrink...

I just googled and found more than 15 reddit posts talking about adding a game they saw there on to their wishlist.

 

[Kyougar]

Just because it's qualitative variables doesn't make it a valid statistic.

Click to shrink...

maybe. But just because it is only 15 people, doesn't mean the data is wrong. All the participants showed that they behaved nearly the same when they shopped for a game on Steam. (they all have 50+ games)
1. Browse
2. Hover
3. click at the store page
4. Looking at the media ("show me gameplay in Trailers, show me UI in screens")
5. looking at the tags
6. skimming the short description ("don't care about story, what is the gameplay about?")
7. skip long description
8. check negative reviews.

 

[firehawk12]

Honestly I feel like the only viable strategy now is to release a game at twice the price you want to sell it for and then keep it on sale at 50% permanently. lol

Even though Steam sales are pretty garbo now, even the illusion of a sale price is enough to at least make people consider buying a game.

 

[Kyougar]

I just googled and found more than 15 reddit posts talking about adding a game they saw there on to their wishlist.

Click to shrink...

did you watch the video or are you just trolling?

 

[Ploid 6.0]

I never wish list games, someone told me the reason people do it, but I still have no use for it. If something is on sale I'll likely find out, or not care enough to keep an eye on it myself. Everybody wishlisting just seem like some marketing dream, where people offer up data for nothing at all. I also have my account opted out of being visible to data collectors and random view, unfortunately it doesn't stop Valve from having all that and using it to recommend stuff like Amazon.

 

[Viale]

it's telling that I saw that the cutoff was "more than 50 games" and my first thought was "that is too small a number".

Click to shrink...

Same. 50 games is ridiculously easy to get just by getting a couple bundles on humble bundle.

 

[fourfourfun]

I find the zero purchases at full price sad, but when Steam regularly and aggressively cuts price, who wouldn't wait around to buy a hatful.

 

[Rosebud]

Honestly I feel like the only viable strategy now is to release a game at twice the price you want to sell it for and then keep it on sale at 50% permanently. lol

Even though Steam sales are pretty garbo now, even the illusion of a sale price is enough to at least make people consider buying a game.

Click to shrink...

Many games are releasing in sale, but I won't bite lol

 

[spam musubi]

did you watch the video or are you just trolling?

Click to shrink...

No, I don't want to watch a video that is an hour long that bases its analysis on extremely shaky ground. You need to take a step back and realize that there is ground between "omg this video is so true" and "lol trolling". I am a statistician by trade, so excuse me if I don't really think his analysis is reasonable. I'm not commenting on the validity of his points. They can be true, but his methodology is still garbage.

 

[-Le Monde-]

I view pc gaming as a way to preserve my fav games. In fact, a lot of the games I have on gig/steam, are the same I bought through the console generations. For that reason, I wait for big discounts, in fact, I've never purchased a game at full price on pc.

 

[SteveWinwood]

he's not doing rigorous statistical analysis here people, the sample size isn't really important

he interviews some people and tells you what he learned, hes not saying it's not anecdotal or even representative

it's fine if you don't care about it or think that's worthless, but I still find it interesting, and most of the behavior I found out I share with them

looking at bad reviews to help understand the overall product and looking for UI in screenshots

I do skim the long description and buy some games at full price though so there's a difference

 

[Gelf]

That is actually pretty close to how I use the store. I do buy some things full price right away but otherwise that's pretty accurate.

 

[low-G]

Holy shit, I forgot that a few years back I actually did a statistical study on Steam sales with a sample size of 13,358 using data mining techniques and found the strongest significant predictor correlating to sales was number of recommendations per unit sold. The game being free was the 2nd most significant, but much, much less significant. Metacritic review score was 3rd most significant.

My conclusion was to never, ever ignore user reviews. Decision tree model was the best correlation (which was very good indeed).

Of course that's just a predictor, and you can't have a ton of recommendations without a lot of owners.

Wait no, I normalized that. Recommendations were a proportional to number of units sold.

Long story short, if you're talking about anything on Steam influencing a game's sales, it's completely the number of recommendations.

 

[dude]

I basically don't give a shit about anything with a sample size n = 15. Not that I'm saying any of his conclusions are necessarily wrong, but there's no reason for me to treat any of this as valid data for an entire market.

Click to shrink...

Basically this. With this small of a sample size, it's not data, it's just assumptions.
Maybe his theories are right, but they're not based on data...

 

[Kyougar]

No, I don't want to watch a video that is an hour long that bases its analysis on extremely shaky ground. You need to take a step back and realize that there is ground between "omg this video is so true" and "lol trolling". I am a statistician by trade, so excuse me if I don't really think his analysis is reasonable. I'm not commenting on the validity of his points. They can be true, but his methodology is still garbage.

Click to shrink...

If you don't have something productive to contribute in this thread except for bashing the sampling size, then I don't know what to tell you.
I already wrote in my opening post that I don't like some of his conclusions and that I am quite different from the buying habits of the participants. The video and the conclusions from it is still worth talking about. The video has so much more to offer than a simple "loooool, sample size"

 

[SteveWinwood]

I would be very interested in a large scale eye tracking study on steam pages while someone goes through their discovery queue. I would be shocked if valve hasn't done something like that.

 

[Kyougar]

I would be very interested in a large scale eye tracking study on steam pages while someone goes through their discovery queue. I would be shocked if valve hasn't done something like that.

Click to shrink...

could be doable with VR.
And it's scary!

 

[Ding Dong Daddy]

Holy shit, I forgot that a few years back I actually did a statistical study on Steam sales with a sample size of 13,358 using data mining techniques and found the strongest significant predictor correlating to sales was number of recommendations per unit sold. The game being free was the 2nd most significant, but much, much less significant. Metacritic review score was 3rd most significant.

My conclusion was to never, ever ignore user reviews. Decision tree model was the best correlation (which was very good indeed).

Of course that's just a predictor, and you can't have a ton of recommendations without a lot of owners.

Wait no, I normalized that. Recommendations were a proportional to number of units sold.

Long story short, if you're talking about anything on Steam influencing a game's sales, it's completely the number of recommendations.

Click to shrink...

That sounds interesting. Could you share this study?

 

[low-G]

That sounds interesting. Could you share this study?

Click to shrink...

It was presented as an academic slideshow. I can strip out some of the identifying (toward myself) information and DM you if you like. But I don't have an accompanying writeup anymore. The focus on the presentation was more about finding a model that identified predictors with good correlation.

 

[spam musubi]

I would be very interested in a large scale eye tracking study on steam pages while someone goes through their discovery queue. I would be shocked if valve hasn't done something like that.

Click to shrink...

You don't necessarily need eye tracking to do that. You can rearrange the page elements and Do an A/B test to measure the impact of different positioning of elements and correlate that to clicks.

 

[SteveWinwood]

You don't necessarily need eye tracking to do that. You can rearrange the page elements and Do an A/B test to measure the impact of different positioning of elements and correlate that to clicks.

Click to shrink...

i don't know much about a/b testing (all my statistical experience is much more traditional) and it always strikes me as possibly the most inefficient way to do anything

if something is fundamentally broken you're either never going to fix it or it's going to be so iterative it would take years. not to mention just going down weird rabbit holes from false positives and other data you don't even know if you're interpreting correctly

in my own personal experience, every site that I've seen claim to be a poster child for the successes of a/b testing has gotten worse and more unusable throughout the years. it also seems to be frequently used in every case of design nowadays whether it's appropriate or not

again though im fairly ignorant of a/b testing so everything I say could probably safely be ignored (and not to mention it's fairly off topic at this point)

 

[texhnolyze]

While the sample is low, it hits so many checklists for me.

• show gameplay in trailers, show UI in screenshots
• customers look for negative reviews first, are an important buying decision
• they look for similarities and not unique experiences (something they can't classify doesn't get their attention, something that is familiar will get looked at
• "this is just like xyz!" isn't a negative, they actively search for games that are like the games they already liked.

Click to shrink...

This is all true for me. I don't need bullshots and narrative trailers, if your store doesn't provide actual gameplay screenshots and video, I probably wouldn't be as interested in your game. I also always look at the general review scores first, if it's not as good as I expected, I'd look for the negative reviews to assess what's wrong with the game. If I can accept the negative points, I might still buy it.

• 0, ZERO people bought games full price (in the last 3-month study where he only sent a questionnaire instead of watching people navigating the steam store live)
• no-one bought game because of twitter mentions/recommendations, nearly no-one bought games because of Discord or Reddit recommendations
• nearly no-one wishlisted games because of twitter, discord or Reddit.
• nearly no-one bought/wishlisted games because of streaming/lets players
• the Reddit stats are especially eye-opening because he recruited all participants on Reddit
• Friends and Bundles are the main drivers of wishlisting and buying games

Click to shrink...

Most of these are true, except the first point. I often buy games at full price, especially for indies and AA games that are cheap to begin with. I also don't pay attention to social media recommendations from influencers, including streamers. The last point is pretty spot on. Friends are just like you, a customer who enjoys good games but probably with limited money to spend on them. So if you see your friends buying and playing certain game on Steam, it might be that the game is good. If your taste aligns with your friends, it'd be even better. Steam community features are created to accommodate this kind of system, they're not just for show.

 

[Dr. Doom]

My only full price purchase on Steam, after 10 years of being on the platform, is Doom Eternal (£50).

I regret it. While a good game, no game is worth spending more than £30 on.

 

[spam musubi]

i don't know much about a/b testing (all my statistical experience is much more traditional) and it always strikes me as possibly the most inefficient way to do anything

if something is fundamentally broken you're either never going to fix it or it's going to be so iterative it would take years. not to mention just going down weird rabbit holes from false positives and other data you don't even know if you're interpreting correctly

in my own personal experience, every site that I've seen claim to be a poster child for the successes of a/b testing has gotten worse and more unusable throughout the years. it also seems to be frequently used in every case of design nowadays whether it's appropriate or not

again though im fairly ignorant of a/b testing so everything I say could probably safely be ignored (and not to mention it's fairly off topic at this point)

Click to shrink...

You need to realize that:
1- you might not be in the majority when it comes to an A/B test result
2- your enjoyment might not have been the primary objective of the test.

 

[lucebuce]

My only full price purchase on Steam, after 10 years of being on the platform, is Doom Eternal (£50).

I regret it. While a good game, no game is worth spending more than £30 on.

Click to shrink...

Hell, I'd suggest taking it a step further.

 

[karnage10]

There is one reason i find it hard to believe "nobody"/the majority does not buy a full priced game. Ther eason being that many of the top sellers are new or pre-order games.
Crusader kings 3 is in the top sellers. Mount and blade 2 is in top sellers and played games. those didnt have any type of major sale so if mostly nobody bought them they wouldn't be on top.

 

[Deleted member 37233]

• nearly no-one bought/wishlisted games because of streaming/lets players

Click to shrink...

That's really surprising for me.

I regret it. While a good game, no game is worth spending more than £30 on.

Click to shrink...

What a terrible take, in my opinion.

 

[Wrexis]

I just sort by Top Selling. It doesn't cover indie or niche titles but forums and review sites keep me updated.

 

[deadman322]

I find the zero purchases at full price sad, but when Steam regularly and aggressively cuts price, who wouldn't wait around to buy a hatful.

Click to shrink...

i thought steam sale suck now and consoles have better sales just as regularly?

 

[texhnolyze]

I find the zero purchases at full price sad, but when Steam regularly and aggressively cuts price, who wouldn't wait around to buy a hatful.

Click to shrink...

It's the publishers, not Steam, that put games on sale and cut their prices.

There is one reason i find it hard to believe "nobody"/the majority does not buy a full priced game. Ther eason being that many of the top sellers are new or pre-order games.
Crusader kings 3 is in the top sellers. Mount and blade 2 is in top sellers and played games. those didnt have any type of major sale so if mostly nobody bought them they wouldn't be on top.

Click to shrink...

True. That's one glaring flaw from this study.

 

[fourfourfun]

It's the publishers, not Steam, that put games on sale and cut their prices.

Click to shrink...

Steam do set the regular events though? After a while, if nobody is buying in great amounts out of the events, then you have to be in them.

I do feel that the economics are shifting though, more emphasis on post-purchase sales to offset the decreasing initial purchase price.

 

[texhnolyze]

Steam do set the regular events though? After a while, if nobody is buying in great amounts out of the events, then you have to be in them.

I do feel that the economics are shifting though, more emphasis on post-purchase sales to offset the decreasing initial purchase price.

Click to shrink...

True, but it's still the publishers who set the prices and discount, they're free to participate or not as well. Also, the discounts aren't always the best in those seasonal events. Sometimes you can get the better deals in regular, weekly sales. They van be tracked in Steamdb.

 

[Kyougar]

There is one reason i find it hard to believe "nobody"/the majority does not buy a full priced game. Ther eason being that many of the top sellers are new or pre-order games.
Crusader kings 3 is in the top sellers. Mount and blade 2 is in top sellers and played games. those didnt have any type of major sale so if mostly nobody bought them they wouldn't be on top.

Click to shrink...

That's an issue with the methodology and the narrow specification of his test. He looked for gamers who are actively buying (indie) games and have more than 50 games already on their account. his last 3-month sampling had the participants buying 8 to 80 games (two were in the 80 range)

The majority of people who buy CK3 and AAA games are most likely not in the group of people who buy a dozen games a month.

You have to remember, that this GDC talk is aimed at indie developers who struggle to sell 1000 copies of a game. Knowing what power-buyers look for in a game is immensely important. Just like other important notes.
- knowing that the first 4 screenshots you add to the game page are the 4 pictures who will be used by Steam for the rolling preview on the frontpage
- that the 4 pictures should show different aspects of the game to tell the customer what the game is about
- knowing which tags should be displayed first and telling the customers through tags what genre the game is. the Tag INDIE is completely nonsense, the customer knows it is an indie

 

[Slayven]

There was a good GDC talk from indy devs showing their anayltics from steam and it was pretty in depth and interesting