Can we 'agilize' game development in regard to the customer?

[plusaflag]

Betas, demos and early-access are all ways used by publishers and developers to gather feedback about their games before launch, giving them some time to typically polish graphics and elements of gameplay. Taken to the extreme, game development could benefit from customer feedback during the earliest development (or even planning) stages to accelerate the natural selection of titles that would eventually make it to the market. But you might see here a problem: while agile might be a good framework to better target audiance expectations, continuous communication might result in a loss of interest over time, particularly when it comes to the development of story-driven games; it would be tough to communicate about such a title without at least spoiling its premises.

In a world in which development costs are steadily rising, it might still be a good idea to ensure that the final product actually delivers. So how do we implement this feedback loop? Activision tried to go this route with Battlefield V but the game received a lukewarm reception when it 'came out' last year. Atlus and Square-Enix straight out do the opposite of agile by unveiling their games at times a generation in advance, while not communicating about them during very long periods - and contribute to pre and post-launch frustration among their fans.

Is there a way? I volonutarily let F2P games out of the focus because while they completely embraced customer feedback as the foundation of their success, they are not the model used by most 'bigger' publishers. But maybe they are the future?

 

[Infinity Times Two]

Are you arguing for like public sprint demos every two weeks?

 

[bawjaws]

I just don't see how this is feasible, really. Games, especially AAA titles, are so resource-intensive and need so much to be in place before there's anything to demonstrate, that this doesn't seem like it could work. And companies typically want to keep their cards close to their chests when it comes to developing new products, because it's a competitive old market out there.

 

[Deleted member 426]

I can only imagine this would cause costs to further spiral. Devs are already accountable for every feature shown in a trailer; what happens if they suddenly because accountable to deliver every feature ever considered during development?

I don't really see how this would cut costs at all to be honest, it seems more about quality than cost.

 

[UCBooties]

We already have early access, which is probably the most agile development model possible for game development.

 

[Fatmanp]

Funny, I thought most betas and early access trials were used as a way to sucker in pre-orders these days?

 

[Platy]

Making demos every few months will only increase the cost of everything as every demo is like a mini release, with lots of its Q&A and everything related to it

I am personaly in favor of trying to say "hey devs, no need for 4k or voice acting" as a way to decrease loss and development costs but nobody here would join me =P

 

[VAD]

Isn't what Fortnite and other gaas are already doing?

 

[Com_Raven]

In an act of shameless self-promotion, I would like to point what we at Ubisoft Blue Byte have done with the Anno Union program and website for Anno 1800:

Announced roughly 1 1/2 years prior to release, weekly dev blogs both about new features and behind the scenes development insights, frequent lengthy playtests for select community members (which lead to some massive changes in the game during development), votes on in-game content and constant communication via channels like Discord.

Would I recommend doing something like this for every game? Definitely not. I don't think there is a best practice that works for everyone. In the end, every studio must decide what they think is the best approach for their game and community.

 

[butzopower]

I think GaaS exists because the game industry is applying LEAN methodologies and agile practices.

 

[Jessie]

Are you arguing for like public sprint demos every two weeks?

Click to shrink...

Sounds like Star Citizen tbh

 

[Deleted member 984]

That's what early access is for, it's going to be developed around community opinion.

I wish game developers would stop trying appease the whims and desires of players and just make the game they want.

 

[SpotAnime]

I think GaaS exists because the game industry is applying LEAN methodologies and agile practices.

Click to shrink...

You hit the nail right on the head. And it's not always received well, see Sea of Thieves. They basically delivered an MVP (minimal viable product) to a) hit a delivery date, and b) seize opportunity cost. We all know how well this turned out - people complained of a game that was the bare minimum and unfinished, requiring multiple iterations to give it substance and value.

All those product roadmaps are basically Planning Iterations - we see the features being delivered on a regular (monthly, quarterly) interval.

 

[dodmaster]

I think game production relies on a core artistic vision that doesn't necessarily translate to fulfilling 'customer requirements'. Especially if those customers are 'gamers'.

 

[KDR_11k]

Early Access pretty much does that. I think it may slow down development a lot though since you gotta keep the game balanced and stuff while it's nowhere near done and also avoid savegame incompatibility. Sure, Agile means you can always ship something but it gets annoying when you do microadjustments on things that are not in their final place and will require readjusting later on anyway.

 

[LuckyLocke]

It's already happening. Look at early access, and maybe more subtly, games as a service. The developers build the core first, then iterate on the game based on feedback. It's pretty much the agile way.

 

[plusaflag]

Funny, I thought most betas and early access trials were used as a way to sucker in pre-orders these days?

Click to shrink...

Yes, that's one way of seeing things! Or to be more precise, they are currently one of the last steps to maximizing pre-orders.

Are you arguing for like public sprint demos every two weeks?

Click to shrink...

No and as I and other suggested, I don't think that it is possible. I am certain however that the development roadmap of a title like Final Fantasy VII Remake is flawed and outdated. The question is more 'what could replace the status quo'.

Early Access pretty much does that. I think it may slow down development a lot though since you gotta keep the game balanced and stuff while it's nowhere near done and also avoid savegame incompatibility. Sure, Agile means you can always ship something but it gets annoying when you do microadjustments on things that are not in their final place and will require readjusting later on anyway.

Click to shrink...

That's an interesting remark. How did that go for Subnautica? It was in early access for a very long time and that didn't stop it from becoming a commercial success. Same for its extension, Below Zero. I wonder how these considerations impacted the development roadmap.

I hoped to find an answer in this interview of the game director but the interview took another turn.

 

[DanteMenethil]

You hit the nail right on the head. And it's not always received well, see Sea of Thieves. They basically delivered an MVP (minimal viable product) to a) hit a delivery date, and b) seize opportunity cost. We all know how well this turned out - people complained of a game that was the bare minimum and unfinished, requiring multiple iterations to give it substance and value.

All those product roadmaps are basically Planning Iterations - we see the features being delivered on a regular (monthly, quarterly) interval.

Click to shrink...

It's all about managing expectations. If you do a minimum viable product but not at the minimum viable price expect backlash.

 

[freakybj]

I don't think it makes sense to do this for all games. Multiplayer games that people are looking to play for years - sure. Single player story driven games - no. That would be as stupid as using agile development to make a movie or TV show. Even with multiplayer games there's risk involved with this approach since you can't please everyone. A change you make will please one group and piss another group off at the same time. Developers have to make the game they want to make while incorporating feedback from customers that make their original vision better. Catering to every whim of customers isn't sustainable.

 

[exodus]

In an act of shameless self-promotion, I would like to point what we at Ubisoft Blue Byte have done with the Anno Union program and website for Anno 1800:

Announced roughly 1 1/2 years prior to release, weekly dev blogs both about new features and behind the scenes development insights, frequent lengthy playtests for select community members (which lead to some massive changes in the game during development), votes on in-game content and constant communication via channels like Discord.

Would I recommend doing something like this for every game? Definitely not. I don't think there is a best practice that works for everyone. In the end, every studio must decide what they think is the best approach for their game and community.

Click to shrink...

That's really cool to hear. Seems to have worked out great for you guys, so congratulations. :)

 

[Hybris]

Nioh is a good example of this that's not a GaaS game, and it turned out great. 2 beta periods with core game system adjustments based on feedback from players.

 

[Com_Raven]

That's really cool to hear. Seems to have worked out great for you guys, so congratulations. :)

Click to shrink...

Thank you. I can tell you that it as a big leap and learning experience for everyone involved, as it can be hard for creatives to show of work in a very rough and unpolished state, as we often did. On the flip side, I think getting that frequent sanity check that the game you are building is actually the game that your players out there want can be a huge boost to team morale. It's a win:win in that scenario.

 

[UserDoesNotExist]

I've long had a sneaking suspicion that a misunderstanding of the agile programming process is the reason we've been seeing so many games launch unfinished and/or remain in "beta" for years after they've started charging money.

 

[Khamsinvera]

Kanban master race, losers!

 

[lucebuce]

Activision tried to go this route with Battlefield V

Click to shrink...

uh, I don't think that's right.

 

[Jessie]

I've long had a sneaking suspicion that a misunderstanding of the agile programming process is the reason we've been seeing so many games launch unfinished and/or remain in "beta" for years after they've started charging money.

Click to shrink...

Agile definitely means to release a broken product, make money, and fix later. Maybe. What's a sprint?

 

[Gentlemen]

Running a successful beta requires tremendous overhead. You need to maintain and deploy fully functional beta builds fit for public release. You need a full time team whose entire job is to recruit beta testers, ensure that they're good beta testers, the types who follow directions, submit comprehensible bug reports, provide constructive, carefully-considered feedback and actually engage with the game so that you aren't just throwing your money at players looking to score free game time. You need to involve QA to formalize bug reports, triage a completely separate list of issues to remove duplicates found by internal QA AND prioritize based on what can get done before ship date. On top of this you need community managers to get the word out about new builds, act as preliminary 'customer service' when the beta testers inevitably have complaints about a broken beta, etc. etc. Larger studios ramp up to this level of staffing. Stuffing this type of process 'at the earliest stage of development' sounds like a recipe for never getting anything done and never shipping your game.

Most companies don't have these resources and likely won't materially benefit from that level of investment. You can't just hand-wave 'well just make more public beta tests for your game' like it's this magic bullet to maximizing the positive reception to your products.

 

[Deleted member 203]

This doesn't work for major AAA productions because the vertical slices you see in big presentations are always hacked-together, carefully scripted representations of what the dev THINKS the final game will be like. And they cost a ton of resources to make, so costs and dev time would only increase. A lot of AAA games don't come together until very very late in development.

But as others have said, Early Access is kind of the videogame equivalent of this. But that doesn't work for a game like Uncharted, for obvious reasons.

 

[EvilBoris]

Isn't this essentially what the various early access programs allow?

I think with AAA games it causes problems, not only do you have to deal with damage control from the wider gaming public not being able to understand some of the decision making processes that happen during a protracted development period, you need to maintain marketing hype all the way up to release. By the time the game actually launches, it's already old - and we know that for the most part, most games sell best during their first couple of months on sale.

Then you have the whole pricing aspect, if you make it cheaper during early access, then some users will begrudge paying full price for the completed product.

 

[jon bones]

I don't think we need to make "gamers" feel like they are stakeholders in these projects any more than they already do.

 

[Massive Nights]

Games As A Service is basically agile already. And Early Access exists on a number of platforms.

 

[Gentlemen]

Atlus and Square-Enix straight out do the opposite of agile by unveiling their games at times a generation in advance, while not communicating about them during very long periods - and contribute to pre and post-launch frustration among their fans.

Click to shrink...

Also to comment on this, post-launch disappointment has always been a function of the ability of fans to manage their own expectations. Every company that wants your money will promise you the moon. If you give in to rabid speculative hype based on absolutely nothing but what you feel will make you the happiest about a game and then feel let down that's entirely on you, not the company.

 

[Vintage]

Isn't post-release GaaS already form of agile? Monthly patch releases, balance changes, new, requested features.

If you talk about pre-release, then I guess early access is similar, but that's still releases for the customers.

If you mean no releases at all, then what kind of agile is it if nothing is delivered.

 

[Alek]

That feedback loop already exists, really. It's called rapid iterative playtesting and it occurs privately, internally.

Hundreds, if not thousands of players play these games in private playtests, before the game is released to consumers.

At my work, we run playtests for games from initial concept, all the way to production. For instance, for a narrative game we might test the story concept very early, with slides and pictures, and then later in development this will move to animation, and voice acting.

Why not just release a test version of the game to everyone? There are a few reasons:

• There's a high cost overhead to manage all of those distributions and collect feedback
• Making sense of all that feedback is difficult, surveys take a lot of time to analyse and they're less valuable at providing rich data than in-person methods such as a detailed interview
• That feedback is influenced by marketing trends at the time, which may not be the same as the trends at the time of release
• That feedback is typically skewed by selection bias. For instance, players that sign up for an Halo 5 beta are more likely to be Halo superfans
• Remote playtesting is really poor at answering important early questions such as 'is the game usable', this is where the one-to-one interviews enable developers to understand what players think when they play the game, moment to moment. It's difficult to answer questions like 'is the game fun' until you verify the game is usable and understandable

Obviously there's merits in big data and analytics is one of them. That's why beta's tend to roll out later in development, where game developers are looking for quantitative data from your play experience. Things like, 'how many times are players dying in this part of the map' or 'Which corners of this race track are too difficult for players?' are questions that need answers from a much larger number of data points.

The implementation varies from company to company, depending on their UX maturity.

 

[Elephant]

An agile approach is pretty much represented in Early Access and GAAS games, where they release a base product and add content and new features over time. I haven't seen anything to suggest this doesn't work, but I personally prefer a waterfall method when it comes to entertainment, so that I have the full finished product.

 

[Velocity:Design:Comfort]

Not everything should or can have board room profit maximizing tactics applied to it, especially in an industry built on overwork and no work home balance in general.

 

[Ryder9]

Isn't early access just that?

 

[Siggy]

Gamers don't really approach studios and contract them to develop a game they want. It hardly seems like they are good judges of what they want to begin with given the amount of completely asinine suggestions fans make. Instead, the initial spark for development usually comes from the publisher/developer, more or less. Of course there is market research to determine if there is an audience for something, producers/directors could be influenced by fan opinions, and so on. Nevertheless, the business relationship between developer and audience in gaming is very different from the one agile is meant for.

That said, I do think Nioh greatly benefited from the two big public betas it did and the feedback that gave Team Ninja. There is definitely precedent for iterating on a product with gamers working out well. Still, even if they did a third beta, it would hardly constitute "agilized" and I doubt the quality would've improved beta after beta. As far as I remember, most of the useful feedback came from the very first and not so much from the second, making it seem like just getting some broader feedback than mere kleenex or focus group testing helps a lot without needing to go full agile.

 

[Coi]

Devs needs to stop announcing games so early. It's so stupid.

 

[KINGOFSHIBUYA Ltd.]

Not really, no. There are methods now, and they can cost a lot extra like demos and betas, just to build something that folks wont complain about as if they bought it. There's early access, but even that garners bad public opinion, because the public is conditioned to expect early access and betas not to be much different than final product, in a world where Gmail was in beta for years as one of the most popular email providers.

I don't see it getting any better than it is now.

What really needs to change is just how we make games, regardless of customer feedback. Publisher demands and expectations are increasing, because they believe consumer demands are.

 

[UserDoesNotExist]

Agile definitely means to release a broken product, make money, and fix later. Maybe. What's a sprint?

Click to shrink...

Hence why I stated "misunderstanding of the agile programming process."

 

[plusaflag]

That feedback loop already exists, really. It's called rapid iterative playtesting and it occurs privately, internally.

Hundreds, if not thousands of players play these games in private playtests, before the game is released to consumers.

At my work, we run playtests for games from initial concept, all the way to production. For instance, for a narrative game we might test the story concept very early, with slides and pictures, and then later in development this will move to animation, and voice acting.

Why not just release a test version of the game to everyone? There are a few reasons:

• There's a high cost overhead to manage all of those distributions and collect feedback
• Making sense of all that feedback is difficult, surveys take a lot of time to analyse and they're less valuable at providing rich data than in-person methods such as a detailed interview
• That feedback is influenced by marketing trends at the time, which may not be the same as the trends at the time of release
• That feedback is typically skewed by selection bias. For instance, players that sign up for an Halo 5 beta are more likely to be Halo superfans
• Remote playtesting is really poor at answering important early questions such as 'is the game usable', this is where the one-to-one interviews enable developers to understand what players think when they play the game, moment to moment. It's difficult to answer questions like 'is the game fun' until you verify the game is usable and understandable

Obviously there's merits in big data and analytics is one of them. That's why beta's tend to roll out later in development, where game developers are looking for quantitative data from your play experience. Things like, 'how many times are players dying in this part of the map' or 'Which corners of this race track are too difficult for players?' are questions that need answers from a much larger number of data points.

The implementation varies from company to company, depending on their UX maturity.

Click to shrink...

Thank you for the informative insight. I didn't know that there was a whole "industry" at play behind the scenes, and that so many people had early access to the early stages of development. I genuinely learnt something. This post would be worth a threadmark if the thread was bigger.

 

[Kwigo]

I like the way the devs of Dauntless managed the alpha of the game with a detailled trello which basically showed players what their priorities were and what they were currently working on.

Star Citizen also has an up to date roadmap on their website with everything planned for the next 3 updates (at least it showed up to 3.8 last time I checked).

 

[Älg]

I was about to say that this is most certainly already done through internal playtesting, but then I realised that Chronospherics already closed the thread (although I dont love the graph in his post).

 

[Komo]

Please don't. I'd rather not have Early Access become the norm. Because it promotes games to come out with no polish and completely buggy but also wildly overhyped. And seeing how that last part still happens. Imagine how it would be if we supported it.

Encourage long dev times, and less crunch and your games will basically come out pretty damn good.