How Rollback and Delay based netcode works in Fighting games

[Zombegoast]

Netcode [p1]: Fightin' Words

Infil goes into detail in why every fighting games should be using Rollback netcode over Delay Based netcode, how good Rollback hides bad connection while bad ones(Street Fighter V) does a bad job and why Japanese fighting games are preventing the growth of the genre for using Delay Based Netcode.

If you're part of the FGC, don't ask for characters, Instead go to tournaments or go on Twitter and ask for rollback netcode. GGPO is now open source. Ask Capcom to take a week off from making costumes to fix their netcode. Because nobody is going to play online fighting game that uses a poor netcode.

 

[hey it's that dog]

Of course you think Capcom could somehow change over their netcode with a little hard work when every explanation of GGPO clearly states that the game has to be built around it in certain ways.

 

[lvl 99 Pixel]

Still going through it. Its a long but interesting read.

 

[Jimnymebob]

Because nobody is going to play online fighting game that uses a poor netcode.

Click to shrink...

I mean, Marvel 3 was atrocious online on PS3, and you'd get matches in an instant.

 

[Zombegoast]

I mean, Marvel 3 was atrocious online on PS3, and you'd get matches in an instant.

Click to shrink...

Have you tried out the Steam version which is accessible? I dropped that after two matches.

 

[Mantrox]

Of course you think Capcom could somehow change over their netcode with a little hard work when every explanation of GGPO clearly states that the game has to be built around it in certain ways.

Click to shrink...

For a company that has been in the loop about GGPO for as long as Capcom has, it sure seems dumb for them not to incorporate it.
There aren't many excuses left, if any...

SFV could have been that game from the ground up.

This recent push from community voices about this particular issue is great, and this article explains the advantages in detail.

 

[Zyrox]

The article is great. Definitely recommend it for everyone who is interested in Fighting Games.
I hope JP devs listen and actually implement rollback in future games. That would be such a huge step forward for them.
I hope ArcSys does this in particular. The communities for their non licensed fighters are so small. Rollback would at least allow you to fight players not just limited to your region and have a decent experience.
Crossplay is another thing FG developers should look into. Fighting games online live and die by their community. Anything to expand that will greatly help the scene.

 

[Deleted member 8860]

FWIW, this was republished on Ars Technica with some additional comments by the author (and others).

Explaining how fighting games use delay-based and rollback netcode

How to design your game for optimal play over a network.

arstechnica.com

 

[Dreamboum]

It's fucking maddening how Tekken 7 could have the best netcode in HISTORY just because it is a game where moves takes several more frames than most fighting games and we're still experiencing delay-based

 

[Mantrox]

It's fucking maddening how Tekken 7 could have the best netcode in HISTORY just because it is a game where moves takes several more frames than most fighting games and we're still experiencing delay-based

Click to shrink...

If Bamco makes the investment to incorporate it on the next game, it would be bananas.
Just imagining a next gen Tekken with good netcode, uffffffff

 

[Zombegoast]

Of course you think Capcom could somehow change over their netcode with a little hard work when every explanation of GGPO clearly states that the game has to be built around it in certain ways.

Click to shrink...

Rising Thunder was like the first game on the Unreal 4 to use GGPO in 2015.

But I guess *work* is too much for them. Even after fixing it in Infinite and not touching SFV's.

It's fucking maddening how Tekken 7 could have the best netcode in HISTORY just because it is a game where moves takes several more frames than most fighting games and we're still experiencing delay-based

Click to shrink...

IIRC, T7 doesn't have a replay feature which is the base of adding rollback netcode.

 

[Jaded Alyx]

IIRC, T7 doesn't have a replay feature which is the base of adding rollback netcode

Click to shrink...

Yet if I remember correctly from reading the same article last week, one of the developers interviewed said tekken is a good candidate for rollback.

Perhaps he was talking about tekken in general and not specifically 7?

 

[hey it's that dog]

For a company that has been in the loop about GGPO for as long as Capcom has, it sure seems dumb for them not to incorporate it.
There aren't many excuses left, if any...

SFV could have been that game from the ground up.

This recent push from community voices about this particular issue is great, and this article explains the advantages in detail.

Click to shrink...

I'm not arguing against the advantages, of course Capcom should have done better with their netcode. It's too late for SFV is my point.

 

[Mantrox]

IIRC, T7 doesn't have a replay feature which is the base of adding rollback netcode.

Click to shrink...

T7 being a 2015 Arcade title with improvements probably doesn't help that part either.
I imagine a clean slate would be a better route now.

 

[Mantrox]

I'm not arguing against the advantages, of course Capcom should have done better with their netcode. It's too late for SFV is my point.

Click to shrink...

I agree.

 

[RazieltheShifter]

Because nobody is going to play online fighting game that uses a poor netcode.

Click to shrink...

That's what happens to me after whenever I try to play a online match in SFV.

 

[Dreamboum]

IIRC, T7 doesn't have a replay feature which is the base of adding rollback netcode.

Click to shrink...

Fundamentally a replay is about saving and replaying a set of inputs. There is not even a need for serialization. That's not the issue as far as I know. Guilty Gear is replaying when you spectate for example

T7 being a 2015 Arcade title with improvements probably doesn't help that part either.
I imagine a clean slate would be a better route now.

Click to shrink...

Tekken will never be a clean slate, everything is iterating since Tekken 3. If they don't do it now they will never do it

However I can potentially see that they have no performance headroom to incorporate rollback, something that the next generation would solve

 

[Zombegoast]

I'm not arguing against the advantages, of course Capcom should have done better with their netcode. It's too late for SFV is my point.

Click to shrink...

It's never too late. I want to play SFVI knowing that Infinite's netcode isn't a fluke.

But it seems to be as they somehow fixed this problem by adding more servers

Which it didn't.

 

[Kaguya]

Because nobody is going to play online fighting game that uses a poor netcode.

Click to shrink...

So how long did it take you to stop playing SFV?

This is only true for ranked modes, long as you have a functional netcode where you can play people from your local scenes games like UNIST, Xrd, DBFZ, etc... Lobby modes will still get played online, hell Tekken 7(2017) have a more active online scene(including ranked) than MK11(2019) by now.

 

[SoH]

Ask Capcom to take a week off from making costumes to fix their netcode.

Click to shrink...

But I guess *work* is too much for them.

Click to shrink...

Good and informative article worth reading. Lazy dev rhetoric less good.

 

[hey it's that dog]

It's never too late. I want to play SFVI knowing that Infinite's netcode isn't a fluke.

But it seems to be as they somehow fixed this problem by adding more servers

Which it didn't.

Click to shrink...

It's too late for SFV is what I said. If you couldn't read an eight-word sentence I have my doubts that you read this netcode article.

And of course we all want SFVI to have the best netcode possible. We'll see if new consoles can support AAA polygonal graphics and a GGPO-like solution simultaneously.

 

[HeihachiMishima]

Does anybody actually know how SFV's or Tekken's networking works?

Maybe it is implementing rollback, but implemented poorly?

 

[Kaguya]

Does anybody actually know how SFV's or Tekken's networking works?

Maybe it is implementing rollback, but implemented poorly?

Click to shrink...

SFV does use rollback, Tekken doesn't.

 

[HeihachiMishima]

SFV does use rollback, Tekken doesn't.

Click to shrink...

What does Tekken do?

 

[lucebuce]

 

[Kaguya]

What does Tekken do?

Click to shrink...

Regular delay based netcode.

 

[DennyDifferent]

Speaking as a network coder, this is a great article. The author describes issues like lock-step, determinism and rollback/replay in an easy to understand way. I'm impressed. I haven't had to implement a networking system such as this in any games in my professional career but earlier in the year I started writing a Pong clone which used deterministic lock-step and rollback just for shits and giggles at home. Really fun. I should go back and finish it off, really.

 

[LCGeek]

However I can potentially see that they have no performance headroom to incorporate rollback, something that the next generation would solve

Click to shrink...

That surprised me that rollback would be that demanding on frame times. Going from a few frames to 32MS is no joke.

Next gen we have beefy ryzen cpus which for fps or networking is insanely good.

Still gotta remind this community most of you still use low level OS settings like the following on your pcs and your routers.

Click to shrink...

Low level OS function in network stack govern anything in a netcode.

rx/tx usecs - This is the number of microseconds to wait before raising an interrupt after a packet has been received. When rx-usecs is set to 0 rx-frames is used.

rx/tx frames - This is the number of frames to queue up before raising an TX interrupt

In the case of my example the router will wait 40ms before transmitting packets to my modem, For most of you its the same or worse by default. This can easily be changed and on the fly if your hardware/software is good. For frames it would be 64 packets before it raises an interrupt and things are sent. Most of you get basic math and I know you wouldn't want that.

As I said in another similar topic a good netcode is necessary even with great tuning but relying on only netcode fixes won't make this genre more enjoyable or playable online. Routing can be addressed low level functions which dictate when your packets are sent, processed or made are another matter that netcode isn't built to deal with is another matter. From the articles I read some companies don't care to track those metrics or assume too much about them.

There are also networking offload features which add latency and when they are broke literally won't send or process your data for seconds.

This one setting could easily be turned off and most of your issues at the source of where packets starts won't be the ones causing it, other hops on routes can but at least you isolated the machines playing from being an issue. Would love to do busy polling testing but getting that environment from the public would be a nightmare. Busy polling would be a god send for fighting games. Busy Poll Sockets (BPS) is a linux kernel native solution for providing low network latency without application changes.

sadly this is one genre in my research that requires a both clients or hosts to tune, adjust, or disable features in the stack before good results are seen.

 

[WellbornWarrior]

Even after reading this, just predicting what players will do and rewinding if you're wrong seems like such a wild idea to hide lag that it's hard to imagine.

 

[Hyun Sai]

Is all the GGPO documentation translated in japanese by now ?

 

[Sixfortyfive]

Even after reading this, just predicting what players will do and rewinding if you're wrong seems like such a wild idea to hide lag that it's hard to imagine.

Click to shrink...

It's pretty straightforward. "Prediction" in this case almost always means "assuming the inputs on this frame will be the same on the next frame," which, as the article points out, is true during gameplay way more often than not.

 

[exodus]

I mean, shooters have been using client-side prediction as standard for over 20 years now. It's about time fighting games catch up to the times.

 

[WellbornWarrior]

I mean, shooters have been using client-side prediction as standard for over 20 years now. It's about time fighting games catch up to the times.

Click to shrink...

I don't think that's true. Maybe someone else can explain it better but most shooters have a client-server structure where what happens on the server takes priority. Theres no predictive element in there.

It's pretty straightforward. "Prediction" in this case almost always means "assuming the inputs on this frame will be the same on the next frame," which, as the article points out, is true during gameplay way more often than not.

Click to shrink...

It's hard to imagine because we are talking about incredibly small amounts of time here so I know predicting a simple action like this isn't that much out of the realm of the possible, but still it feels kinda diminishing if someone tells you the game can predict what you do whatever you do.

 

[smurfx]

I mean, Marvel 3 was atrocious online on PS3, and you'd get matches in an instant.

Click to shrink...

the game was super popular so that's why you would get matches. the game is super dead these days and if it had good netcode it might still have a community there.

 

[Augemitbutter]

never stop bugging them about good netcode. This should've already been resolved last gen. We are now at the end of current gen and still far-off. The rate of play is also very poor with all those load times in between. Online modes need better optimization.

 

[Delriach]

Highly encourage everyone to read the entire article, despite its length. Page 6 has an interview with our network engineer too, which gives some really good insight on rollbacks.

Also, infil has the best fighting game guide on the planet. Check it out and learn some KI ^_^

 

[Sixfortyfive]

never stop bugging them about good netcode. This should've already been resolved last gen. We are now at the end of current gen and still far-off. The rate of play is also very poor with all those load times in between. Online modes need better optimization.

Click to shrink...

I've said this in similar threads before, but it's just wild to me that accomplished major fighting game devs will put all of this thought and nuance into their gameplay systems and character designs, yet so little thought into how people actually play the game.

1v1 multiplayer is the entire core of these games and yet so many developers don't even make it a priority to ensure that it actually works well.

 

[exodus]

I don't think that's true. Maybe someone else can explain it better but most shooters have a client-server structure where what happens on the server takes priority. Theres no predictive element in there.

Click to shrink...

There's interpolation and extrapolation in games as far back as Quake. The server also needs to consider the client-side timing as to when a button was pressed according to the client. This is pretty much exactly the same as described in the article. The only difference is that the definitive state of the game is on a remote server and not on one of the player's machines.

In a shooter (and most multiplayer games), the server holds the authoritative game state and the clients predict the actual game state as best they can through interpolation and extrapolation. The server needs some level of rollback (or lag compensation as it might otherwise be known) to synchronize client states with its own state. Updates are then sent back out to the clients so they can correct their own states.

Just thinking of it now, the reason most people would say WoW feels better to play than FF XIV is that (adopting fighting game netcode terminology) WoW has rollback based netcode and FF XIV has delay based netcode. Well, not entirely true. FF XIV is a weird half assed implementation where the client position is out of sync with the server but client actions have a delay.

 

[Blackage]

GGPO is free now.

FGC needs to crowdfund a Japanese translation of all the documentation because we know they're not gonna do it themselves.

Basically gonna have to drag Japanese devs kicking and screaming to rollback future.

 

[SoH]

I don't think that's true. Maybe someone else can explain it better but most shooters have a client-server structure where what happens on the server takes priority. Theres no predictive element in there.

Click to shrink...

Quake Engine Code Review: Prediction

 

[Keits]

We'll see if new consoles can support AAA polygonal graphics and a GGPO-like solution simultaneously.

Click to shrink...

This is not an issue now and definitely wont be in the future.

 

[Jaded Alyx]

GGPO is free now.

FGC needs to crowdfund a Japanese translation of all the documentation because we know they're not gonna do it themselves.

Basically gonna have to drag Japanese devs kicking and screaming to rollback future.

Click to shrink...

Even if that translation happens - and it absolutely should - the other big hurdle is convincing the companies that this is something they need to prioritise/invest time in. I've seen a number of people who work at places like ArcSys say that it's tough to get changes like that approved in a corporate structure.

 

[Blackage]

Perhaps$ they$ are not$ speaking$ the correct$ language$$$$$ in that cooperate$ structure$$$ hehe.

 

[DanteMenethil]

Yet if I remember correctly from reading the same article last week, one of the developers interviewed said tekken is a good candidate for rollback.

Perhaps he was talking about tekken in general and not specifically 7?

Click to shrink...

The reason Tekken in general is a good candidate is that all the moves in that game has a high number of startup frames (sf5 lowest startup is 3 frames, not sure about tekken about its around 10 give or take). This gives you a lot of leeway to rollback the state without disoriententing the players / having that teleport effect.

 

[lucebuce]

not sure about tekken about its around 10 give or take.

Click to shrink...

The fastest generic move in the game is the jab at 10 frames however some characters have an 8 frame move. I believe Jack's palm and Yoshi's flash sword move are the only ones. So yeah.

 

[Jaded Alyx]

The reason Tekken in general is a good candidate is that all the moves in that game has a high number of startup frames (sf5 lowest startup is 3 frames, not sure about tekken about its around 10 give or take). This gives you a lot of leeway to rollback the state without disoriententing the players / having that teleport effect.

Click to shrink...

Yeh that was it.

 

[Uraizen]

If NRS was able to retroactively add rollback netcode to MKX then Capcom sure as hell can fix theirs for SFV. Will they? No. Could they? Sure.

 

[Zombegoast]

FWIW, this was republished on Ars Technica with some additional comments by the author (and others).

Explaining how fighting games use delay-based and rollback netcode

How to design your game for optimal play over a network.

arstechnica.com

Click to shrink...

Fighting games really need these coverage. You only ever see this in Eventhub but not something like Polygon or IGN. This article needs to grab as many people'saattentionm

FWIW, this was republished on Ars Technica with some additional comments by the author (and others).

Explaining how fighting games use delay-based and rollback netcode

How to design your game for optimal play over a network.

arstechnica.com

Click to shrink...

We need more coverage like these in other gaming news outlet. Get people's attention in how important this is.

 

[Blackage]

If NRS was able to retroactively add rollback netcode to MKX then Capcom sure as hell can fix theirs for SFV. Will they? No. Could they? Sure.

Click to shrink...

"When NetherRealm Studios decided to patch in rollback netcode to Mortal Kombat X, a game that had already shipped, it took approximately 8 man-years of time split between many programmers over 10 months."

For NetherRealm that was their first foray into rollback, SFV was Capcom's 2nd, and Marvel Infinite was Capcom's 3rd and best attempt(So I've heard), which means they've figured it out, but given that Marvel Infinite was 2 years ago, and SFV was 1 year and 9 months before that, if they were going to do anything to improve it they would have already.

NetherRealm could justify the investment/time/manpower because that gave them their foundation for good netcode in all of their games going forward. Capcom probably can't. So we can just hope they've learned all the right lessons for SFVI or Marvel 4 or whatever fighting game they make in the future.

SFV is done, I just expect costumes and characters until it dies and little else. =/

 

[Deleted member 8860]

Even if that translation happens - and it absolutely should - the other big hurdle is convincing the companies that this is something they need to prioritise/invest time in. I've seen a number of people who work at places like ArcSys say that it's tough to get changes like that approved in a corporate structure.

Click to shrink...

I'd imagine a lot of it just comes down to rollback being traditionally unnecessary in mainland Japan, where population density is high, internet connectivity is excellent, and the maximum distance/hops between endpoints is miniscule.