How Oodle Kraken and Oodle Texture supercharge the IO system of the Sony PS5

[nib95]

How Oodle Kraken and Oodle Texture supercharge the IO system of the Sony PS5

The Sony PS5 will have the fastest data loading ever available in a mass market consumer device, and we think it may be even better than yo...

cbloomrants.blogspot.com

Confirmation 8/9 GB/s were not using oodle texture.

Click to shrink...

If the previous compression ratio was 1.5, and in this quote a 2:1 ratio is now being mentioned, as a few people in here have stated that would take the new compressed bandwidth to 11GB/s. Out of curiosity, how is this other 15GB/s and 3:16 ratio that a couple have mentioned being arrived at?

 

[gofreak]

I haven't seen the blog yet, not following much lately (but I did pre-order three PS5 digital edition 😬). Are they claiming 17GB/s-18GB/s like you've said or ~10GB/s+ like I've said?

Click to shrink...

They're claiming ~2x overall on game data (11GB/s), with I presume some assumed ratio of texture data, and ~3x on texture data specifically.

Nothing different from what could have been extrapolated from other blog posts of theirs per se, at least if one made some reasonable assumption about the ratio of texture:non-texture data in a game. But they're sort of rubber stamping that as reasonable estimates.

If the previous compression ratio was 1.5, and in this quote a 2:1 is now being mentioned, as a few people in here have stated that would take the new compressed bandwidth to 11GB/s. Out of curiosity, how is the 15GB/s and 3:16 ratio that a couple have mentioned being arrived at?

Click to shrink...

Texture data only vs an overall mix of game data.

---

We should always remember that these ratios will vary game by game, and depending on texture formats used etc. But I think the biggest takeaway from this is more or less confirmation that the earlier figure from Sony wasn't accounting for texture encoding improvements since.

 

[Bold One]

Someone explain this to me in DBZ terms.

 

[nib95]

They're claiming ~2x overall on game data (11GB/s), with I presume some assumed ratio of texture data, and ~3x on texture data specifically.

Nothing different from what they've said in other blog posts per se, at least if one made some reasonable assumption about the ratio of texture:non-texture data in a game. But they're sort of rubber stamping that as reasonable estimates.



Texture data only vs an overall mix of game data.

---

We should always remember that these ratios will vary game by game, and depending on texture formats used etc. But I think the biggest takeaway from this is more or less confirmation that the earlier figure wasn't accounting for texture encoding improvements since.

Click to shrink...

So 11GB/s would be an overall average compressed bandwidth for a mix of data, whilst the 15GB/s number is focusing more on texture data?

 

[Skeff]

Someone explain this to me in DBZ terms.

Click to shrink...

We thought it was SSJ2 But actually it's probably SSJ3.

 

[chris 1515]

I haven't seen the blog yet, not following much lately (but I did pre-order three PS5 digital edition 😬). Are they claiming 17GB/s-18GB/s like you've said or ~10GB/s+ like I've said?

Click to shrink...

They said on average 11 GB/S with all type of data but it can go higher depending of the texture set. It will change like in Racthet and clank from level to level I suppose. I never said it will be 17 GB/s all time. I said it can peak to there. On R&C Rift Apart some portal load faster than other, it will be the same depending of the set of texture to load.

So 11GB/s would be an overall average compressed bandwidth for a mix of data, whilst the 15GB/s number is focusing more on texture data?

Click to shrink...

It will vary from level to level in the same games. This is compression. Sometimes it will be 10 GB/S other time it is 13 GB/s. This is the reason some portal load faster than other in R&C Rift Apart.

 

[Alek]

Click to shrink...

That image looks untextured though. Not the best point of comparison. I'm sure that there's a texture applied to those assets, but it lacks detail.

You can see the real differences here...

Oodle Texture RDO Examples

RAD Game Tools' web page. RAD makes Bink Video, the Telemetry Performance Visualization System, and Oodle Data Compression - all popular video game middleware.

www.radgametools.com

Pretty much non-existent.

I dispute the idea that the differences aren't visible to the human eye though. I can see a difference in the compressed and uncompressed textures, but not a meaningful one.

 

[isahn]

So 11GB/s would be an overall average compressed bandwidth for a mix of data, whilst the 15GB/s number is focusing more on texture data?

Click to shrink...

It is my understanding that oodle encoded texture can be compressed by kraken close to 2:1. Not sure where the 3:1 figure has been derived from.

A question: is the oodle codec still in the hw block?

 

[DrKeo]

We should always remember that these ratios will vary game by game, and depending on texture formats used etc. But I think the biggest takeaway from this is more or less confirmation that the earlier figure from Sony wasn't accounting for texture encoding improvements since.

Click to shrink...

If those numbers did't use any form of RDO then that's pretty unprofessional of Cerny. ROD is extremely common long before Oodle textures.

 

[Mergesort]

Can someone get Cherno to make a video on this? Best ELI5 we can get!

 

[chris 1515]

It is my understanding that oodle encoded texture can be compressed by kraken close to 2:1. Not sure where the 3:1 figure has been derived from.

A question: is the oodle codec still in the hw block?

Click to shrink...

This comes from an examples of a texture set/level of a games it means for this particular level, the speed will be higher. Speed will vary by level and texture set. It can be slower than 11GB/s too.

they even give an example of texture set where they reach 3,99 to 1 with lambda 40 it means on this case they can nearly saturate PS5 SSD. It will just means this data will load very fast but it will not have any meaning for data streaming.

Out of loading, portals or cazy setpieces I don't think it will be useful and Fabian Giesen from RAD Gametool said the same on twitter.

 

[Tora]

This should also really help out the Ram

Click to shrink...

Yep

 

[gofreak]

It is my understanding that oodle encoded texture can be compressed by kraken close to 2:1. Not sure where the 3:1 figure has been derived from.

A question: is the oodle codec still in the hw block?

Click to shrink...

The oodle encoding is an offline preprocess that leaves the textures in a state that is readable natively by the GPU. It doesn't need any 'decoding', it then just uses the standard kraken compression which is decompressed in hardware on PS5...

... except if you use BC7Prep on BC7 textures - an optional additional bit of encoding for even more compression on those textures, which does require a bit of extra decoding. The decode for that is done on the cpu or gpu.

 

[behOemoth]

So 11GB/s would be an overall average compressed bandwidth for a mix of data, whilst the 15GB/s number is focusing more on texture data?

Click to shrink...

Game sizes can be compressed by the factor of 2 using their newest compressions, e.g. a 50 GB game becomes 25 GB. Games aren't completely made of textures or other easily compressible data.
However, data related to stream vom the disk to the ram is usually the data which can be compressed by a factor of 3. Hence the 15GB/s bandwidth.

 

[jroc74]

Look how slow this thread is moving say compared to a post about a game not getting a free next gen upgrade. That should tell you how interested the general public is about these sorta details. Lol

But if you show them this, now that would grab attention for a little while anyway until they find other things to complain about.

Click to shrink...

Yeah, this is easy to understand.

I'm going to be generous and assume that 98% of the people commenting in this thread have no idea what any of this means.

I'm here for the oodles of Kracken that Sony is about to unleash.

Click to shrink...

lol!

Looks like Cerny is going to have to do a Road to PS5 Part 2 soon.

Click to shrink...

Yup, lol.

 

[BreakAtmo]

They said on average 11 GB/S with all type of data but it can go higher depending of the texture set. It will change like in Racthet and clank from level to level I suppose. I never said it will be 17 GB/s all time. I said it can peak to there. On R&C Rift Apart some portal load faster than other, it will be the same depending of the set of texture to load.



It will vary from level to level in the same games. This is compression. Sometimes it will be 10 GB/S other time it is 13 GB/s. This is the reason some portal load faster than other in R&C Rift Apart.

Click to shrink...

Hell, I expect it to vary from frame to frame.

 

[BreakAtmo]

Looks like Cerny is going to ha e to do a Road to PS5 Part 2 soon.

Click to shrink...

Does it switch to Ken Kutaragi halfway through?

 

[chris 1515]

Hell, I expect it to vary from frame to frame.

Click to shrink...

Yes but I think there is enough room to not affect streaming. Out of loading, portals or crazy setpieces this will not be useful.

 

[gofreak]

If those numbers did't use any form of RDO then that's pretty unprofessional of Cerny. ROD is extremely common long before Oodle textures.

Click to shrink...

I wouldn't want to pass too many judgments or make too many assumptions, but he was looking at data in general rather than texture data specifically, and tbh, I would guess he was offering numbers that his IO/compression team gave him - I don't think he's necessarily an expert in the field himself. And they likely offered figures that were independent of tools/data that you weren't necessarily getting out of the box at that time, (pre Oodle Texture licensing). So it was conservative, but I wouldn't stretch to unprofessional. His target audience (devs) would know what better texture encoding would mean.

I don't think he was necessarily anticipating that consumers would latch onto those numbers too preciously, in apples:apples comparisons with other systems or whatever.

 

[isahn]

The oodle encoding is an offline preprocess that leaves the textures in a state that is readable natively by the GPU. It doesn't need any 'decoding', it then just uses the standard kraken compression which is decompressed in hardware on PS5...

... except if you use BC7Prep on BC7 textures - an optional additional bit of encoding for even more compression on those textures, which does require a bit of extra decoding. The decode for that is done on the cpu or gpu.

Click to shrink...

Cool, so the AMD GPU supports natively the oodle texture format, is that correct?

 

[gofreak]

Cool, so the AMD GPU supports natively the oodle texture format, is that correct?

Click to shrink...

Correct, the ones that don't use BC7Prep anyway. That needs another little extra step before the texture is gpu native again.

That is to say - the texture format isn't oodle specific. It's just standard BCn texture data. The encoding is specific, but to BCn specs.

 

[MrKlaw]

does this work as a comparison to get across the differences?

- HDD in the PS4 is like a faster version of the CD and DVD drives in previous consoles. They're fast, but still too slow for game engines so you need to plan and predict when to pull data

- SSD in the PS5 is like a slower version of RAM. You don't need to organise data - you aren't limited by seek times, you can pretty much just access any of your game data at any time.

 

[chris 1515]

For crazy setpiece maybe we will see DBZ, Man of Steel final fight level of destuction in the next God of War where a boss fight punch send you at the other side of a realm at high speed destroying some of the scenery like a mountain.

 

[Mubrik_]

So 11GB/s would be an overall average compressed bandwidth for a mix of data, whilst the 15GB/s number is focusing more on texture data?

Click to shrink...

yes, from the blogpost:

Oodle Texture is a software library that game developers use at content creation time to compile their source art into GPU-ready BC1-7 formats. All games use GPU texture encoders, but previous encoders did not optimize the compiled textures for compression like Oodle Texture does. Not all games at launch of PS5 will be using Oodle Texture as it's a very new technology, but we expect it to be in the majority of PS5 games in the future. Because of this we expect the average compression ratio and therefore the effective IO speed to be even better than previously estimated.

Click to shrink...

I assume kraken compresses all usual game data but textures get a 2-pass compression
1st pass from Oodle texture compression, which makes it easier for kraken to compress
2nd pass from kraken

So without oodle texture and just kraken you are looking at around 9-11GB/s like cerny estimated
But using oodle texture further compresses the textures(which are the largest part of game data?) and case seem to be 15GB/s+

For crazy setpiece maybe we will see DBZ, Man of Steel final fight level of destuction in the next God of War where a boss fight punch send you at the other side of a realm at high speed destroying some of the scenery like a mountain.

Click to shrink...

I'm still thinking we destroy the room in the center of the map that allows movement through different realms in such a huge boss fight then somehow get the ability to quick travel putting the SSD to good use lol
Do it cory.

 

[Corralx]

Cool, so the AMD GPU supports natively the oodle texture format, is that correct?

Click to shrink...

There's no Oodle texture format.
Oodle is just another BCn texture encoder, so the outputs are standard BCn textures that all modern desktop GPUs can use.

Correct, the ones that don't use BC7Prep anyway. That needs another little extra step before the texture is gpu native again.

Click to shrink...

I suppose you're talking about BCPack, and in that case there's specific hw to decompress that data.
That is, using BCPack to compress texture data (encoded with Oodle or any other tool) is not any different than using Kraken to compress Oodle data, and leverage the hw decompressor on PS5.

 

[Dan Thunder]

For crazy setpiece maybe we will see DBZ, Man of Steel final fight level of destuction in the next God of War where a boss fight punch send you at the other side of a realm at high speed destroying some of the scenery like a mountain.

Click to shrink...

Sounds similar to what we've already seen from this bit at the end of the UE5 demo.

 

[Deleted member 20297]

I suppose you're talking about BCPack, and in that case there's specific hw to decompress that data.
That is, using BCPack to compress texture data (encoded with Oodle or any other tool) is not any different than using Kraken to compress Oodle data, and leverage the hw decompressor on PS5.

Click to shrink...

So the Xbox Series consoles do zlib in hardware as a "general" codec but bcpack is hw-decompressed, too?

 

[gofreak]

I suppose you're talking about BCPack, and in that case there's specific hw to decompress that data.
That is, using BCPack to compress texture data (encoded with Oodle or any other tool) is not any different than using Kraken to compress Oodle data, and leverage the hw decompressor on PS5.

Click to shrink...

I meant BC7Prep, which is another Oodle encoding step:

Oodle Texture bc7prep data flow

We mostly talk about Oodle Texture encoding to BCN from source art, usually with RDO (rate-distortion optimization). In the previous po...

cbloomrants.blogspot.com

With BCPack (the MS tech), I'd assumed it was an encoder much the same as Oodle Texture rather than the compressor itself. And wouldn't be available, I guess, on PS5 anyway, unless and until MS make decode implementations available for generic CPU/GPU. (I honestly don't know what the status of BCPack is wrt licensing etc - I assume it will eventually be more generally available on PC and maybe even PS with sw decoding).

 

[Gamer17]

Sounds great but since this just was released to devs , it takes time till we see the results in games , correct ?

 

[Dan Thunder]

Sounds great but since this just was released to devs , it takes time till we see the results in games , correct ?

Click to shrink...

Not all games at launch of PS5 will be using Oodle Texture as it's a very new technology, but we expect it to be in the majority of PS5 games in the future.

Click to shrink...

 

[Corralx]

I meant BC7Prep, which is another Oodle encoding step:

Oodle Texture bc7prep data flow

We mostly talk about Oodle Texture encoding to BCN from source art, usually with RDO (rate-distortion optimization). In the previous po...

cbloomrants.blogspot.com

Click to shrink...

Ah, I forgot about that extra tool existed.
Not sure if it's worth going through with it on consoles, you'd either require CPU assistance, or have the GPU re-swizzle the data back into BC7 format on the GPU which can potentially become a quite demanding job as gigabytes of data starts to be streamed in.

EDIT: tbh I suppose it's a very memory intensive operation so maybe you can find some space for it in async compute, paired with ALU/geometry heavy operations.

 

[Calabi]

That image looks untextured though. Not the best point of comparison. I'm sure that there's a texture applied to those assets, but it lacks detail.

You can see the real differences here...

Oodle Texture RDO Examples

RAD Game Tools' web page. RAD makes Bink Video, the Telemetry Performance Visualization System, and Oodle Data Compression - all popular video game middleware.

www.radgametools.com

Pretty much non-existent.

I dispute the idea that the differences aren't visible to the human eye though. I can see a difference in the compressed and uncompressed textures, but not a meaningful one.

Click to shrink...

The lighting is baked textures all those shadows and highlights etc are baked into the textures and so compressed. The point is you can't see a difference.

Click to shrink...

I think what he's saying is there's still processing that you have to do, like setting variables moving stuff around except that requires processing time, that will slow the load times down a bit. Like I doubt we will quite see 1 second load times, even if it can fully fill the memory in that time.

 

[Bold One]

Dat Cocaine SSD leds

 

[platocplx]

The lighting is baked textures all those shadows and highlights etc are baked into the textures and so compressed. The point is you can't see a difference.



I think what he's saying is there's still processing that you have to do, like setting variables moving stuff around except that requires processing time, that will slow the load times down a bit. Like I doubt we will quite see 1 second load times, even if it can fully fill the memory in that time.

Click to shrink...

yeah exactly.

 

[Alek]

The lighting is baked textures all those shadows and highlights etc are baked into the textures and so compressed. The point is you can't see a difference.

Click to shrink...

I get that, but on the site I linked I can see a difference when more detailed textures are compressed. Even though that difference is very slight, you can pick it up in the minor details. I doubt that I would be able to see that different at all, in a scene as simplistic as the Warframe scene.

I'd like to see the comparison on say, close the in a detailed scene in something like Modern Warfare.

I'm pretty certain you won't be able to see a difference regardless of the scene / game. It's just that, in the close-up textures presented on the radgame tools website, you can see a difference despite the claim 'invisible to the human eye'. There's a visible difference as you flicker between the compressed and uncompressed, highly detailed textures, in their demo (linked below).

Oodle Texture RDO Examples

RAD Game Tools' web page. RAD makes Bink Video, the Telemetry Performance Visualization System, and Oodle Data Compression - all popular video game middleware.

www.radgametools.com

 

[StereoVSN]

I was going to make a joke about next Bethesda game finally having no load times to get into a building, but then remembered the whole buyout situation... so yeah. How about Witcher 4 without loading times!?

 

[platocplx]

I was going to make a joke about next Bethesda game finally having no load times to get into a building, but then remembered the whole buyout situation... so yeah. How about Witcher 4 without loading times!?

Click to shrink...

hell lets see what they do with Witcher 3 and Cyberpunk Next Gen Versions next year.

 

[StereoVSN]

hell lets see what they do with Witcher 3 and Cyberpunk Next Gen Versions next year.

Click to shrink...

I want to see what the do with DLSS and RTX on PC this year :P. But yeah, it will be amazing to see Witcher 3 all spruced up and Cyberpunk will have some power to utilize on XSX/PS5.

 

[platocplx]

I want to see what the do with DLSS and RTX on PC this year :P. But yeah, it will be amazing to see Witcher 3 all spruced up and Cyberpunk will have some power to utilize on XSX/PS5.

Click to shrink...

the PC version is going to be the best version of that game. They Dev PC as lead platform so PC def gets all the goodies

 

[Deusmico]

does this work as a comparison to get across the differences?

- HDD in the PS4 is like a faster version of the CD and DVD drives in previous consoles. They're fast, but still too slow for game engines so you need to plan and predict when to pull data

- SSD in the PS5 is like a slower version of RAM. You don't need to organise data - you aren't limited by seek times, you can pretty much just access any of your game data at any time.

Click to shrink...

Yes previously you need to do loading screens for the part of the game you want, now its so fast you dont need to stop and load. 2sec should be the most in normal situations

 

[dgrdsv]

- SSD in the PS5 is like a slower version of RAM. You don't need to organise data - you aren't limited by seek times, you can pretty much just access any of your game data at any time.

Click to shrink...

You still need to plan ahead on when you will be pulling what data off SSD because even PS5 SSD isn't fast enough to serve as a "slow RAM".
The complexity of this streaming system and the amount of predictions it must handle will be down significantly though.
It'll also use a lot less RAM for caching and this will allow higher sized RAM pools to be used for frame rendering.

 

[PLASTICA-MAN]

Up to 17.38 Gb/s they say? Dunno how it can even exceed the whole PS5 VRAM amount.
Iirc, didn't Cerny mention 22Gb/s during the PS5 presentation, What happened to that?

 

[MrKlaw]

Up to 17.38 Gb/s they say? Dunno how it can even exceed the whole PS5 VRAM amount.

Click to shrink...

just doesn't take the full second :)

I think the RAD guys said when they were designing it, that Cerny specced a system that wouldn't be bottlenecked until 22GB/s

 

[PLASTICA-MAN]

just doesn't take the full second :)

I think the RAD guys said when they were designing it, that Cerny specced a system that wouldn't be bottlenecked until 22GB/s

Click to shrink...

Ah. I gotcha.
Iirc, didn't Cerny mention 22Gb/s during the PS5 presentation, What happened to that?

 

[MrKlaw]

Ah. I gotcha.
Iirc, didn't Cerny mention 22Gb/s during the PS5 presentation, What happened to that?

Click to shrink...

Just a theoretical cap on possible output. Never likely to be necessary as a practical, consistent througput with a 5.5GB/s source.

 

[Crayon]

not sure it matters when you baseline is 25x a sata 3 drive, and 50x the sata 2 drives in the ps4/xb1

Click to shrink...

How soon do you think games will come along that have an nvme as min requirement? I thought it would be quite awhile.

 

[gofreak]

Ah. I gotcha.
Iirc, didn't Cerny mention 22Gb/s during the PS5 presentation, What happened to that?

Click to shrink...

It all depends on the data you're fetching.

If it compresses at 4:1 or more, then you'll technically transfer it at 22GB/s.

Like any of these examples here that reach 4:1 or higher on Kraken:

So nothing happened to it, as Cerny said, it's a peak that could be reached if the data happens to compress particularly well vs the typical ('2:1').

 

[eathdemon]

How soon do you think games will come along that have an nvme as min requirement? I thought it would be quite awhile.

Click to shrink...

the only game I know of is star citizen, but sevral including wow's latesr xpack recamends it.

 

[Igorth]

PS5 is a beast

 

[Crayon]

the only game I know of is star citizen, but sevral including wow's latesr xpack recamends it.

Click to shrink...

Whoa does sc require nvme? I thougmt it was just sata ssd. I dream squadron 42 comes to console (doubt) because my pc is pretty modest as i dont like throwing too much money at hardware.