• Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.

Lukas Taves

Banned
Oct 28, 2017
5,713
Brazil
Do we actually know anything about the discrete audio processing hardware in the Series X yet? Other than that it exists, and that it's called (I think) "Project Acoustics"?
Project acoustics is not the hardware, it's a MS framework for sound propagation. It supports Decay, reverberation, etc, all considering the actual scene geometry.

The hardware part is likely an evolution of the audio block they already had on xbone. One of the characteristics of the audio block was that it allowed to read a shared memory block with the cpu/gpu and had some programmable cores, which is why they managed to support atmos and other object based framework. (And likely how on SX they will be able to use raytracing)
 

Lukas Taves

Banned
Oct 28, 2017
5,713
Brazil
PS4 didn't have an audio chip. According to this interview, the Xbox One chip did make some stuff free from a processing standpoint, but it wasn't programmable and couldn't do everything. https://gamingbolt.com/audiokinecti...next-gen-audio-solutions-for-ps4-and-xbox-one



So PS5's Tempest does offer more in the obvious power boost compared chip to chip to the one seen in Xbox One, and that it is progammable. Xbox Series X also has a new audio block. We just really dont know as much about it as the PS5 solution. https://news.xbox.com/en-us/2020/03/16/xbox-series-x-glossary/
I don't know where they got that the shape wasn't programmable on xbone but that's wrong. It had some fixed parts mostly encoding decoding, but it had some programmable cores and coherent memory access. Ms made a point of that in the hotchips presentation and later they said that windows sonic was leveraging that.
 

Segafreak

Member
Oct 27, 2017
2,756
So after watching Cerny for an hour I looked into 3D audio a bit more since I completely disregarded before. I am using my stereo headphones and the speakers in my monitor (like some barbarian).

Hellblade has been mentioned as a really good demo for 3D audio. Some parts of the game were recorded as binaural which is what the PS5 tries to replicate with in game sound sources.

To my surprise 3D audio is a big deal with headphones, and can arguably be better for it than surround systems. The funny thing is, this is totally possible on current gen. I know this might be common knowledge for some people but it was certainly new for me.

Basically both Microsoft and Dolby already has a solution which is basically the same as what Sony is doing with the PS5, with different technical parameters. Well, even Sony has a solution, which is the Platinum Headset.


On Xbox One and Windows both Windows Sonic for Headphones and Dolby Atmos for Headphones (paid) is available right now with some support from games. As far as I understand, they do basically the same thing as the Tempest Engine. They take 3D object metadata and using a HRTF model they process sound for headphones. Sony's Platinum Headset should be doing the same thing, but instead of the console, the headset processes the 3D object metadata (along with the HRTF spectral transforms?), although I have no clue how that works.
What is the "PS5 Tempest" and why did Sony make such a big deal out of it if others and even they themselves are already doing it?
Well he told us about it. He even mentioned things that are literally impossible right now (personal HRTF). But sure, he must be hiding another bottle of secret sauce...
What other bottles of secret sauce is he hiding?
 

DavidDesu

Banned
Oct 29, 2017
5,718
Glasgow, Scotland
It is because your HRTF is far from the HRTF needed in the video, this is one problem Sony want to help to solve.
Yeah that's what Cerny said in his talk. Some people they demoed it with just thought it sounded like alright virtual surround rather than getting the full experience because their HRTF differed further from the generic one they used, hence why getting as personalised an HRTF to you as possible will be important to make the most of it. I cannot honestly wait, if we can get virtual surround that's something like the good binaural recordings out there that we see in ASMR it will be incredible. For example there's some videos out there where someone might be tapping on glass right next to your ear and it's not just one sound, you can virtually hear and place each fingertip as they rattle on the glass. It really starts fooling your brain into thinking their hand is literally next to your head rather than it sounding like a recording. This is the immersion and presence Cerny is trying to bring to the audio realm on consoles and I'm so damn excited about that. Like if in a game a character places a gun to your head, you will hear and place the sound of their finger on the trigger, the mechanisms in the gun moving. The fidelity has the potential to be truly next generational.
 

Surface of Me

Banned
Oct 25, 2017
3,207
You're only focusing on the output. Basically what those programs are doing is mapping 5.1 and 7.1 sources to HRTF profiles to create virtual surround through stereo headsets. Audio is being processed by the CPU and largely hasn't changed (arguably become worse) over the last 15 years.

What Sony is doing with PS5 is totally different and "game changing" in that the audio is now running on powerful, dedicated hardware with HRTF profile outputs for any headset. With this (and on XSX's GPU) you're going to get things like ray traced audio that reacts to materials and sound stage area. On Tempest, all of that processing is freed up from the GPU and taken on by hardware dedicated to maximizing audio processing and output for the console.

It's apples and oranges.

This is false info that is proven demonstrably false by several links in this thread.


I don't know where they got that the shape wasn't programmable on xbone but that's wrong. It had some fixed parts mostly encoding decoding, but it had some programmable cores and coherent memory access. Ms made a point of that in the hotchips presentation and later they said that windows sonic was leveraging that.

Thanks for the info. I noticed it's surprisingly hard to find info on this subject.
 

panda-zebra

▲ Legend ▲
Member
Oct 28, 2017
5,737

FFS, I just started playing this, thought it wasn't playing, then got distracted by some kerfuffle in the next room... suspecting my son wasn't busy doing his school work I got up to rush next door then heard some guy making noises as I took the headphones off and realised it was the damn video. lol.
 

Praedyth

Member
Feb 25, 2020
6,519
Brazil
I won't pretend I know anything about audio, but I kinda dig the way Cerny talked about transforms and etc. That being said, I think Tempest main goal isn't to make a good 3D audio, I believe its main goal is to be available to everyone, from a cheap headset to an expensive surround system. I guess that's the main difference between PS5's Tempest vs XSX's Atmos. Regarding Windows Sonic, I don't think it would be free for Sony to implement (just guessing, because nowadays everything must be licensed) or would it suit Sony's needs for VR.
 

dimitri2401

Member
Oct 27, 2017
175
Montreal
Sony's Platinum Headset should be doing the same thing, but instead of the console, the headset processes the 3D object metadata (along with the HRTF spectral transforms?), although I have no clue how that works.

I'm pretty sure the dongle is responsible for the processing. Therefore it's a very weak processor, which probably doesn't really have access to the full 3D objects data. In comparison, the Tempest engine seems insanely powerful. And since it will be standard on all PS5, games will be designed with it in mind.

3D audio is awesome on PSVR, I cannot wait to try it on PS5.
 

AllBizness

Banned
Mar 22, 2020
2,273
So after watching Cerny for an hour I looked into 3D audio a bit more since I completely disregarded before. I am using my stereo headphones and the speakers in my monitor (like some barbarian).

Hellblade has been mentioned as a really good demo for 3D audio. Some parts of the game were recorded as binaural which is what the PS5 tries to replicate with in game sound sources.

To my surprise 3D audio is a big deal with headphones, and can arguably be better for it than surround systems. The funny thing is, this is totally possible on current gen. I know this might be common knowledge for some people but it was certainly new for me.

Basically both Microsoft and Dolby already has a solution which is basically the same as what Sony is doing with the PS5, with different technical parameters. Well, even Sony has a solution, which is the Platinum Headset.

On Xbox One and Windows both Windows Sonic for Headphones and Dolby Atmos for Headphones (paid) is available right now with some support from games. As far as I understand, they do basically the same thing as the Tempest Engine. They take 3D object metadata and using a HRTF model they process sound for headphones. Sony's Platinum Headset should be doing the same thing, but instead of the console, the headset processes the 3D object metadata (along with the HRTF spectral transforms?), although I have no clue how that works.

Point is, if you have an Xbox or a Windows PC and want to try it out, there are a couple of games that support it (Insect is a really good free tech-demo on Xbox). Even if they don't, both Atmos for Headphones and Sonic for Headphones can downmix a 7.1/5.1 output into virtual surround using the same HRTF technique.

While I am interested in the Tempest, the tech certainly doesn't feel as magical to me anymore. Don't get me wrong, I was totally blown away by both Windows Sonic and Atmos on headphones after I tried it, but by the look of things, this doesn't feel like it's as groundbreaking as Cerny said.

Also, Windows Sonic is free and cross-platform, which means PC gamers get the same treatment too. I am extremely interested how the Tempest might popularize this tech. The Platinum Headset's 3D audio support was certainly held back by the hardware requirement. With Windows Sonic and an Xbox One/PC any simple headphone is enough, yet support is not there. I really hope the Tempest will popularize this technology. Right now it's virtually impossible to know if a game supports Windows Sonic. Atmos is easier, since Dolby lists them on their site. In general, the PR behind these is pure crap. Sony managed to get more people interested in 20 mins than these collectively over years. The customizable HRTF models are also interesting. I believe both Microsoft's and Dolby's solution uses just one general model.

What do you guys think? Have you tried this or you heard from this before? I try to keep myself well informed but I was completely blind about this and virtual surround in general.
From my understanding the true benefit of Tempest is it works with any headphones even the cheapest ones, that's what makes it worth it.
 
OP
OP
Cats_Schrodinger
Oct 29, 2017
2,998
From my understanding the true benefit of Tempest is it works with any headphones even the cheapest ones, that's what makes it worth it.

Xbox One and Windows Dolby Atmos for Headphones and Windows Sonic works. with cheap any headset too. They work the same way as Tempest on Xbox One. On PC you have to sacrifice CPU though.
 

jroc74

Member
Oct 27, 2017
28,992
Holy crap, this one gave me a spook!
The knocking was coming EXACTLY from the same position my main door is located from where I was sitting and I instantly put my earbuds out and jumped up from my chair. o_O
Same, lol. I was like who the hell is knocking at the door this early in the morning?
 

AllBizness

Banned
Mar 22, 2020
2,273
Xbox One and Windows Dolby Atmos for Headphones and Windows Sonic works. with cheap any headset too. They work the same way as Tempest on Xbox One. On PC you have to sacrifice CPU though.
That's good, this however is a 1st for PlayStation and I think thats the point. Just like Xbox Series X will now have a share button. This is why competition is good. I'm pretty sure the sound specs of the consoles will end up being equal anyways with Series X 3D audio doing 100 objects like PS5. No big deal. I prefer surround sound personally.
 

ShutterMunster

Art Manager
Verified
Oct 27, 2017
2,449
Indeed, but HRTF transforms and 3D spatial audio is already working flawlessly in supported games. Games that include Gears 5 and Forza Horizon 4. It probably uses the existing sound block in the Xbox One, which already boasts some 15 GFLOPS. I know, I know, FLOPS all over again, no doubt that the Tempest engine is going to be a totally different level, but purely HRTF 3D audio is certainly possible today even with quite performance intensive games.

And by the way, Windows Sonic is also available for all games through Microsofts API. In fact it's cross-platform with Windows.

Found in documentation:

Sony's main aim seems to be ensuring 3D audio is supported at a system level and creating rendering & virtualization systems that make it possible for anyone to experience it without the need for additional hardware, which isn't the case on PS4. They've dedicated considerable power to audio to ensure that bit-depth and sample rates stay high as we scale up not only the amount of sound sources but the environmental responses for each individual sound.

But isn't the same true for Xbox One and PS4 already?

Yes and no.

I'm aware...
PS4 did too with TruAudio DSP.

As a whole, I feel like Audio has always been neglected. But all those PR 3D Audio aren't going to push it to the right direction.

If anything, when most people already neglect audio quality, I dont even see the point.
This is a bit more involved than just having a DSP.
 

ClamBuster

Member
Oct 27, 2017
4,092
Ipswich, England
Personnalised HRTF is very important. I know someone who only hear the sound coming from left and right in the video. In this video I heard the sound coming from everywhere but some people hear the sound coming from the right, the left and behind but when it is suppose to be above the head they heard it like it is from behind. Depending of your HRTF it works well or not.



i just grabbed my airpods and this was really impressive

now, it might have been 'programmed' like that, but at no point did i hear anything in front of me

the best thing i noticed was that i could very easily judge individual sounds height
 
Oct 26, 2017
6,151
United Kingdom
i just grabbed my airpods and this was really impressive

now, it might have been 'programmed' like that, but at no point did i hear anything in front of me

the best thing i noticed was that i could very easily judge individual sounds height

Beacause it's a barber shop. The mirror is in front of you affixed to a wall. Have you never been to a barber shop before?
 

disco_potato

Member
Nov 16, 2017
3,145
I'm pretty sure the dongle is responsible for the processing. Therefore it's a very weak processor, which probably doesn't really have access to the full 3D objects data. In comparison, the Tempest engine seems insanely powerful. And since it will be standard on all PS5, games will be designed with it in mind.

3D audio is awesome on PSVR, I cannot wait to try it on PS5.
You're thinking we'll be able to use psvr on ps5 without the need for the breakout box? Bypass the 3d audio chip in that and use tempest?
 
OP
OP
Cats_Schrodinger
Oct 29, 2017
2,998
You're thinking we'll be able to use psvr on ps5 without the need for the breakout box? Bypass the 3d audio chip in that and use tempest?

That's a pretty good assumption. I do not know exactly how the PSVR connects because I never had one but the setup should definitely be simplified since Tempest is superior in every way to anything we have currently.
 

dimitri2401

Member
Oct 27, 2017
175
Montreal
You're thinking we'll be able to use psvr on ps5 without the need for the breakout box? Bypass the 3d audio chip in that and use tempest?

That would be great for cable management.

The problem might be that the cable from the headset is split in two. I don't believe Sony would put a dual input on the PS5 just for PSVR when you only need a USB to connect the processing unit. Console manufacturers usually tend to have the least possible number of connectors, to be more cost efficient and also because it might make the motherboard more complicated I would guess.
 

medyej

Member
Oct 26, 2017
6,424
I have never found this to be the case in any shooter I've played with stereo headphones, on consoles or PC.

In something like Rainbow Six Siege, in the variety of setups I've tried with headphones, stereo is the only one that represents the games location audio accurately in a variety of situations. Virtual surround sound into headphones on the other hand distorts that accuracy.

Siege's positional audio is terrible so it's a bad comparison point for 3d audio or virtual surround. If it had normal 3d audio like many other games it'd actually be way better off, but they've done a bunch of specific changes and tricks to try to make audio work better for their destruction but it ends up making it unintuitive as hell and even more messed up when you have virtual surround algorithms on top.

For example if you walk toward a window there is an invisible 'bubble' around it where you get accurate positional sounds, once you get outside of that bubble you'll just hear the sound coming through one specific point even if that person is full on sprinting around the room.
 
Oct 25, 2017
13,246
Siege's positional audio is terrible so it's a bad comparison point for 3d audio or virtual surround. If it had normal 3d audio like many other games it'd actually be way better off, but they've done a bunch of specific changes and tricks to try to make audio work better for their destruction but it ends up making it unintuitive as hell and even more messed up when you have virtual surround algorithms on top.

For example if you walk toward a window there is an invisible 'bubble' around it where you get accurate positional sounds, once you get outside of that bubble you'll just hear the sound coming through one specific point even if that person is full on sprinting around the room.

Be that as it may, my point was largely geared towards stereo vs virtual surround sound and my understanding, with regards to quality and accuracy for headphones is:
3D audio >= Biunaral audio > Stereo > Virtual Surround Sound

Further, it is my understanding that most use cases of Windows Sonic and Dolby Atmos, for headphones, is also virtual surround sound because it attempts to take spatial sound (including height data), made largely for dedicated hardware and attempt to translate that to headphones.

Thus, the focus of the PS5, which is 3D audio geared towards headphones is different than the focus of existing solutions (Windows Sonic, Dolby Atmos), which is taking existing signals made for dedicated speaker setups and translating them to headphones.
 

P40L0

Member
Jun 12, 2018
7,602
Italy
Current Windows Sonic, Dolby Atmos for Headphones and the upcoming DTS: Headphone X v2 already deliver true 3D Spatial Audio with HRTF algorithms (for any Stereo Headphones) on both OG Xbox One, Xbox One X and Windows 10 trough Microsoft owned "Spatial Audio APIs", which already works at a system level with no additional hardware needed.

This is not some "Stereo upmix" basic solution, but all of the above will natively accept true Atmos or 5.1/7.1 audio tracks, process them with HRTF and binaural algorithms and deliver an optimized Stereo final output for the headphones, with accurate surround and also vertical positional audio, all at low latency.

OG Xbox One and Xbox One X also have dedicated audio chip on board to offset these processing from the CPU.

Sony is just catching up current Xbox audio achievements with PS5 (possibly improving it further), while Microsoft is taking what is already there for years now, and upgrading it further.
 
Jun 2, 2019
4,947
Current Windows Sonic, Dolby Atmos for Headphones and the upcoming DTS: Headphone X v2 already deliver true 3D Spatial Audio with HRTF algorithms (for any Stereo Headphones) on both OG Xbox One, Xbox One X and Windows 10 trough Microsoft owned "Spatial Audio APIs", which already works at a system level with no additional hardware needed.

This is not some "Stereo upmix" basic solution, but all of the above will natively accept true Atmos or 5.1/7.1 audio tracks, process them with HRTF and binaural algorithms and deliver an optimized Stereo final output for the headphones, with accurate surround and also vertical positional audio, all at low latency.


OG Xbox One and Xbox One X also have dedicated audio chip on board to offset these processing from the CPU.

Sony is just catching up current achievements with PS5 (possibly improving it further), while Microsoft is taking what is already there for years now, and upgrading it further.

Sony

Catching up on Sound

SONY

I mean, yeah, they shat the bed wih the PS4 sound wise, but this is sony we're talking about.

They're now getting serious on sound, one of their specialities, and when they get serious on Sound they don't catch up, they leave their competitors eating dust
 
Oct 25, 2017
13,246
Current Windows Sonic, Dolby Atmos for Headphones and the upcoming DTS: Headphone X v2 already deliver true 3D Spatial Audio with HRTF algorithms (for any Stereo Headphones) on both OG Xbox One, Xbox One X and Windows 10 trough Microsoft owned "Spatial Audio APIs", which already works at a system level with no additional hardware needed.

This is not some "Stereo upmix" basic solution, but all of the above will natively accept true Atmos or 5.1/7.1 audio tracks, process them with HRTF and binaural algorithms and deliver an optimized Stereo final output for the headphones, with accurate surround and also vertical positional audio, all at low latency.

OG Xbox One and Xbox One X also have dedicated audio chip on board to offset these processing from the CPU.

Sony is just catching up current Xbox audio achievements with PS5 (possibly improving it further), while Microsoft is taking what is already there for years now, and upgrading it further.

What you described is what virtual surround sounds like. Taking a signal made for a dedicated speaker setup, such as in the case of Dolby Atmos, and using processing to fit that signal for headphones. 3D and biaunral audio are different.
 

P40L0

Member
Jun 12, 2018
7,602
Italy
Sony

Catching up on Sound

SONY

I mean, yeah, they shat the bed wih the PS4 sound wise, but this is sony we're talking about.

They're now getting serious on sound, one of their specialities, and when they get serious on Sound they don't catch up, they leave their competitors eating dust
I'm not talking about Sony Audio department in general.
We're specifically talking about Playstation and Xbox audio offerings.

Xbox is objectively capable of all I stated above since OG Xbox One, while Playstation will support all those and more only starting with PS5.

Xbox Series X will just incrementally improve/upgrade what it's already there.
 
Jun 2, 2019
4,947
I'm not talking about Sony Audio department in general.
We're specifically talking about Playstation and Xbox audio offerings.

Xbox is objectively capable of all I stated above since OG Xbox One, while Playstation will support all those and more only starting with PS5.

Xbox Series X will just incrementally improve/upgrade what it's already there.

I would object that that's not Sony getting behind Xbox due to incapability, but sheer greed

15-1.jpg


If Nintendo could get a HRTF solution working on both Wii U and the 3DS of all things, Sony could too, but they had headsets to sell

(Yes, i'm salty for not system wide HRTF on PS4, that's its biggest flaw for me)
 
Oct 25, 2017
13,246
Sony

Catching up on Sound

SONY

I mean, yeah, they shat the bed wih the PS4 sound wise, but this is sony we're talking about.

They're now getting serious on sound, one of their specialities, and when they get serious on Sound they don't catch up, they leave their competitors eating dust

I think people are confusing virtual surround sound options (Windows Sonic), things like Dolby Atmos (which is primarily geared towards speaker setups) and 3D audio.

The three are not the same thing, don't provide the same quality for headphones and have different end goals.

3D audio, which is what Sony's focus is, is not virtual surround sound like Windows Sonic for headphones.
 
Jun 2, 2019
4,947
I think people are confusing virtual surround sound options (Windows Sonic), things like Dolby Atmos (which is primarily geared towards speaker setups) and 3D audio.

The three are not the same thing, don't provide the same quality for headphones and have different end goals.

3D audio, which is what Sony's focus is, is not virtual surround sound like Windows Sonic for headphones.

Windows Sonic, Dolby Atmos for Headset and DTS Headphone:X are all 3D sound solutions (DTS even has a test app for 3D audio)
 

P40L0

Member
Jun 12, 2018
7,602
Italy
What you described is what virtual surround sounds like. Taking a signal made for a dedicated speaker setup, such as in the case of Dolby Atmos, and using processing to fit that signal for headphones. 3D and biaunral audio are different.
3D HRTF/Binaural is exactly that = a special Stereo mix created with HRTF sound recorders (in real life with "actual ears-shaped" microphones) or with HRTF digital algorithms which will convert all sounds for spatial awareness in real time.

In the past this was offered on a game-by-game basis, selecting a specific "Headphones" audio preset (see Hellblade 1 for example), while now sincr Microsoft Spatial Sound APIs and Windows Sonic, this is possible system wide for everything (games and also TV shows and Movies, while Music will be left untouched Stereo) with no devs additional work (a 5.1/7.1 or Atmos regular audio track is sufficient).

Sony will finally do exactly the same thing starting with Tempest on PS5, for sure further improving the amount of objects it can process.

Microsoft will also do the same thing, but incrementally upgrading and unlocking all the features of current Atmos for Headphones and DTS: Headphone X v2 implementations.
 
Oct 25, 2017
13,246
Windows Sonic, Dolby Atmos for Headset and DTS Headphone:X are all 3D sound solutions (DTS even has a test app for 3D audio)

Dolby Atmos works by taking atmos data and converting that into 3D audio using processing algorithm.

That's a bit different than how 3D audio tends to be recorded, which is in a virtual space in direct relation to two ears representing headphones.
 
Oct 25, 2017
13,246
3D HRTF/Binaural is exactly that = a special Stereo mix created with HRTF sound recorders (in real life with "actual ears-shaped" microphones) or with HRTF digital algorithms which will convert all sounds for spatial awareness in real time.

In the past this was offered on a game-by-game basis, selecting a specific "Headphones" audio preset (see Hellblade 1 for example), while now sincr Microsoft Spatial Sound APIs and Windows Sonic, this is possible system wide for everything (games and also TV shows and Movies, while Music will be left untouched Stereo) with no devs additional work (a 5.1/7.1 or Atmos regular audio track is sufficient).

Sony will finally do exactly the same thing starting with Tempest on PS5, for sure further improving the amount of objects it can process.

Microsoft will also do the same thing, but incrementally upgrading and unlocking all the features of current Atmos for Headphones and DTS: Headphone X v2 implementations.

No, I don't believe this is my understanding at all. A 5.1/7.1 signal being down mixed using processing is not what Sony intends to do with Tempest.
 

Ombala

Banned
Oct 30, 2017
2,241
Dolby Atmos works by taking atmos data and converting that into 3D audio using processing algorithm.

That's a bit different than how 3D audio tends to be recorded, which is in a virtual space in direct relation to two ears representing headphones.
I feel like you have explained that this is not the same thing so many times now, ppl don't seem to understand, or don't want to.
 

P40L0

Member
Jun 12, 2018
7,602
Italy
No, I don't believe this is my understanding at all. A 5.1/7.1 signal being down mixed using processing is not what Sony intends to do with Tempest.
There is no "downmix" as the regular therm of the word.

With current Atmos for Headphones (to be not mistaken for regular Dolby Atmos for Home Theather setup) and DTS: Headphone X v2, by natively taking 5.1/7.1 and true Atmos from games and movies, the final Stereo output you will get will be processed in real-time.

So this means you will actually and accurately hear sounds around you, above and below you, in real-time.

It's really a mind-blowing tech, especially considering there's no external hardware need and it will also works totally wirelessly on Xbox thanks to the low latency Microsoft Xbox Wireless protocol (which is miles ahead better than Bluetooth).

You can even try it yourself with the "DTS: Sound Unbound" app for any Windows 10 PC (Windows Store) and ANY pair of Stereo headphones.

There is a Test page where you dynamically interact moving sound objects around and above/below you in the room (so no pre-recorded tracks) and you will listen how your regular Stereo headphones will be completely transformed thanks to the DTS 3D HRTF Audio.

These things are already here and already works great. ;)
 
Oct 25, 2017
13,246
There is now "downmix" as the regular therm of the word.

With current Atmos for Headphones (to be not mistaken for regular Dolby Atmos for Home Theather setup) and DTS: Headphone X v2, by natively taking 5.1/7.1 and true Atmos from games and movies, the final Stereo output you will get will be processed in real-time.

So this means you will actually and accurately hear sounds around you, above and below you, in real-time.

It's really a mind-blowing tech, especially considering there's no external hardware need and it will also works totally wirelessly on Xbox thanks to the low latency Microsoft Xbox Wireless protocol (which is miles ahead better than Bluetooth).

You can even try it yourself with the "DTS: Sound Unbound" app for any Windows 10 PC (Windows Store) and ANY pair of Stereo headphones.

There is a Test page where you dynamically interact moving sound objects around and above/below you in the room (so no pre-recorded tracks) and you will listen how your regular Stereo headphones will be completely transformed thanks to the DTS 3D HRTF Audio.

These things are already here and already works great. ;)

I have used it and I think you are missing the point.

3D audio is not recorded the same way as taking a 5.1/7.1 signal and applying a proprietary audio process to it. That's virtual surround sound in essence, sometimes more immersive but not necessarily more accurate for headphones.

What you are describing and what Sony described with Tempest are two different things.
 

P40L0

Member
Jun 12, 2018
7,602
Italy
I have used it and I think you are missing the point.

3D audio is not recorded the same way as taking a 5.1/7.1 signal and applying a proprietary processing process to it. That's virtual surround sound in essence, sometimes more immersive but not necessarily more accurate for headphones.

What you are describing and what Sony described with Tempest are two different things.
I said by taking 5.1/7.1...and true Atmos :)

As I said Atmos for Headphones and DTS:H X v2 are NOT just "virtual surround" downmixes, but are 3D/HRTF, real-time, object based audio processing in real-time for Stereo Headphones.

Both are already system wide since OG Xbox One thanks to Microsoft "Spatial Audio" APIs.

Tempest will just be the "Spatial Audio" APIs alternative for Playstation.

There's not so much left to understand I guess.
 
Oct 25, 2017
13,246
I said by taking 5.1/7.1...and true Atmos :)

As I said Atmos for Headphones and DTS:H X v2 are NOT just "virtual surround" downmixes, but are 3D/HRTF, real-time, object based audio processing in real-time for Stereo Headphones.

Both are already system wide since OG Xbox One thanks to Microsoft "Spatial Audio" APIs.

Tempest will just be the "Spatial Audio" APIs alternative for Playstation.

There's not so much left to understand I guess.

Ah okay. I kept getting confused because 5.1/7.1 kept being mentioned.

Yea, for the 3D stuff, I recall that games need to provide that specific spatial information otherwise it is using the "down mix" functionality (E. G some games have that functionality atm, where as others don't and rely on the surround sound being processed for headphones).

Tempest isn't an alternative to the 5.1/7.1 stuff, it's strictly Sony's upgrade of the latter.
 

Pargon

Member
Oct 27, 2017
11,996
I think people are confusing virtual surround sound options (Windows Sonic), things like Dolby Atmos (which is primarily geared towards speaker setups) and 3D audio.
The three are not the same thing, don't provide the same quality for headphones and have different end goals.
3D audio, which is what Sony's focus is, is not virtual surround sound like Windows Sonic for headphones.
Windows Sonic, Dolby Atmos for Headphones, and DTS Headphone:X all support 3D Audio via the Spatial Sound API.
That's real 3D Audio with height information; but games have to support this.
I believe this is an incomplete list, but the number of games which support Spatial Sound is relatively small.
  • Assassin's Creed Origins
  • Battlefield 1
  • Borderlands 3
  • Call of Duty Modern Warfare
  • Final Fantasy XV
  • For Honor
  • Forza Horizon 4
  • Gears 4
  • Gears 5
  • Mass Effect: Andromeda
  • Metro Exodus
  • Overwatch
  • Resident Evil 2
  • Rise of the Tomb Raider
  • Super Lucky's Tale
  • Shadow of the Tomb Raider
  • The Division 2
  • Zone of the Enders: The 2nd Runner M∀RS
I thought I remembered Watch Dogs 2 being listed as supporting Atmos, but I don't see it listed anywhere now, and see a number of posts with people saying that they're having difficulty getting it to work.

Sonic/Atmos/DTS:X also support virtual surround in games that only have multichannel (5.1/7.1) audio support rather than Spatial Sound support.
On Windows, you can also get 3D audio in older games that use DirectSound3D HW (mid '90s to ~2008) if you have an X-Fi sound card, via its CMSS-3D support.

Be that as it may, my point was largely geared towards stereo vs virtual surround sound and my understanding, with regards to quality and accuracy for headphones is:
3D audio >= Biunaral audio > Stereo > Virtual Surround Sound

Further, it is my understanding that most use cases of Windows Sonic and Dolby Atmos, for headphones, is also virtual surround sound because it attempts to take spatial sound (including height data), made largely for dedicated hardware and attempt to translate that to headphones.

Thus, the focus of the PS5, which is 3D audio geared towards headphones is different than the focus of existing solutions (Windows Sonic, Dolby Atmos), which is taking existing signals made for dedicated speaker setups and translating them to headphones.
This is not correct at all.
  • 3D Audio is a form of Virtual Surround.
  • 3D Audio and Virtual Surround are both types of processing to create Binaural audio.
The easiest way to describe it is to call them 2D Virtual Surround and 3D Virtual Surround.
Stereo pans sounds left and right. When you're wearing headphones, it's like the sound goes right through the middle of your head, and there are hard cuts when a sound moves from the left to the right side.

stereo2vj5m.png



2D Virtual Surround only has flat directional information.
It takes the sound "out of your head" and can position sounds in a ring around you:

vssxekqk.png

(technically, that should be a filled-in circle, but it's easier to represent like this)

How accurately it does that depends on the processing and the source.
If it's a game using DirectSound3D it has access to 3D object positions and can place them very well.
Most 2D virtual surround sound processing is derived from a 5.1 or 7.1 audio signal though, so the positional information is not as accurate.
While that may not be as accurate, it's still far more accurate than stereo. If you want to simplify things, a 7.1 system could be considered as having seven stereo pairs of speakers it can use to position audio.


3D Virtual surround can position audio in a sphere around your head:

3daudio2kkv2.png


Instead of a flat ring of sound around the player, height information is now included to create "3D Audio".
This may be achieved via 3D positional information, as is the case for DirectSound3D HW+CMSS-3D, Microsoft's Spatial Audio API, or Sony's Tempest Engine.
Or it can be done by downmixing discrete channel formats like an Atmos 7.2.4 mix which has four height channels.
But it's still virtual surround, and all virtual surround technologies are trying to create a Binaural audio source.


"Binaural audio" as it's commonly known, is generally referring to pre-recorded audio tracks, which were recorded using a pair of microphones, a specific binaural microphone with ears, or an entire dummy head.
d61e73d85a51a1048fc7784keb.jpg
g905ahazpj89.jpg


This is the "easy" way to create binaural audio, since it doesn't require any fancy processing.
But you can't really do anything to reposition the sound after the fact; you can only play it back as-is.
So it works well in a game like Hellblade where they use it as an effect to have voices coming from all kinds of directions around your head, but not during gameplay where you have camera control and sounds have to change position in real-time.
And the HRTF is baked-in, since you've recorded it with a specific set of ears. It still has the problem of not working nearly as well if your ears are too far from that model; though I think the baseline effectiveness is likely to still be higher than an "incompatible" virtual surround.

All virtual surround is trying to essentially mix the audio to recreate the experience of these binaural recordings in real-time.
Since that's a very complex thing, and since it depends on the listener, it only works to varying degrees of effectiveness. That's why many people have their own preferences for which virtual surround tech they think sounds best.

Maybe Sony's Tempest Engine will be the most effective implementation yet, since they're planning to offer multiple HRTFs and are throwing a lot of audio processing capabilities behind it; but it's still virtual surround.
In some ways, I do wonder if the bigger difference is actually going to be the complexity of the audio simulation being done to enrich the soundscape, than their virtual surround tech.
Maybe CMSS-3D's virtual surround was perfectly matched to your ears and the positional information was completely accurate. Things still wouldn't have sounded real because audio design and simulation of the acoustic environment back in 2005 was far more simplistic. That's where the ability to process hundreds of sounds at once while using ray-tracing comes in.
 

medyej

Member
Oct 26, 2017
6,424
Windows Sonic, Dolby Atmos for Headphones, and DTS Headphone:X all support 3D Audio via the Spatial Sound API.
That's real 3D Audio with height information; but games have to support this.
I believe this is an incomplete list, but the number of games which support Spatial Sound is relatively small.
  • Assassin's Creed Origins
  • Battlefield 1
  • Borderlands 3
  • Call of Duty Modern Warfare
  • Final Fantasy XV
  • For Honor
  • Forza Horizon 4
  • Gears 4
  • Gears 5
  • Mass Effect: Andromeda
  • Metro Exodus
  • Overwatch
  • Resident Evil 2
  • Rise of the Tomb Raider
  • Super Lucky's Tale
  • Shadow of the Tomb Raider
  • The Division 2
  • Zone of the Enders: The 2nd Runner M∀RS
I thought I remembered Watch Dogs 2 being listed as supporting Atmos, but I don't see it listed anywhere now, and see a number of posts with people saying that they're having difficulty getting it to work.

Sonic/Atmos/DTS:X also support virtual surround in games that only have multichannel (5.1/7.1) audio support rather than Spatial Sound support.
On Windows, you can also get 3D audio in older games that use DirectSound3D HW (mid '90s to ~2008) if you have an X-Fi sound card, via its CMSS-3D support.


This is not correct at all.
  • 3D Audio is a form of Virtual Surround.
  • 3D Audio and Virtual Surround are both types of processing to create Binaural audio.
The easiest way to describe it is to call them 2D Virtual Surround and 3D Virtual Surround.
Stereo pans sounds left and right. When you're wearing headphones, it's like the sound goes right through the middle of your head, and there are hard cuts when a sound moves from the left to the right side.

stereo2vj5m.png



2D Virtual Surround only has flat directional information.
It takes the sound "out of your head" and can position sounds in a ring around you:

vssxekqk.png

(technically, that should be a filled-in circle, but it's easier to represent like this)

How accurately it does that depends on the processing and the source.
If it's a game using DirectSound3D it has access to 3D object positions and can place them very well.
Most 2D virtual surround sound processing is derived from a 5.1 or 7.1 audio signal though, so the positional information is not as accurate.
While that may not be as accurate, it's still far more accurate than stereo. If you want to simplify things, a 7.1 system could be considered as having seven stereo pairs of speakers it can use to position audio.


3D Virtual surround can position audio in a sphere around your head:

3daudio2kkv2.png


Instead of a flat ring of sound around the player, height information is now included to create "3D Audio".
This may be achieved via 3D positional information, as is the case for DirectSound3D HW+CMSS-3D, Microsoft's Spatial Audio API, or Sony's Tempest Engine.
Or it can be done by downmixing discrete channel formats like an Atmos 7.2.4 mix which has four height channels.
But it's still virtual surround, and all virtual surround technologies are trying to create a Binaural audio source.


"Binaural audio" as it's commonly known, is generally referring to pre-recorded audio tracks, which were recorded using a pair of microphones, a specific binaural microphone with ears, or an entire dummy head.
d61e73d85a51a1048fc7784keb.jpg
g905ahazpj89.jpg


This is the "easy" way to create binaural audio, since it doesn't require any fancy processing.
But you can't really do anything to reposition the sound after the fact; you can only play it back as-is.
So it works well in a game like Hellblade where they use it as an effect to have voices coming from all kinds of directions around your head, but not during gameplay where you have camera control and sounds have to change position in real-time.
And the HRTF is baked-in, since you've recorded it with a specific set of ears. It still has the problem of not working nearly as well if your ears are too far from that model; though I think the baseline effectiveness is likely to still be higher than an "incompatible" virtual surround.

All virtual surround is trying to essentially mix the audio to recreate the experience of these binaural recordings in real-time.
Since that's a very complex thing, and since it depends on the listener, it only works to varying degrees of effectiveness. That's why many people have their own preferences for which virtual surround tech they think sounds best.

Maybe Sony's Tempest Engine will be the most effective implementation yet, since they're planning to offer multiple HRTFs and are throwing a lot of audio processing capabilities behind it; but it's still virtual surround.
In some ways, I do wonder if the bigger difference is actually going to be the complexity of the audio simulation being done to enrich the soundscape, than their virtual surround tech.
Maybe CMSS-3D's virtual surround was perfectly matched to your ears and the positional information was completely accurate. Things still wouldn't have sounded real because audio design and simulation of the acoustic environment back in 2005 was far more simplistic. That's where the ability to process hundreds of sounds at once while using ray-tracing comes in.

Great explanation!
Do you know if Atmos and DTS-X now on Win10 have fixed the issue where games are just sending stereo (for not actual Spacial Sound suppoted games) because Atmos/DTS-X was setting the windows sound config to Stereo? That was my big issue with Atmos before but if they fixed it I'm willing to give it another try.
 

Pargon

Member
Oct 27, 2017
11,996
Great explanation!
Do you know if Atmos and DTS-X now on Win10 have fixed the issue where games are just sending stereo (for not actual Spacial Sound suppoted games) because Atmos/DTS-X was setting the windows sound config to Stereo? That was my big issue with Atmos before but if they fixed it I'm willing to give it another try.
They seem to have improved it, but it still doesn't work in every game.
Someone in the DTS thread linked to this XAudio2 patch, which is supposed to fix the problem, but I'm very hesitant to modify/replace system DLL files.

I have been meaning to test placing a copy of the file in the game directory and modifying that, rather than the system file, to see if it works.
 

SmashN'Grab

Member
Oct 27, 2017
525
This stuff is amazing - I'm very excited for this - the current generation is the first generation where I've played using headphones - thanks to the controllers featuring the ability to plug in a pair of headphones - and it's a game-changer, way better than using tv speakers (obviously) but probably even better than a surround-sound system.

I've even taken to watching movies on netflix or blu-ray with my headphones plugged into the controller. It's great. Probably not the absolute ideal set up, I might look for a better pair of wireless headphones for next-gen.

Very excited for the increased focus on audio in PS5, hopefully the XSX will keep up.
 

DrDeckard

Banned
Oct 25, 2017
8,109
UK
Sony's main aim seems to be ensuring 3D audio is supported at a system level and creating rendering & virtualization systems that make it possible for anyone to experience it without the need for additional hardware, which isn't the case on PS4. They've dedicated considerable power to audio to ensure that bit-depth and sample rates stay high as we scale up not only the amount of sound sources but the environmental responses for each individual sound.



Yes and no.


I'm aware...

This is a bit more involved than just having a DSP.

So, basically Sony is stepping up to make sure the PS5 is offering dedicated hardware for sound. Like MS did with xbox one and now series X.
 

deadmonkeyuk

Member
Oct 27, 2017
1,216
Highlands, Scotland
This stuff is amazing - I'm very excited for this - the current generation is the first generation where I've played using headphones - thanks to the controllers featuring the ability to plug in a pair of headphones - and it's a game-changer, way better than using tv speakers (obviously) but probably even better than a surround-sound system.

I've even taken to watching movies on netflix or blu-ray with my headphones plugged into the controller. It's great. Probably not the absolute ideal set up, I might look for a better pair of wireless headphones for next-gen.

Very excited for the increased focus on audio in PS5, hopefully the XSX will keep up.
Same, Audio improvements are right up there in what I am most excited to experience next gen.

I have mainly used headphones for gaming over the last twenty years, even when I had a good 5.1 system setup.

I often for convenience plug my headphones into the controller, but sound is so much better wired. Even plugging your headphones into your TV will give you a better audio experience. Best thing I did was buy quality headphones and a headphone amp.

My advice is always get yourself a quality set of wired headphones, I have never been a big fan of wireless headphones or gaming headsets.

Current headphones are Philips X1s, have had them for about 5 years and use them every day.