Nvidia's Marbles RTX Hinting at Tensor Core Denoising?

[ILikeFeet]

senior art director at Nvidia, Gavriil Klimov, posted some high resolution screens of their Marbles RTX project; and in his description, he states:

It's a fully playable game that is ENTIRELY ray-traced, denoised by NVIDIA's AI and DLSS, and obeys the laws of physics - all running in real-time on a single RTX GPU. Absolutely mind-blowing.

Click to shrink...

so far, Nvidia only uses the tensor cores for DLSS and DLSS doesn't provide any denoising, just upscaling, sharpening, and anti-aliasing. I think it was in Digital Foundry's interview that Nvidia said they were working on using the tensor cores for denoising, so could this be the first showing of it?

for those that don't know, denoising is pretty resource heavy. in some instances even heavier than ray tracing itself. finding a way to speed up denoising and increasing the quality would be a major component in making RT a more viable tool for rendering

also images of Marbles RTX cause it's pretty

https://www.artstation.com/artwork/e0Eyv6

 

[R dott B]

Wow those shots are great.

 

[GaussianMultivariate]

Let's just give these people a couple years and we'll get something out of this world

 

[Deleted member 12317]

In Blender 2.82 you can use an RTX card's AI to denoise renders when using the Cycles renderer with OptiX (using RT Cores to render), and in 2.83 beta you can also denoise the viewport in realtime when previewing in Cycles.

In the 2.83 beta it's quite useful to be able to preview renders without noise really quickly, and it works really well on renders also, you can save some time by rendering with less samples by pixels and let the AI denoise it.

Cycles - Blender Developer Documentation

wiki.blender.org

Cycles - Blender Developer Documentation

wiki.blender.org

 

[ILikeFeet]

In Blender 2.82 you can use an RTX card's AI to denoise renders when using the Cycles renderer with OptiX (using RT Cores to render), and in 2.83 beta you can also denoise the viewport in realtime when previewing in Cycles.

In the 2.83 beta it's quite useful to be able to preview renders without noise really quickly, and it works really well on renders also, you can save some time by rendering with less samples by pixels and let the AI denoise it.

Cycles - Blender Developer Documentation

wiki.blender.org

Cycles - Blender Developer Documentation

wiki.blender.org

Click to shrink...

it uses AI, but it runs denoising on shader cores like everything else. RT cores don't do rendering

 

[KKRT]

Yeah, we need to this ASAP :) It will be such a boost to RT performance based on Minecraft frame time distribution.

I think it was in Digital Foundry's interview that Nvidia said they were working on using the tensor cores for denoising, so could this be the first showing of it?

Click to shrink...

I'm pretty sure that Nvidia already talked about it on RTX reveal.

--edit--
Even earlier it seems:

NVIDIA OptiX™ AI-Accelerated Denoiser

developer.nvidia.com

--

So why OptiX can only runs on RTX cards?
It states here : https://wiki.blender.org/wiki/Reference/Release_Notes/2.81/Cycles#NVIDIA_RTX


What I understand is that it uses the RT Cores to render, and it's quite faster than CUDA on the same card.
They don't tell how the denoiser run, but they say you need an RTX card and use OptiX, thus it should use some RTX specific hardware.

If it only runs on shader cores, why limiting it to RTX cards ? (and OptiX when using an RTX one)

Click to shrink...

Optix can use Tensor cores. From my link above:
" The OptiX AI denoising technology, combined with the new NVIDIA Tensor Cores in the Quadro GV100, delivers 3x the performance of previous-generation GPUs and enables fluid interactivity in complex scenes. "

 

[Deleted member 12317]

it uses AI, but it runs denoising on shader cores like everything else. RT cores don't do rendering

Click to shrink...

So why OptiX can only runs on RTX cards?
It states here : https://wiki.blender.org/wiki/Reference/Release_Notes/2.81/Cycles#NVIDIA_RTX

Cycles now has experimental support for rendering with hardware-accelerated raytracing on NVIDIA RTX graphics cards. To use, enable OptiX

Click to shrink...

What I understand is that it uses the RT Cores to render, and it's quite faster than CUDA on the same card.
They don't tell how the denoiser run, but they say you need an RTX card and use OptiX, thus it should use some RTX specific hardware.

If it only runs on shader cores, why limiting it to RTX cards ? (and OptiX when using an RTX one)

 

[ILikeFeet]

So why OptiX can only runs on RTX cards?
It states here : https://wiki.blender.org/wiki/Reference/Release_Notes/2.81/Cycles#NVIDIA_RTX


What I understand is that it uses the RT Cores to render, and it's quite faster than CUDA on the same card.
They don't tell how the denoiser run, but they say you need an RTX card and use OptiX, thus it should use some RTX specific hardware.

If it only runs on shader cores, why limiting it to RTX cards ? (and OptiX when using an RTX one)

Click to shrink...

ask Nvidia, ain't the first time they have arbitrary limitations; just see the whole deal with Pascal and RT. and RT Cores don't render, they're for accelerating BVH traversal and intersection testing. the shader cores do the rendering. same with games

 

[Deleted member 11276]

Very good catch, I didn't notice that. It really seems like AI denoising on the tensor cores for games will be ready soon.

I suppose the AI training took a long time to get good results, similar to DLSS 2.0.

 

[EVIL]

Here are some more links on artstation to works for this demo:

https://www.artstation.com/artwork/KaXZdG

https://www.artstation.com/artwork/KaXV9r

https://www.artstation.com/artwork/aRv4Q8

https://www.artstation.com/artwork/QzdV83

https://www.artstation.com/artwork/3o644g

 

[Zaimokuza]

In this video Eric Haines says Nvidia is very interested in denoising as it dramatically reduces the number of rays that need to be traced.

Edit: grammar

Ray Tracing Essentials Part 7: Denoising for Ray Tracing

In the final video of the series: NVIDIA’s Eric Haines describes the process of denoising for ray tracing. A critical element in making realistic, high-quali...

www.youtube.com

 

[Deleted member 11276]

In this video Eric Haines says Nvidia is very interested in denoising as it dramatically reduces the number of rays that need to be traced.

Edit: grammar

Ray Tracing Essentials Part 7: Denoising for Ray Tracing

In the final video of the series: NVIDIA’s Eric Haines describes the process of denoising for ray tracing. A critical element in making realistic, high-quali...

www.youtube.com

Click to shrink...

Yep. A significant speedup in RT performance could happen with AI denoising for two reasons:

1. Currently denoising is done on the shader cores, meaning the GPU has less power to render the graphics itself. By offloading that task to the tensor cores, the ALUs are freed up, so they can invest more rendering power to the graphics pipeline, resulting in significantly higher framerates.

2. A well trained AI denoiser could potentially be much more efficient and better at denoising than a shader filter, meaning less rays have to be created which significantly reduces load on both, RT and shader cores.

This is going to be good. I hope it's implemented in a game soon.

 

[ILikeFeet]

Here are some more links on artstation to works for this demo:

https://www.artstation.com/artwork/KaXZdG

https://www.artstation.com/artwork/KaXV9r

https://www.artstation.com/artwork/aRv4Q8

https://www.artstation.com/artwork/QzdV83

https://www.artstation.com/artwork/3o644g

Click to shrink...

they need to release this on steam