If you’re a frequent gamer, you’ve heard of ray tracing and NVIDIAs by now DLSS (Deep Learning Super Sampling), two powerful tools that together improve performance in games and provide stunning visual fidelity by replicating realistic lighting and reflections. NVIDIA pioneered gaming with the introduction of the RTX 20 series graphics cards, and in the two hardware generations since then, the techniques and AI hardware behind them have improved significantly. With DLSS 3.5 and a new Ray Reconstruction system working hand in hand, ray-traced games can look all the more lifelike while running significantly smoother. How do these technologies work?
DLSS is an evolving technology. Early on, the focus was on rendering games at a lower, easier-to-process resolution and then increasing the output resolution by filling in the gaps between pixels, giving gamers the benefit of sharper visuals at higher frame rates in a lower resolution. NVIDIA accomplished this by training its AI model on high-quality game footage so it could understand what they should look like and know how to fill in the gaps when stretching a game’s lower-resolution frames to higher resolutions. This process could also be reversed somewhat with DLAA (Deep Learning Anti-aliasing), which can display at a display’s native resolution but uses the same AI logic to figure out what an even higher resolution frame would look like and then downsample that to output an effectively anti-aliased image.
With DLSS 3, it introduced Frame Generation, which fills in pixel data between frames. While DLSS has always relied on the dedicated Tensor Cores in RTX GPUs, which handle AI operations, DLSS 3 Frame Generation leverages even more hardware in the RTX 40 series GPUs, such as Optical Flow Accelerators, to understand how objects in games move and move intelligently. blend between frames for extra smooth images. By combining the supersampling and Frame Generation technology in DLSS, frame rates in games can be dramatically increased without sacrificing visual quality.
In addition to these developments in DLSS, NVIDIA has continued to advance its RTX ray tracing technology. Ray tracing remains a computationally intensive process, especially to obtain the most realistic results. It requires simulating a staggering amount of light rays and all the reflections they make between a light source and the viewer, making it unreasonable to do this for every pixel in every frame of a fast-paced game. But taking a reasonable number of light beam samples produced a noisy image and required specially trained denoisers to get a usable image. Even then, the results may lack detail or exhibit unusual ghosting, and the more denoisers needed to handle numerous ray-traced effects can further slow down performance.
NVIDIAs DLSS 3.5 Ray introduced Reconstruct to solve this. With Ray Reconstruction, all denoisers (and associated computing needs) are replaced. Just as DLSS and Frame Generation use AI to figure out how to fill in pixels intelligently, Ray Reconstruction uses AI to fill in the gaps between simulated light rays. Ray Reconstruction has a better understanding of the games it’s running and their ray-traced effects, allowing it to know when to use different techniques to fill in missing data, allowing it to provide sharp images without artifacts.
For gamers, higher resolutions and higher graphics settings used to mean making big sacrifices in framerate. But the technologies incorporated into DLSS 3.5, such as Frame Generation and Ray Reconstruction, flip the script.
Now gamers can experience lifelike worlds with ray-traced lighting, shadows and reflections while enjoying higher resolutions on any RTX GPU. And those gaming on RTX 40 series graphics processors can take advantage of Frame Generation for another frame rate boost. Taking advantage of these technologies is as simple as running an NVIDIA RTX graphics processor on your desktop or laptop (or even taking advantage of it through NVIDIA’s GeForce NOW Ultimate or Priority service), and making sure you get these features in your games switches on. You can find all games and apps that support this NVIDIA’s DLSS technologies here.
The way NVIDIA has used AI to improve the gaming experience is just one way the company is putting AI to work. With NVIDIA’s RTX hardware you can do much more than just gaming NVIDIA’s AI Decoded blog series highlights many of the different ways you can take advantage of your RTX hardware with all kinds of AI-powered tools.