DLSS and DLAA represent groundbreaking image enhancement technologies leveraging AI and deep learning to either boost gaming performance or achieve new levels of visual fidelity. This comprehensive technical guide will dive deep on how these technologies work, key perf and quality considerations, and specific recommendations for choosing the best option for YOUR games and setup.
Contents
- What is DLSS and DLAA? A Technical Explanation
- How Games Integrate and Use DLSS and DLAA
- Technical Improvements and Optimizations in Latest Versions
- Image Quality and Performance Considerations
- Frame Generation with DLSS 3
- VRAM Usage Comparisons
- Input Latency Benchmarks
- Choosing For Individual Games
- Genre, Engine, GPU Architecture Specific Advice
- Showcasing More Examples Across New Releases
- Closing Advice
- FAQs
What is DLSS and DLAA? A Technical Explanation
DLSS and DLAA belong to a class of graphics techniques called spatially scalable image synthesis. The goal is to increase either visual fidelity or performance/efficiency related to pixel workloads by leveraging intelligence and data from neural networks.
Both techniques work on the same fundamental principle – using AI to enhance lower resolution rendered frames to match the quality of higher resolution frames, thereby reducing costs related to actually rendering at that higher native resolution.
DLSS (Deep Learning Super Sampling) focuses more on boosted performance and efficiency by rendering frames internally at a lower resolution, then using AI to upscale those frames back up to the target output resolution. With the effectively lighter per-frame workload, DLSS allows considerable performance gains.
DLAA (Deep Learning Anti-Aliasing) focuses purely on further improving the maximum achievable image quality at a game‘s native rendering resolution. It uses AI to apply specialized anti aliasing and additional enhancements that smooth out edges, textures, shimmering, pixelation etc. beyond what traditional AA solutions can achieve.
Now let‘s look under the hood to truly understand how this works.
Neural Network Training
The "magic" behind both DLSS and DLAA lies in Nvidia designing specialized deep convolutional neural networks to excel specifically at these graphics-related enhancement tasks.
These networks are trained over thousands of hours on extremely powerful cloud TPU clusters using tens of thousands of high-quality example game image pairs:
- For DLSS, pairs consist of a high resolution frame and a lower resolution frame from same scene
- For DLAA, pairs contain image with and without anti-aliasing from same scene
By exposing the network to such diverse range of examples during training, over millions of iterations it learns to recognize key visual patterns, edges, textures, objects that characterize perceived high image quality in a rendered game scene at target resolution or with AA applied.
Just as crucially, the network also learns to predict and fill in missing pixel data that occurs when going from lower res to higher res. It understands textural structures, patterns and can plausibly simulate more detailed imagery in upscaled frames.
The end result is a network capable of intelligently manipulating lower res input frames to recover extremely high quality, convincing 4K (or other target) resolution images for DLSS or add sophisticated AA that outperforms even the best brute-force algorithmic methods.
And this processing happens in real-time, on the fly as you game, thanks to the exceptional computational efficiency of these networks when run on Tensor core equipped Nvidia RTX GPUs.
Both networks continue retraining over time using temporal data like motion vectors and game telemetry to keep up with new rendering techniques, textures etc. This allows image quality and compatibility to keep improving.
DLSS 3 Enables AI Frame Generation
The latest major evolution in DLSS technology is the release of DLSS 3 alongside the new GeForce RTX 40 series GPUs based on the Ada Lovelace architecture.
DLSS 3 incorporates all existing DLSS 2 temporal upscaling features for super sharp, artifact-free image quality.
But it adds an additional breakthrough capability dubbed Frame Generation.
Frame Generation effectively boosts FPS beyond what would otherwise be possible by having the DLSS 3 network synthesize entirely new frames without direct corresponding inputs from the game engine.
It works by using prior rendered frames along with optical flow motion vectors to predict forthcoming frames. This means smoother, more responsive input lag as well as much higher FPS ceilings to take best advantage of high refresh rate monitors.
Initial testing shows 2-4x performance gains in supported games which is incredible, putting high fidelity 4K 144+ FPS gaming now firmly within reach even on mid-range cards like the RTX 4070.
However, Frame Generation can introduce new artifacts in challenging scenes with complex lighting, reflections or camera movement. Nvidia is continuously further training their networks to improve reproduction quality over time.
Overall DLSS 3 represents a monumental leap, boosting both performance as well as bettering fluidity and input latency thanks to Frame Gen being synchronized with new Reflex technologies for minimizing rendering queues.
How Games Integrate and Use DLSS and DLAA
Of course, all these advanced deep learning networks mean nothing if games can‘t actually leverage DLSS and DLAA effectively. Seamless integration with modern game engines is crucial.
Thankfully, Nvidia provides game developers extensive support to achieve great end results across Unreal Engine 4 & 5, Unity, id Tech, REDEngine and more.
Optimal Training Data Generation
Firstly, Nvidia works closely with devs to generate optimal representative training data paired sets covering diverse portions of game worlds.
Special focus is given to complex edges, textures, transparency/particle effects and other areas previous DLSS integrations have struggled with.
Insight Into Model Architectures
Nvidia also provides valuable insight into current DLSS model architectures, their strengths and weaknesses, and how they can best be invoked given scene rendering characteristics and desired enhancement goals.
This allows tailoring model choice, config and sequencing priority for minimizing artifacts while maximizing either efficiency (DLSS) or effective AA coverage (DLAA).
Additionally there are extensive compatibility guides across GPU generations (Turing, Ampere etc.) to achieve consistent quality.
Advanced Integration Options
For developers seeking maximum quality, Nvidia exposes several advanced integration options including:
- Custom network training for game specific artifact issues
- Multiple model chaining – run output through additional models specialising in key areas like shimmer reduction
- Integration with rendering pipeline – leverage scene geometry, buffers, shader output for better spatiotemporal analysis
When combined together, these cutting edge techniques allow creators incredible flexibility achieving new heights of visual splendor in titles like Cyberpunk 2077 while retaining great FPS.
Technical Improvements and Optimizations in Latest Versions
Both DLSS and DLAA see constant improvement and optimization with every new iteration that further boosts image fidelity, compatibility across scenes, and efficiency.
Temporal Data Usage
One huge area leveraged in the latest DLSS 2.3 and DLAA 3.5 versions is greater use of temporal data – essentially using information derived from previous frames when enhancing the current frame.
Because adjacent frames have significant visual coherence in areas like motion, textures etc. temporal approaches greatly aid the networks‘ spatiotemporal perception.
This manifests as reduced shimmering and artifacting in DLSS upscaling as well as faster, higher quality anti-aliasing via DLAA.
Specific temporal techniques integrated now include:
- Motion vectors
- Previous frame feature maps
- Reprojection warping
- Render queue feedback
Each approach contributes valuable data that improves stability and accuracy of frame enhancements. Nvidia also ensures temporal techniques operate at millisecond scales to avoid introducing perceived lag during gameplay.
Reduced Model Latency
Extensive focus has also been spent on optimizing model architectures and pruningfilters that don‘t meaningfully impact end results. This allows reducing overall model latency to just 2-9 ms – ensuring DLSS and DLAA pipelines don‘t become a performance bottleneck.
Specialised Model Training
Additionally Nvidia trains separate specialised models targeted at efficiency (performance-focused) and image quality respectively depending on the end goal. This allows tailoring to better fit the exact usage context.
So not only is core algorithm quality drastically improving over time, but Nvidia is also making under-the-hood optimizations to further minimize overheads and keep input-to-output latency low.
Image Quality and Performance Considerations
Now that we understand exactly how DLSS and DLAA function at a deeper technical level, let‘s analyze resultant image quality and performance tradeoffs you can expect across supported gaming setups.
VRAM Usage Comparison
One area many gamers considering DLSS are curious about is potential differences in VRAM usage compared to native resolution rendering.
After all DLSS processes the game at a lower resolution before upscaling – so does that mean it requires less VRAM?
The answer is no, DLSS does NOT reduce VRAM requirements vs native rendering. In terms of GPU memory costs it is nearly equivalent.
This is because while the input frames rendered by the game engine may be at a lower res, the final output frames that get displayed and cached still end up at your target resolution – whether 1440p, 4K etc.
DLSS only reduces the per-frame processing loads on the rest of the GPU pipeline, not memory costs.
So don‘t look at DLSS as a means to run better texture qualities or heavier mods packs that ordinarily overload your VRAM capacity at certain resolutions. You‘ll run into the same limitations.
However, because DLSS greatly increases usable FPS headroom, you can potentially dial down some VRAM heavy settings specifically to avoid bottlenecks there while retaining improved gameplay fluidity thanks to the performance boost.
Input Latency Benefits
A clear benefit DLSS does provide relative to native rendering is much improved input-to-display latency – crucial for competitive gaming.
By producing rendered frames much faster due to its efficiency gains from lower internal rendering res, DLSS inherently allows earlier start on frame processing.
So not only do you get much higher FPS, those frames themselves get delivered faster from time you provide input.
Compared to native rendering, DLSS cuts input latency by up to 50%. This translates to a faster, highly responsive feel in gameplay.
DLAA meanwhile adds roughly 1 frame of processing time, so expect slightly elevated input lag. But it still remains very low thanks to highly optimized model execution on Tensor cores.
Here‘s a look at some real-world input latency benchmarks:
As you can see, DLSS has clear advantages here. Particularly DLSS 3 with Frame Generation further compresses the input latency pipeline.
Choosing For Individual Games
Now let‘s build on the existing game specific analysis from the first version and cover even more titles across a wider variety of engines and release dates.
Genre and Engine Specific Behavior
An important insight regarding DLSS and DLAA effectiveness in certain games relates to inherent engine and genre specific rendering characteristics.
For example, competitive FPS games emphasize fast response and high FPS over all else. Scenes are largely static arena maps with short draw distances. This lends particularly well to DLSS with lower input latency and maximized fluidity.
Compare that to open world adventure games full of erratic camera movement, sweeping vistas, particles and transparency effects. Those complex, temporal scenes expose flaws with DLSS much more leading to visible artifacting. Yet fast pacing means DLAA‘s heavy costs are also unattractive.
Or cinematic story games like Hellblade 2 which use UE5 Nanite and Lumen achieve unprecedented levels of geometry, lighting realism. There DLAA for absolute max IQ could make sense for owners playing at 4K120.
The point is some general genre and engine specific guidelines exist when deciding between native rendering, DLAA or different DLSS modes.
Let‘s see how these manifest across more examples:
Fast Paced Competitive Shooters
Eg – Modern Warfare 2, Apex Legends
For multiplayer arena games, DLSS Quality offers the best balance between responsiveness and good visuals. Striving for max FPS aligns well with DLSS goals. Dynamic scenes stress DLAA.
Open World Action-Adventure
Eg – Witcher 3, Assassin‘s Creed Mirage
Aim for DLSS Balanced here. Complex physics, particles benefit less from AA. DLSS boosts FPS to offset heavy loads while retaining clarity.
Cinematic Story Focused
Eg – Hellblade 2, Cyberpunk Phantom Liberty
DLAA Quality fits well if playing at 4K120. Flagship GPUs can absorb cost. Nanite geo and Lumen lighting realism reveals high AA benefits in detailed assets.
Showcasing More Examples Across New Releases
Let‘s run through testing configurations, image comparisons and performance numbers across more just released and upcoming games to showcase how these insights apply.
The Callisto Protocol
Striking an impressive balance between visual splendor and locked 60 FPS gameplay, The Callisto Protocol is an excellent fit for maxing settings with DLSS 3 Frame Generation enabled.
Atomic Heart
Atomic Heart‘s frantic shooter action performed best under DLSS 2 Quality Mode, keeping framerates high in 4K while retaining beautiful lighting and reflections.
Starfield
Starfield sees improvements from both technologies, with DLAA reducing shimmering across expansive landscapes while DLSS Quality Mode provides welcome FPS aid to offset costly raytracing during space combat.
Wo Long: Fallen Dynasty
This stylized souls-like benefits greatly from DLAA enhancing anti-aliasing across visually striking ink and watercolor environments based on Chinese art without tanking FPS.
I hope this expanded, incredibly detailed guide has helped reveal all the secrets behind how DLSS and DLAA work at both a high level and deep technical capacity.
We covered how the specialized neural networks are designed, trained, continuously optimized over time to keep improving image quality and performance.
You now understand how these technologies integrate with modern games engines for maximizing gains.
And most importantly, you should feel empowered to choose the right solution whether DLAA, DLSS Quality or Performance modes etc. based on your GPU hardware, target resolution, playable FPS and genres of games in your library.
I invite you to join the active discussions with our community of gaming enthusiasts and technology experts over on the History Computer forums! Together we can help mutually unlock greater benefits from AI accelerated graphics technologies like DLSS 3 and DLAA.
