We are more than an asset library

Watch our Showreel
Search
Cart
Sign in
  • offerCustomer zone
  • offerYour orders
  • offerEdit account

  • offerAdd project
  • offerLiked projects
  • offerView your artist profile

  • Dark mode
    ✔
NVIDIA's Instant NuRec turns driving footage into 3D Gaussian worlds in 1.5 seconds

NVIDIA's Instant NuRec turns driving footage into 3D Gaussian worlds in 1.5 seconds

Evermotion 2026-11-16 11:25 article  > All

NVIDIA's feed-forward neural model reconstructs multi-camera driving logs into full 3D Gaussian Splatting scenes in a single pass, achieving 28 dB PSNR 3000× faster than optimization-based approaches. Code and checkpoints are now available, but the model was trained only on autonomous driving data.

NVIDIA Research has released Instant NuRec, a feed-forward neural reconstruction model that turns a 10 to 20-second multi-camera driving log into a fully simulatable 3D Gaussian Splatting world in a single forward pass. The model was presented as an oral at NeurIPS 2026 and is now integrated into NVIDIA Omniverse NuRec v26.04, with code available on GitHub and a checkpoint on Hugging Face.

The speed gain is dramatic. Instant NuRec reconstructs a scene in roughly 1.5 seconds, achieving 28.26 dB full-image PSNR on the Waymo Open Dataset - 2.01 dB above the strongest baseline. On NVIDIA's internal data, it reaches 29.93 dB in 1.5 seconds versus the original NuRec's 34.38 dB in approximately 75 minutes, a 3000× speedup. The efficiency comes from a Vision Transformer encoder (following Depth-Anything-v3 design) that predicts Gaussian parameters end-to-end rather than optimizing them per-scene.

Memory and inference costs are further reduced via a Selective strategy that cuts the Gaussian budget by roughly 3×, from 351K Gaussians per context view down to 120K, bringing total scene size from 6.3M to 2.2M Gaussians across an 18-frame clip. The encoder handles non-pinhole camera models via 3D Gaussian Unscented Transform (3DGUT) to accommodate fisheye and rolling shutter effects common in autonomous vehicle datasets. Training used approximately 40K filtered clips from NVIDIA's internal autonomous vehicle platform across 8 nodes for roughly 6 days.

A key trade-off: Instant NuRec's dynamic layer translates Gaussians but does not rotate or deform them. This limitation means turning vehicles do not deform convincingly - as one analysis noted, turning vehicles "break the 'fully simulatable' claim." For closed-loop autonomous driving policy evaluation, however, NVIDIA Research reports that policy ranking is preserved between Instant NuRec and optimized NuRec, suggesting sufficient fidelity for simulation-based decision-making.

What this means for archviz capture pipelines

Instant NuRec's speed and memory efficiency point toward a potential reshaping of real-time scene capture workflows. For archviz artists working with Gaussian splatting in Blender and real-time engines, feed-forward reconstruction could replace hours of per-scene optimization with seconds of inference. The 3× Gaussian budget reduction is especially relevant for memory-constrained real-time pipelines.

The critical limitation: the model was trained exclusively on autonomous driving footage. Its performance on static interiors, photogrammetry-derived scans, or architectural scenes remains unknown. Generalization to non-driving scenarios - whether outdoor architecture, indoor spaces, or mixed lighting conditions - is untested. Whether NVIDIA or the community will release architecture-specific checkpoints trained on building scans, interior photography, or professional photogrammetry datasets is unclear.

For studios currently optimizing Gaussian Splats or exploring alternative relighting and integration workflows, the technique signals a direction worth monitoring. If the approach can be adapted to architectural data, it could become a faster alternative to traditional Structure-from-Motion and per-scene optimization - provided the static scene assumption holds and the memory gains translate to production pipelines.

Sources

Lead image: NVIDIA Research

Author: Evermotion Source: arXiv
Tags: blender nvidia real-time gaussiansplatting 3d-reconstruction
You may also like...
TipOfTheWeek: create Christmas scene in Blender

TipOfTheWeek: create Christmas scene in Blender

Find out how to create glass ball using vary modeling techniques. Discover power of Cycles and Blender's internal post production system!
×

LEAVE A COMMENT

You need to be logged in to leave a comment. Don't have account? Register now.