The real world is immense, and creating 3D representations of it requires algorithms that can handle vast spatial data and high resolutions. Traditional deep learning infrastructure falls short in scaling to these demands.
At SIGGRAPH this week, NVIDIA announced they would be releasing a new open-source, deep learning framework for 3D data – fVDB – which provides an advanced infrastructure for massive sparse volume datasets at high resolutions, enabling generative physical AI with spatial intelligence to scale to the size of reality.
We are excited to announce that OpenVDB users will soon have access to fVDB, as the OpenVDB Technical Steering Committee is currently working to merge the code, which will be available in the OpenVDB GitHub repository. Users can also apply for the fVDB Early Access Program to access the fVDB PyTorch extension. 
OpenVDB is a C++ library for efficient representation and simulation of volumetric effects like water, fire, smoke, and clouds. Development of OpenVDB started in 2009 at DreamWorks Animation for use in volumetric applications typically encountered in feature film production. It was released to the open source community in 2012, and with the establishment of the Academy Software Foundation (ASWF) in 2018, it became the Foundation’s first hosted project.
fVDB provides the framework for large-scale, high-resolution spatial intelligence. It builds accelerated AI operators on top of the graphics and simulation framework of OpenVDB, enabling reality-scale digital twins, neural radiance fields, 3D generative AI, and more.
fVDB features:
- Generative physical AI with spatial intelligence at scale: fVDB provides 3D deep learning infrastructure for massive datasets and high resolutions, allowing generative physical AI to scale to the size of reality.
- High performance and high resolution: fVDB AI operators are built on top of NanoVDB, which provides GPU acceleration for OpenVDB. This framework supports operations like sparse convolution and ray tracing optimized for real time. fVDB enables faster training and real-time inference, minimizing memory footprint and maximizing data processing throughput.
- Seamless integration: fVDB is available as a PyTorch extension. If you’re already using OpenVDB in your existing pipeline, fVDB can read and write existing OpenVDB datasets out of the box. It interoperates with other libraries and tools, such as Warp for simulation and Kaolin for 3D deep learning.
- Comprehensive operator set and ray tracing: fVDB provides differentiable operators, including convolution, ray tracing, attention, and gaussian splatting, all designed for generative physical AI with spatial intelligence. These operators allow you to build complex 3D neural networks, like large-scale point cloud reconstruction and 3D generative modeling.
- fVDB NIMs: Coming soon, fVDB functionality will be available as NVIDIA NIM inference microservices that enable developers to incorporate the fVDB core framework into OpenUSD workflows. NVIDIA fVDB NIMs generate OpenUSD-based geometry in NVIDIA Omniverse.
- fVDB Mesh Generation NIM: Generates an OpenUSD-based mesh, rendered by Omniverse Cloud APIs, from point cloud data.
- fVDB Physics Super-Res NIM: Performs AI super-resolution on a frame or sequence of frames to generate an OpenUSD-based high-resolution physics simulation.
- fVDB NeRF-XL NIM: Generates large-scale NeRFs in OpenUSD using NVIDIA Omniverse Cloud APIs.
Ken Museth is the founder of OpenVDB, current Chair of its Technical Steering Committee (TSC), and a Senior Director in Simulation and Technology at NVIDIA. He shared, “Effectively scaling traditional deep learning infrastructure to handle the vast spatial data demands of the real world is challenging. fVDB offers a framework for achieving truly large-scale and high-resolution spatial intelligence.”
For additional information on fVDB, read more on the NVIDIA technical blog here. You can also download the SIGGRAPH technical paper, fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence, here.
