3D Gaussian Splatting for Real-Time Radiance Field Rendering in Julia
bicycle.mp4
- Julia 1.10.
- AMD GPU (ROCm) or Nvidia (CUDA) capable machine.
fused=true |
fused=false |
|
|---|---|---|
| RGB | + | + |
| Depth rendering | + | + |
| Differentiable depth | - | + |
| Differentiable camera pose | in-progress |
in-progress |
| Antialiasing | - | + |
- Because Julia is JIT-compiled, first iteration will take longer because of kernels compilation. In GUI you may even see that an application is unresponsive, so you should just wait a bit.
- Julia relies on GC that is not GPU-aware, excessive allocations will impact the performance. To free GPU arrays we have to perform GC-sweep (in contrast to refcounting in Python) which is significantly slower. Because of that, you'll see significantly higher memory usage (compared to Python), but that is because of delayed freeing, not because the algorithm requires more memory.
- Install GaussianSplatting.jl package:
] add https://github.com/JuliaNeuralGraphics/GaussianSplatting.jl.git- AMD GPU:
-
Add necessary packages:
] add AMDGPU -
Run:
julia> using AMDGPU, GaussianSplatting
julia> GaussianSplatting.gui("path-to-colmap-dataset-directory"; scale=1)- Nvidia GPU:
-
Add necessary packages:
] add CUDA, cuDNN -
Run:
julia> using CUDA, cuDNN, GaussianSplatting
julia> GaussianSplatting.gui("path-to-colmap-dataset-directory"; scale=1)-
AMD GPU:
julia> using Flux julia> Flux.gpu_backend!("AMDGPU")
-
Nvidia GPU:
julia> using Flux julia> Flux.gpu_backend!("CUDA")
Download one of the reference datasets from the MIP-NeRF-360: https://jonbarron.info/mipnerf360/