owensgroup/optix_splats

Testing different OIT methods for Gaussian Splatting

Optix Splats

This repo contains code for a differentiable gaussian renderer that leverages ray-tracing and OIT techniques to render gaussian splatted scenes.

Setup and Compilation

First install Anaconda. Then create an environment from the repo yaml

conda env create -f environment.yml

Compilation

You first need to change the variable optix_root in setup.py to the path where optix is located on your machine.

You can build the extension with

python setup.py install

You can test the install is functional by running the test script

python test/render_test.py

Common Issues

You may have issues with pygame if on Ubuntu 22.04. To fix you need to delete the conda versions of libstdc++ so it falls back to your OS. You can find the path to the conda drivers by exposing the debug environment variable.

export LIBGL_DEBUG=verbose

And then going to the anaconda path and deleting libstdc++ (deleting the symlink works too)

optix_splats's Issues

Change in_ellipsoid in kernels.cu to use transform matrices instead of scale/center/rotation

Update gas to a mesh for the unit sphere

Update gas from two triangles to icosphere for each gaussian (this will be used for visualization purposes)

Find and fix GPU memory leak when rendering

GPU memory leak when rendering, way less severe then before:

python test/render_test.py

Refactor build_ias in renderer.cu to take a list of means, scales, and rotations

Get CUDA/OpenGL interop

Change renderer.cu to properly swap the Optix image with an OpenGL framebuffer, and modify render_test.py to use OpenGL for rendering. You will need to figure out how to load GL functions on the C++ side.