hyperlogic/splatapult

A 3d gaussian splatting renderer in C++ and OpenGL

Splatapult

A program to display 3d gaussian splats files

splatapult [OPTIONS] ply_filename

Options

-v, --openxr launch app in vr mode, using openxr runtime

-f, --fullscreen launch window in fullscreen

-d, --debug enable debug logging

-h, --help show help

Desktop Controls

• wasd - move
• q, e - roll
• t, g - up, down
• arrow keys - look
• right mouse button - hold down for mouse look.
• gamepad - if present, right stick to rotate, left stick to move, bumpers to roll
• c - toggle between initial SfM point cloud (if present) and gaussian splats.
• n - jump to next camera
• p - jump to previous camera
• y - toggle rendering of camera frustums
• h - toggle rendering of camera path
• return - save the current position and orientation of the world into a vr.json file.

VR Controls

• left stick - move
• right stick - snap turn
• f - show hide floor carpet.
• single grab - translate the world.
• double grab - rotate and translate the world.
• triple grab - (double grab while trigger is depressed) scale, rotate and translate the world.
• c - toggle between initial SfM point cloud (if present) and gaussian splats.
• y - toggle rendering of camera frustums
• h - toggle rendering of camera path
• return - save the current position and orientation/scale of the world into a vr.json file.

Config Files

If a "_vr.json" file is found, it will be used to determine the proper starting position, scale and orienation for vr mode. You can create your own _vr.json file by manipulating the scene via grab in vr mode, then press return to save.

Splatapult supports the same dir structure that Gaussian Splatting code will output. Which is as follows:

dir/
    point_cloud/
        iteration_30000/
            point_cloud.py
            point_cloud_vr.json
    input.ply
    cameras.json

input.ply contains the point cloud and cameras.json will contain the camera orientations from the SfM stage.

If the "cameras.json" file is found in the same dir as the ply_filename or it's parent dirs, it will be loaded. The 'n' and 'p' keys can then be used to cycle thru the camera viewpoints.

It will also support files downloaded from lumalabs.ai, but in this case there will be no point clouds or cameras.

dir/
    mycapture.ply
    mycapture_vr.json

splatapult's Issues

pre sort by depth sould be "float depth = p.z / p.w"?

splatapult/shader/presort_compute.glsl

Line 41 in 818ba0f

float depth = p.w;

Some questions. Thank you

splatapult/src/splatrenderer.cpp

Line 146 in 7b0416a

glm::mat4 modelViewMat = glm::inverse(cameraMat);

:Why we need to do glm::inverse here?

Quest 3: VR head not working

Hi, Thanks for helping me open and install splatapult.
Now that it is running on my system I'm trying to get it to work in VR.
I have opended it with oculus link running and with Steam VR running but I keep getting:

C:\Users\lodew\splatapult>splatapult.exe -v C:\Users\lodew\ansel\mmemBLENDERexport.ply
[ERROR] xrCreateSession [XR_ERROR_GRAPHICS_DEVICE_INVALID]
[ERROR] OpenXR Init failed
[ERROR] App::Init failed

I have tryed many times, yet can't get it to project and track my headset.
Thanks for the help.

Lolo

Error on loading files

Thanks for putting this alternative viewer out! And, trying the splatapult-0.1-x64.zip release ...

But, I had to try multiple PLY files before a model was successfully read; and that was one with the truck.

What occurs nearly every time are multiple instances of a message like

[ERROR] Error parsing ply file "D:\Apps\GaussianSplatting\splats\Colab\bicycle\b4000.ply", missing f_rest property

VR creator with limited amount of CLI experience

Hi,

My name is Lolo, and I would love to use your VR viewer to demo my 3DGS scenes.
But my knowlage of CLI is limited and installing a project like this from GitHub without a detailed install tutorial has been tried, and failed....
I understand that a lot of people are able to "install" this but i'm simply not able to look at this repo and go, yeah this, then that then boom!
I have watched a ton of tutorials to learn how to install Nvidia Nerf, Nerfstudio, Inria GS but with a lot of hand holding. And now that i'm trying to simply download, place on HD and execute a command in CLI i'm completly lost. Also because of the custom prompt you are using in the youtube video.
Would you mind doing an install video for dummies like me? or is there another way i can learn how to install your project? Because i really like what i see in the Youtube video and i'd really love to try!

Thank you for your patience.

Lodewijk

How to change the height of the camera?

Thanks for the great work.
My computer does not have enough spec to run the SIBR viewer with CUDA, but could run this viewer.

I have two questions:

1. How to change the height of the camera? WASD for moving, UpLeftDownRight for rotating, QE for rolling. But it seems that we could not control the height position now.
2. Is it possible to add a new mode, like camera rotating around a point (e.g. (0,0,0), or can be setting), to see the main object just like bullet time?

Thank you.

Camera is moving too fast when fps is low

Desktop.2024.01.27.-.13.41.15.04-1.mp4

Question about render result on different PC

Hi, I tried to run the same program with the same ply on 2 different PC (one using NV graphic card, another using AMD)
And I found the display result is quite different.
(running in window and than maximum the window, than take the screenshot)

NV (GTX 1050):

AMD (Z1E):

The NV PC has even high desktop resolution than AMD PC.

It seems that NV render result is quite better than AMD.
Do you have any idea about this?

Thank you.

Multi VAO loading

I tried loading multiple Gaussian ploys and instantiating multiple VAOs. After loading, it was found that there were some issues with the rendering of the Gaussian sphere at the edges. Some angles are normal, while others are very poor

Compilation errors on Linux.

I understand that this project may not be targeted to Linux. I'm just putting this information here for anyone who may be interested.

base ❯ make -j
[ 13%] Building CXX object CMakeFiles/splatapult.dir/src/core/image.cpp.o
[ 13%] Building CXX object CMakeFiles/splatapult.dir/src/core/debugrenderer.cpp.o
[ 18%] Building CXX object CMakeFiles/splatapult.dir/src/core/inputbuddy.cpp.o
[ 18%] Building CXX object CMakeFiles/splatapult.dir/src/core/log.cpp.o
[ 27%] Building CXX object CMakeFiles/splatapult.dir/src/core/program.cpp.o
[ 27%] Building CXX object CMakeFiles/splatapult.dir/src/core/texture.cpp.o
[ 31%] Building CXX object CMakeFiles/splatapult.dir/src/core/util.cpp.o
[ 40%] Building CXX object CMakeFiles/splatapult.dir/src/core/vertexbuffer.cpp.o
[ 40%] Building CXX object CMakeFiles/splatapult.dir/src/app.cpp.o
[ 50%] Building CXX object CMakeFiles/splatapult.dir/src/core/textrenderer.cpp.o
[ 50%] Building CXX object CMakeFiles/splatapult.dir/src/core/xrbuddy.cpp.o                                                                           [ 54%] Building CXX object CMakeFiles/splatapult.dir/src/ply.cpp.o
[ 59%] Building CXX object CMakeFiles/splatapult.dir/src/camerasconfig.cpp.o
[ 63%] Building CXX object CMakeFiles/splatapult.dir/src/gaussiancloud.cpp.o
[ 68%] Building CXX object CMakeFiles/splatapult.dir/src/magiccarpet.cpp.o
[ 77%] Building CXX object CMakeFiles/splatapult.dir/src/flycam.cpp.o
[ 77%] Building CXX object CMakeFiles/splatapult.dir/src/pointcloud.cpp.o
[ 81%] Building CXX object CMakeFiles/splatapult.dir/src/pointrenderer.cpp.o
[ 86%] Building CXX object CMakeFiles/splatapult.dir/src/sdl_main.cpp.o
[ 90%] Building CXX object CMakeFiles/splatapult.dir/src/splatrenderer.cpp.o
[ 95%] Building CXX object CMakeFiles/splatapult.dir/src/vrconfig.cpp.o
In file included from /home/sunjiaming/Repositories/splatapult/src/core/inputbuddy.cpp:6:
/home/sunjiaming/Repositories/splatapult/src/core/inputbuddy.h: In constructor ‘InputBuddy::Joypad::Joypad()’:
/home/sunjiaming/Repositories/splatapult/src/core/inputbuddy.h:46:13: error: ‘memset’ was not declared in this scope
   46 |             memset(this, 0, sizeof(Joypad));
      |             ^~~~~~
/home/sunjiaming/Repositories/splatapult/src/core/inputbuddy.h:11:1: note: ‘memset’ is defined in header ‘<cstring>’; did you forget to ‘#include <cst
ring>’?                                                                                                                                                  10 | #include <map>
  +++ |+#include <cstring>
   11 | #include <stdint.h>
In file included from /home/sunjiaming/Repositories/splatapult/src/core/xrbuddy.h:33,
                 from /home/sunjiaming/Repositories/splatapult/src/core/xrbuddy.cpp:6:
/usr/include/openxr/openxr_platform.h:114:5: error: ‘Display’ does not name a type                                                                      114 |     Display*                    xDisplay;
      |     ^~~~~~~                                                                                                                                   /usr/include/openxr/openxr_platform.h:116:5: error: ‘GLXFBConfig’ does not name a type
  116 |     GLXFBConfig                 glxFBConfig;
      |     ^~~~~~~~~~~
/usr/include/openxr/openxr_platform.h:117:5: error: ‘GLXDrawable’ does not name a type
  117 |     GLXDrawable                 glxDrawable;
      |     ^~~~~~~~~~~
/usr/include/openxr/openxr_platform.h:118:5: error: ‘GLXContext’ does not name a type
  118 |     GLXContext                  glxContext;
      |     ^~~~~~~~~~

Environment information:

base ❯ c++ -v
Using built-in specs.
COLLECT_GCC=c++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 11.4.0-1ubuntu1~22.04' --with-bugurl=file:///usr/share/doc/gcc-11/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-11 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)

VR control does not working when using other VR headmount

Hi, I am now trying on another VR headmount (HTC vive),
but VR control does not working.
Render result can be seen.
Last time, the program went well on Quest 3.
Do you have any idea about this?
Thank you.