lichtso/splatter

Splatter is a WebGPU based gaussian splatting engine written in Rust

Splatter

Inspired by gaussian-splatting but using a somewhat different approach to rendering.

Features

• Correctly computes the perspective projection of ellipsoids by intersecting the bounding elliptic cone with the view plane
• Uses the rasterizer instead of a tiled compute shader
• Rasterizes rotated rectangles instead of axis aligned squares
• GPU depth sorting using onesweep radix sort (except that the block sort is not WLMS because WebGPU does not support subgroup operations yet)
• CPU depth sorting as a fallback
• Frustum culling (optionally using stream compaction via indirect drawing)
• File parser and progressive loading via segmentation in chunks
• Lots of rendering configuration parameters to customize

Dependencies

Dependencies of the Library

• Graphics API: wgpu
• Geometric Algebra: geometric_algebra

Dependencies of the Example

• Window API: winit
• Logging: log

Example

You can download some pre-trained models from the original paper here.

cargo run --example showcase -- models/garden/point_cloud/iteration_7000/point_cloud.ply

Controls

• A / D: Move left / right
• W / S: Move forward / backward
• Q / E: Move up / down
• Z / X: Roll left / right
• Mouse: Pitch and yaw

splatter's Issues

Does sorting actually sort?

Apologies for the drive-by issue, feel free to direct me to a more appropriate place to ask the question.

I'm evaluating parallel sorting algorithms that can run on GPU, and the Onesweep implementation in this repo is one of the most promising. However, on trying to understand the code I came across what I think is a serious flaw. In particular, the ranking section just uses atomicAdd() for all invocations in the workgroup with no additional synchronization. Consider for simplification the case where all keys have the same digit. The allowable results are all permutations such that the values delivered to a single invocation are ascending. Thus, what is supposed to be a stable multi-split appears not to actually be stable.

It is easy to believe that on particular hardware, failure of stability might be hard to observe, in particular I wouldn't be surprised at all if atomics from a single subgroup were resolved in order. Additionally, for this application an approximate sort may be tolerable.

Looking at the history, I wonder if commit 4aa3754 might have traded a correct but slower implementation for a faster but approximate one. Obviously, if we had subgroups (which I'm tracking closely), we could do proper WLMS, but I am interested in how well we can do with basic WGSL.

In any case, thanks for the implementation, it's an impressive and valuable resource even if we don't end up going that direction.

Error on drawing window size on macOS 14 with M1 Max

Thanks for this great repo! I'm having trouble launching it on my M1 Mac. Am I doing anything wrong? I found that 4294967295 = 2^32 - 1, if this is related to anything.

Thanks for your time!

(base) ➜  splatter git:(main) ✗ cargo run --example showcase /Users/sunjiaming/Downloads/output/point_cloud/iteration_30000/point_cloud.ply
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/examples/showcase /Users/sunjiaming/Downloads/output/point_cloud/iteration_30000/point_cloud.ply`
INFO - Unable to open libEGL: Library(DlOpen { desc: "dlopen(libEGL.dylib, 0x0005): tried: \'libEGL.dylib\' (no such file), \'/System/Volumes/Preboot/Cryptexes/OSlibEGL.dylib\' (no such file), \'/usr/lib/libEGL.dylib\' (no such file, not in dyld cache), \'libEGL.dylib\' (no such file), \'/Users/sunjiaming/Repositories/splatter/target/debug/build/objc_exception-cbe1ed39f1788974/out/libEGL.dylib\' (no such file), \'/Users/sunjiaming/Repositories/splatter/target/debug/deps/libEGL.dylib\' (no such file), \'/Users/sunjiaming/Repositories/splatter/target/debug/libEGL.dylib\' (no such file), \'/opt/homebrew/Cellar/rust/1.72.1/lib/rustlib/aarch64-apple-darwin/lib/libEGL.dylib\' (no such file), \'/Users/sunjiaming/lib/libEGL.dylib\' (no such file), \'/usr/local/lib/libEGL.dylib\' (no such file), \'/usr/lib/libEGL.dylib\' (no such file, not in dyld cache)" })
INFO - Adapter Metal AdapterInfo { name: "Apple M1 Max", vendor: 0, device: 0, device_type: IntegratedGpu, driver: "", driver_info: "", backend: Metal }
INFO - Using Apple M1 Max (Metal)
INFO - Created compute pipeline Valid((0, 1, Metal)) with ComputePipelineDescriptor { label: None, layout: Some((1, 1, Metal)), stage: ProgrammableStageDescriptor { module: (0, 1, Metal), entry_point: "radixSortA" } }
INFO - Created compute pipeline Valid((1, 1, Metal)) with ComputePipelineDescriptor { label: None, layout: Some((1, 1, Metal)), stage: ProgrammableStageDescriptor { module: (0, 1, Metal), entry_point: "radixSortB" } }
INFO - Created compute pipeline Valid((2, 1, Metal)) with ComputePipelineDescriptor { label: None, layout: Some((1, 1, Metal)), stage: ProgrammableStageDescriptor { module: (0, 1, Metal), entry_point: "radixSortC" } }
INFO - Created render pipeline Valid((0, 1, Metal)) with RenderPipelineDescriptor { label: None, layout: Some((0, 1, Metal)), vertex: VertexState { stage: ProgrammableStageDescriptor { module: (0, 1, Metal), entry_point: "vertex" }, buffers: [] }, primitive: PrimitiveState { topology: TriangleStrip, strip_index_format: None, front_face: Ccw, cull_mode: None, unclipped_depth: false, polygon_mode: Fill, conservative: false }, depth_stencil: None, multisample: MultisampleState { count: 1, mask: 18446744073709551615, alpha_to_coverage_enabled: false }, fragment: Some(FragmentState { stage: ProgrammableStageDescriptor { module: (0, 1, Metal), entry_point: "fragment" }, targets: [Some(ColorTargetState { format: Bgra8Unorm, blend: Some(BlendState { color: BlendComponent { src_factor: DstAlpha, dst_factor: One, operation: Add }, alpha: BlendComponent { src_factor: Zero, dst_factor: OneMinusSrcAlpha, operation: Add } }), write_mask: ColorWrites(RED | GREEN | BLUE | ALPHA) })] }), multiview: None }
INFO - Created buffer Valid((0, 1, Metal)) with BufferDescriptor { label: None, size: 224, usage: BufferUsages(COPY_DST | UNIFORM), mapped_at_creation: false }
INFO - Created buffer Valid((1, 1, Metal)) with BufferDescriptor { label: None, size: 4, usage: BufferUsages(COPY_DST | UNIFORM), mapped_at_creation: true }
INFO - Created buffer Valid((2, 1, Metal)) with BufferDescriptor { label: None, size: 4, usage: BufferUsages(COPY_DST | UNIFORM), mapped_at_creation: true }
INFO - Created buffer Valid((3, 1, Metal)) with BufferDescriptor { label: None, size: 4, usage: BufferUsages(COPY_DST | UNIFORM), mapped_at_creation: true }
INFO - Created buffer Valid((4, 1, Metal)) with BufferDescriptor { label: None, size: 4, usage: BufferUsages(COPY_DST | UNIFORM), mapped_at_creation: true }
INFO - Created buffer Valid((5, 1, Metal)) with BufferDescriptor { label: None, size: 2101268, usage: BufferUsages(COPY_SRC | COPY_DST | STORAGE | INDIRECT), mapped_at_creation: false }
INFO - Created buffer Valid((6, 1, Metal)) with BufferDescriptor { label: None, size: 33554432, usage: BufferUsages(COPY_SRC | STORAGE), mapped_at_creation: false }
INFO - Created buffer Valid((7, 1, Metal)) with BufferDescriptor { label: None, size: 33554432, usage: BufferUsages(COPY_SRC | STORAGE), mapped_at_creation: false }
INFO - Created buffer Valid((8, 1, Metal)) with BufferDescriptor { label: None, size: 67559520, usage: BufferUsages(COPY_DST | STORAGE), mapped_at_creation: false }
INFO - configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8Unorm, width: 1600, height: 1200, present_mode: Fifo, alpha_mode: Opaque, view_formats: [Bgra8Unorm] }
INFO - build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8Unorm, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
INFO - Created texture Valid((0, 1, Metal)) with TextureDescriptor { label: None, size: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, mip_level_count: 1, sample_count: 1, dimension: D2, format: Depth24PlusStencil8, usage: TextureUsages(RENDER_ATTACHMENT), view_formats: [Depth24PlusStencil8] }
INFO - configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8Unorm, width: 4294967295, height: 4294967295, present_mode: Fifo, alpha_mode: Opaque, view_formats: [Bgra8Unorm] }
WARN - Requested size 4294967295x4294967295 is outside of the supported range: Extent3d { width: 4, height: 4, depth_or_array_layers: 1 }..=Extent3d { width: 16384, height: 16384, depth_or_array_layers: 1 }
INFO - build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8Unorm, extent: Extent3d { width: 4294967295, height: 4294967295, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
2023-10-02 02:36:09.793 showcase[27291:18002237] CAMetalLayer ignoring invalid setDrawableSize width=4294967295.000000 height=4294967295.000000
ERROR - Handling wgpu errors as fatal by default
thread 'main' panicked at 'wgpu error: Validation Error

Caused by:
    In Device::create_texture
    Dimension X value 4294967295 exceeds the limit of 8192

', /Users/sunjiaming/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.17.0/src/backend/direct.rs:3056:5
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: wgpu::backend::direct::default_error_handler
             at /Users/sunjiaming/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.17.0/src/backend/direct.rs:3056:5
   3: core::ops::function::Fn::call
             at /private/tmp/rust-20230923-6524-l6xj0w/rustc-1.72.1-src/library/core/src/ops/function.rs:79:5
   4: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at /private/tmp/rust-20230923-6524-l6xj0w/rustc-1.72.1-src/library/alloc/src/boxed.rs:2007:9
   5: wgpu::backend::direct::ErrorSinkRaw::handle_error
             at /Users/sunjiaming/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.17.0/src/backend/direct.rs:3042:17
   6: wgpu::backend::direct::Context::handle_error
             at /Users/sunjiaming/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.17.0/src/backend/direct.rs:324:9
   7: <wgpu::backend::direct::Context as wgpu::context::Context>::device_create_texture
             at /Users/sunjiaming/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.17.0/src/backend/direct.rs:1316:13
   8: <T as wgpu::context::DynContext>::device_create_texture
             at /Users/sunjiaming/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.17.0/src/context.rs:2329:31
   9: wgpu::Device::create_texture
             at /Users/sunjiaming/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.17.0/src/lib.rs:2455:13
  10: <showcase::Application as showcase::application_framework::Application>::resize
             at ./examples/showcase.rs:84:37
  11: showcase::application_framework::ApplicationManager::resize
             at ./examples/application_framework.rs:199:9
  12: showcase::application_framework::ApplicationManager::start_loop::{{closure}}
             at ./examples/application_framework.rs:231:21