ccaven/tinyslam

Compute shader visual SLAM

tiny_slam is a visual SLAM (Simultaneous Localization and Mapping) library. It heavily relies on general-purpose GPU computing via the tiny_wgpu library (primarily using the Vulkan backend).

tiny_slam is a work in progess.

Motivation

tiny_slam aims to:

1. Make visual SLAM accessible to developers, independent researchers, and small companies
2. Decrease the cost of visual SLAM
3. Bring edge computing to cross-platform devices (via wgpu)
4. Increase innovation in drone / autonomous agent applications that are unlocked given precise localization

Constraints

tiny_slam imposes these constraints on itself:

1. Minimize number of dependencies
2. Rely on computer shaders whenever possible
3. Run in realtime on a Raspberry Pi 5
4. Ergonomic design (Rust-like)

Progess

• Obtain required hardware
  • Raspberry Pi 5
  • High framerate camera, like this one
  • Drone materials
• Receive realtime data from USB camera
  • Use MediaFoundation API on Windows, video4linux on Linux, and AVFoundation on MacOS
  • Software decode Motion JPEG (MJPEG) stream from webcam
  • Utilize hardware decoding with higher framerate cameras
• Build helper library (tiny_wgpu) to increase compute shader workflow
  • Increase default limits for push constants and number of bindings
  • Enable read/write storage textures
  • Support render pipelines
  • Support reading data back to CPU via staging buffers
  • Support multiple shader files
• Feature detection
  • Color to grayscale conversion
    • Implement manual luminance calculation
    • (Optional) Use Y channel of YUV stream directly
  • Oriented FAST corner detection
    • Implement workgroup optimizations
    • Implement bitwise corner detector
    • Implement 4-corner shortcut
    • Replace storage buffers with textures to improve memory reads
  • Rotated BRIEF feature descriptors
    • Two-pass gaussian blur
    • Use linear sampler filtering to decrease number of samples
    • Implement workgroup optimizations
  • Read data back to CPU
• Local mapping
  • Keyframe selection
  • Insertion into current Map
  • Cull unnecessary map points
  • Local bundle adjustment