leetvr/hotham

Hotham is a tool for creating incredible standalone VR games.

hotham's Issues

The player transitions into deploy mode

Description

The player is aware that now they need to do something. Deploy their mechs and defend against aliens.

TODO

• Clean up the scene so it's more clear what's going on
• Write copy explaining to the player what needs to happen
• Have the audio play in the headset
• Implement some kind of state machine to move into the right state (eg. so we aren't endlessly looping) (depends on #40)

The player can clearly see the asteroid

Description

The user sees the asteroid in front of them, in the centre of their guardian space

TODO

• The asteroid is scaled to about 50% of the play area
• The asteroid is located just below the user's eye level
• The asteroid is rendered with a normal map and ambient occlusion
• The asteroid is lit by a visible sun to help orientate the player
• There is a background starfield to assist with orientation

Fix simulator

Description

Make the simulator great again!

TODO

• Make it work
• Make it work in release mode

Music of some kind is playing

Description

Music is great! Let's have some of that.

TODO

• Source some ambient space music
• Play the music in the headset

[BUILD] Fix `build.rs` in `hotham-simulator`

Description

The bindgen bindings keep getting regenerated over and over again. This is.. frustrating.

TODO

• Investigate
• Fix

[SYSTEMS] Add simple animation system

Description

It's kind of cool when things move in games, doesn't it?

TODO

• Initial research
• Add animation.rs
• Write test to load Animation, with channel etc.
• Implement that shit!
• Rewrite Model to use Nodes
• Implement Skin etc.
• Update animation in frame
• Update Vertex
• Update model.vert

Switch to `nalgebra` everywhere

Description

cgmath has been great to get us started but it's starting to produce a bit of friction.

TODO

• Remove cgmath dependency from toml
• Fix all the things that break

Investigate `hecs` again

Description

Using legion, there's been a lot to love, but there's also been some pretty annoying issues.

• system macros aren't picked up by rust-analyzer
• There's a lot of nice sugar around legion but a lot of it isn't well documented. hecs is a bit more barebones.
• legion doesn't look to be actively maintained and I DO NOT want to be maintaining a fork of an ECS - I'm just not smart enough for that. :P

Fix logging

Description

Everyone loves logging!

TODO

• See which logging library is fashionable these days
• Investigate tracing
• Implement it

[SYSTEMS] Move load_sound to utils

Description

load_sound doesn't belong in lib.rs

TODO

• [ ]

Improve Memory Management

Description

Currently, Buffer is just a fancy wrapper around a vk::Buffer handle. It'd be nice if we could improve its ergonomics a bit.

TODO

• [ ]

Scripting

Description

We need some way of creating "if this/then that" sequences.

TODO

• Research
• Implement

[SYSTEMS] Collision detection

Description

Things need to know when they're touching other things

TODO

• Research
• Implement

Implement current functionality in an ECS.

Overview

Great! We picked an ECS, legion. Now comes the fun part - implementing it.

I think the best place to start is Hotham::Program. Ultimately one of the most important things in hotham is making sure that the interface a developer will use to create an app is as simple as possible.

The asteroid could look something like:

struct Asteroid {
  model: Model,
  transform: Transform,
  // new functionality like physics, sound etc. will go here
}

struct Transform {
  position: Vector3<f32>,
  rotation: Quaternion<f32>,
  scale: Vector3<f32>,
}

let mut world = World::new();
let asteroid = Asteroid {
  model: asteroid_model,
  transform: Transform {
    position: vec3(0.0, 1.0, 0.0),
    rotation: Default::default(),
    scale: vec3(1.0, 1.0, 1.0),
  }
}

world.push(asteroid);

That's pretty straightforward. The refinery is the same, except we need to make sure its parent is the asteroid, somehow.

struct Refinery {
  model: Model,
  transform: Transform,
}

struct Transform {
  position: Vector3<f32>,
  rotation: Quaternion<f32>,
  scale: Vector3<f32>,
  parent: Option<Box<Transform>>, // ??
}

This is pretty different from our current Node based structure, which is just lifted from gltf. We'll need to approach the migration of Node to legion based components separately.

What's even trickier is things like Hands, for instance.

struct Hand {
   model: Model,
   transform: Transform,
   animation: Animation,
}

We'd then have something like a HandsSystem that takes the current XR state and updates the Animation and Transform components accordingly.

TODO

• Add legion dependency
• Modify Hotham::Program to return a World
• #45
• #47
• #50
• #51

Settle on 0.1 API

Context

#131 resulted in a massive refactoring of so much of Hotham's internal systems. That's meant the previous App API we had is now deceased. We now need a rewrite

TODO

• Spend a couple hours researching different approaches
• Pick something
• Don't fucking bikeshed it
• Don't shave yaks

Indicate to the player that they need to put their mech onto the asteroid

Description

The player needs to know what they need to do next. Otherwise, how will they do it?

TODO

• Write and record copy and play with script (depends on #40)
• Get a mech model
• Get a "mech carrier" model
• Have the carrier appear onscreen

0.1 release roadmap

Intro

After a bunch of work, hotham is dangerously close to being ready for other human beings to use.

Here's a list of things that need to be done first.

TODO

Code stuff 💻

• #65
• #54
• #46
• #83
• #40
• #52
• #59
• #55
• #42
• #44
• #66
• #67
• #84
• #85

Examples

• #75
• #69
• #68
• #70

Evil Marketing stuff 💀

• Website
• FAQ
• Book (4-5 pages explaining what Hotham is, how it works, with diagrams)
• "Beat Saber" clone tutorial - step by step with video
• "3 minute Launch" video (who I am, what Hotham is, what it's trying to do, etc.)

Repo stuff 🚧

• #62
• #71
• #72
• #73
• #74
• #76
• #79
• Upload to crates.io
• Verify docs.rs works
• Make sure sponsor stuff is working 🤑

Switch back to crates.io OpenXR

Description

Looks like OpenXR has added Android support over here. Let's use that instead.

TODO

• Switch Cargo.toml dependency
• Fix
• Test

"Good enough" glTF PBR support

Background

Currently things in hotham look terrible. Let's fix that!

Research

• Real-Time Rendering, Fourth Edition
• Sascha's Implementation
• Real Shading in Unreal
• Substance's PBR guide
• LearnOpenGL - PBR
• Moving Frostbite to PBR

TODO

• Write an integration test for rendering_system that takes a path to a gltf file and outputs the render to a file
• Import @SaschaWillems PBR shaders

RenderContext

• Update descriptor sets
  • Update SceneSet
  • Update MeshSet
  • Add MaterialSet
• Add Material struct
• Update PushConsts
• Update Pipeline
• Add SceneParams
• Update scene UBO -> can we stick with push constants for vertex shader?
• Investigate handling of non-skinned entities

Mesh

• Add Vec<Primitive>
• Push VertexBuffer and IndexBuffer into Primitive
• Add Material to Primitive
• Add all material bindings
• Add texture_descriptor_set to Primitive

renderer_system

• Bind to correct descriptor sets
• Multiview

gltf_importer

• Update Vertex
• Import correct Vertex attributes
• Import correct images
• Create Material

Test models

• Broken helmet model correctly
• Sponza correctly
• THE GLORIOUS MCRN DONAGER correctly

Add glb files to LFS

Players can grab objects with their controllers

Description

Our hands will need to be used to grab things

TODO

• Implement collision detection #36
• Research Unity implementation
• Investigate correct usage of RigidBody in hands scenario
• Add a Collider and RigidBody to hands
• Add a Collider and RigidBody to asteroid
• Move the Hand with RigidBody
• Update all Transforms based on the position of their RigidBody
• In HandsSystem, check to see if the hand is gripped, and if there are any collisons
• If so, store GrabbedEntity in Hand and set its RigidBody type to kinematic
• If not, remove the reference to GrabbedEntity
• Move the GrabbedEntity along with the Hand

Break apart Node into Components and Systems

Overview

Well this is going to be really fun.

When attempting to integrate legion I discovered that Node, the core structure in Hotham is completely incompatible. This is because, in essence Node is doing what legion wants to do - hold all the data and behaviour of the system in one place. Specifically, it's because Node looks like this:

struct Node {
  parent: Weak<Node>, // not 'static + Sized + Send + Sync !
  children: Vec<Rc<Node>>, // 'static + Sized + Send + Sync !
}

The solution is to break Node apart into its various Components (structs) and Systems (functions).

In essence, this is far closer to the actual representation of a gltf file: a discrete set of components with loose references to each-other, rather than an OOP-style graph structure like I've currently got.

Onwards!

TODO

• Create Transform component
• Create transform system
• Create Skin component
• Create Joint component
• Create skinning system
• Create Mesh component
• Create render system
• Create prepare_frame function
• Create XrContext resource
• Make Renderer a resource
• Make VulkanContext a resource
• Separate VulkanContext from Renderer

Indicate to the user where the mech will land

Description

The user needs to see where their mech will end up when they let go.

TODO

• Show some kind of "projection" of the user's Mech.
• Have the mech be red/green depending on whether or not this is a valid move

Get input from the touch controllers

Implement CI

Description

How can we integrate if it's not done continuously?

TODO

• Research GitHub actions
• Make it so

Modify gltf-importer to work with `legion`

Overview

Now that we've split apart Node, we now need to get gltf-importer to integrate with our new system.

TODO

• Investigate
• Load Mesh
• Load Transform
• Load Skin

Instruct the user to release the mech when ready

Description

The user has held on to a projection, is about to place it somewhere. Give them a gentle instruction on what to do

TODO

• Write/record audio for this section

Implement `Hands` using `legion`

Overview

Now that we've got static gltf models rendering, let's try something a bit more challenging.

TODO

• Create Hand component
• Create Hand system
• Update transform
• Update grip status
• Add hand models to scene
• Add animations
• Update animation in hands system

Implement first version of Hotham audio API

Shooting alley example

Have the mech drift down to the asteroid

Description

Once the player has released the projection, the mech needs to fly off the platform and head down to the Asteroid

TODO

• Research
• Implement

Beat Saber example

Description

One of the most important things when releasing complex software is providing examples. Not only does it provide users with a transparent way to see "how" to do things with the software, it gives users confidence in "what" can be done with the software.

Since Hotham is first and foremost a VR game engine, we need to prove that we can make VR games with it. What better game to make than Beat Saber?

This will require careful planning - it'd be very easy to get derailed and stuck in the weeds here. Beat Saber is a big and complex game, but there should be a way to cut it down to its simplest elements and recreate them.

TODO

Research

• Play the game and break down all its elements systematically
• Look for other examples of people cloning the game - what did they leave in, what did they out?

Assets

• Create a "bill of materials" - what will need to be included in the game?
• Acquire/build assets

Programming

• Sketch out a rough shape of what the code would look like
• Identify missing engine features
• #94
• Implement Saber System
• Implement Game System
• #105
• Implement #93
• Implement #52

The asteroid is scaled to about 50% of the play area

Have the mech orientate itself

Description

The mech needs to land on the Asteroid in a.. convincing fashion.

TODO

• Ensure the mech lands on the asteroid
• Make sure it's facing upright

Bowling alley example

Show the user's hands

Description

One of the most important things in VR is to orient the user and make them feel immersed, By showing the user their hands, they'll have a much better time.

TODO

• Get a model for hands
• #20
• #21
• Implement right hand

Basic Physics

Description

Many of the systems in this phase require "physics". What is the Minimum Physics we need for the game?

TODO

• Research libraries
• Implement

Render Asteroid and Refinery with `legion` architecture

Overview

Now that we've successfully ported most of our code to legion, it's time to integrate everything.

For our first pass, we'll just try rendering the Asteroid and the refinery.

TODO

• Modify App to run the legion schedule.
• Modify Renderer accordingly
• Bind to SceneData
• Modify hotham-asteroid to return the asteroid and the refinery parented to it

Implement ECS

Description

We're now starting to have a large number of actors within our system. We need an efficient way of organising them.

TODO

• Research ECS
• Research Rust ECS libraries
• Pick one
• #41

Deploy Phase Issues

Description

This is the deploy phase. The user can see where the asteroid is, where the refinery is, and begin to mentally model where to best place their mech to defend it.

Features

• #27
• #29
• #30
• #31
• #32
• #33
• #34

Systems

• #35
• #36
• #37
• #38
• #39

Fix model loading

Description

It's currently taking forever to load a model, and texture paths are completely hard coded.

TODO

• Figure out if it's because of running in release mode
• Look at using KTX textures
• Encapsulate all model data in a single struct

The player receives instructions on what to do next

Description

We know what the point of this game is, but the user is probably completely in the dark. Having some audio to tell the user what's going to happen is super useful.

TODO

• Get some cool synthesised speech
• Have it play in the headset
• Sound working on Android
• Sound only plays 5 seconds into game load

Release objects

The player can see a refinery on the asteroid

Description

The player can see a refinery on the asteroid. It's clear to the player that this is important, and needs to be protected

TODO

• Get a model of a refinery
• Place the model on the asteroid

Implement simple scene system

Description

Rendering a single asteroid is fun! You know what's better than a single asteroid?

Multiple.. things!

TODO

• Add Model
• Add SceneObject
• Implement load_models
• Add normal_map to descriptor_layout
• Have Program::init() return Vec<SceneObject>
• Return multiple scene_objects
• Fetch Matrix from gltf
• Add scene_objects to App
• Modify Renderer to have push_constants
• Rename shaders to model.*
• Load shaders inside hotham
• Have shaders read push_constant for matrix
• Have shaders read normal
• Have shaders read from normal map
• Pass scene_objects to Renderer::draw()
• Loop through each scene_object in prepare_frame
  • Update push constant
  • Bind descriptor_set, index_buffer and vertex_buffer from scene_object
• Create Android images
• Load models from Asset Manager
• Verify working on Android

Have the model animate in response

Description

When the user triggers some kind of input on the touch controllers, we want this to drive the animation

TODO

• Get the grip input
• Have that drive the animation

Implement Clippy

Description

Clippy will probably get mad.

TODO

• Incorporate Clippy
• Too Many Arguments problems
• Non Send Fields in Send Type problem
• Type Complexity Problems
• Remove commented references to Clippy in code

Problems

Clippy: Too Many Arguments

1. panel.rs add_panel_to_world (10/7)

  --> hotham/src/components/panel.rs:79:1                                                                                                                                                                       
   |                                                                                                                                                                                                            
79 | / pub fn add_panel_to_world(                                                                                                                                                                               
80 | |     text: &str,                                                                                                                                                                                          
81 | |     width: u32,                                                                                                                                                                                          
82 | |     height: u32,                                                                                                                                                                                         
...  |                                                                                                                                                                                                          
89 | |     world: &mut World,                                                                                                                                                                                   
90 | | ) -> Entity {                                                                                                                                                                                            
   | |___________^                                                                                                                                                                                              
   |                                                                                                                                                                                                            

2. gltf_loader.rs load_node (8/7)

  --> hotham/src/gltf_loader.rs:88:1                                                                                                                                                                            
   |                                                                                                                                                                                                            
88 | / fn load_node(                                                                                                                                                                                            
89 | |     node_data: &gltf::Node,                                                                                                                                                                              
90 | |     gltf_buffer: &[u8],                                                                                                                                                                                  
91 | |     vulkan_context: &VulkanContext,                                                                                                                                                                      
...  |                                                                                                                                                                                                          
96 | |     images: &[gltf::image::Data],                                                                                                                                                                        
97 | | ) -> Result<()> {                                                                                                                                                                                        
   | |_______________^                                                                                                                                                                                          
   |                                                                                                                                                                                                            

3. image.rs new (8/7)

warning: this function has too many arguments (8/7)                                                                                                                                                             
  --> hotham/src/image.rs:17:5                                                                                                                                                                                  
   |                                                                                                                                                                                                            
17 | /     pub fn new(                                                                                                                                                                                          
18 | |         handle: vk::Image,                                                                                                                                                                               
19 | |         view: vk::ImageView,                                                                                                                                                                             
20 | |         device_memory: vk::DeviceMemory,                                                                                                                                                                 
...  |                                                                                                                                                                                                          
25 | |         layer_count: u32,                                                                                                                                                                                
26 | |     ) -> Self {                                                                                                                                                                                          
   | |_____________^                                                                                                                                                                                            
   |                                                                                                                                                                                                            

4. vulkan_context.rs create_texture_image (9/7)

   --> hotham/src/resources/vulkan_context.rs:467:5                                                                                                                                                          
    |                                                                                                                                                                                                           
467 | /     pub fn create_texture_image(                                                                                                                                                                        
468 | |         &self,                                                                                                                                                                                          
469 | |         name: &str,                                                                                                                                                                                     
470 | |         image_buf: &[u8], // Clippy &Vec<u8>, ptr_arg for texture.rs                                                                                                                                    
...   |                                                                                                                                                                                                         
476 | |         offsets: Vec<vk::DeviceSize>,
477 | |     ) -> Result<(Image, vk::Sampler)> {
    | |_____________________________________^
    |

5. vulkan_context.rs create_textures_descriptor_sets (8/7)

   --> hotham/src/resources/vulkan_context.rs:802:5
    |
802 | /     pub fn create_textures_descriptor_sets(
803 | |         &self,
804 | |         set_layout: vk::DescriptorSetLayout,
805 | |         material_name: &str,
...   |
810 | |         emissive_map: &Texture,
811 | |     ) -> VkResult<Vec<vk::DescriptorSet>> {
    | |_________________________________________^

Clippy: Type Complexity

warning: very complex type used. Consider factoring parts into type definitions

1. texture.rs

   --> hotham/src/texture.rs:183:6
    |
183 | ) -> Result<(Vec<u8>, u32, u32, vk::Format, u32, u32, Vec<vk::DeviceSize>)> {
    |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |

2. vertex.rs

  --> hotham/src/vertex.rs:35:12
   |
35 |           t: (
   |  ____________^
36 | |             Vector3<f32>,
37 | |             Vector3<f32>,
38 | |             Vector2<f32>,
...  |
41 | |             Vector4<f32>,
42 | |         ),
   | |_________^
   |

3. mod.rs

  --> examples/crab-saber/src/systems/mod.rs:15:9
   |
15 |         PreparedQuery<With<Visible, With<Cube, (&'a Colour, &'a RigidBody, &'a Collider)>>>,
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |

Clippy: Non Send Fields in Send Type

1. audio_context.rs impl Send for AudioContext

warning: this implementation is unsound, as some fields in `AudioContext` are `!Send`                                                                                                                           
  --> hotham/src/resources/audio_context.rs:97:1                                                                                                                                                                
   |                                                                                                                                                                                                            
97 | unsafe impl Send for AudioContext {}                                                                                                                                                                       
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                       
   |                                                                                                                                                                                                            
   = note: `#[warn(clippy::non_send_fields_in_send_ty)]` on by default                                                                                                                                          
note: the type of field `stream` is `!Send`                                                                                                                                                                     
  --> hotham/src/resources/audio_context.rs:22:5                                                                                                                                                                
   |                                                                                                                                                                                                            
22 |     pub stream: Arc<Mutex<Stream>>,                                                                                                                                                                        
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                         
   = help: use a thread-safe type that implements `Send`                                                                                                                                                        

Have the player pick up, or somehow select their mech

Description

This is the important part. The player needs to be able to interact with their Mech.

TODO

• The player should be able to "grab" the mech #37
• When they've grabbed their mech, they see a "projection" moving from the actual mech a la Battlestar Galactica Deadlock

Investigate colour profile

Description

Looks like we may have to set the colour profile manually for the Quest with XR_FB_color_space.

TODO

• [ ]

Document public API

TODO

• Add #[deny(docs)]
• Get typing