fractalfir/tmf

Tight Model format is an experimental lossy 3D model format focused on reducing file size as much as posible without decreasing visual quality of the viewed model or read speeds.

tmf's Issues

Add size comparisons with GLTF

Is your feature request related to a problem? Please describe.
GTLF is gaining massive traction as a reliable royalty-free general purpose scene description format. Consider adding comparisons to a model only gltf.

Describe the solution you'd like
Just as OBJ and Draco have sections in the readme, so should GLTF.

Describe alternatives you've considered
N/A

Additional context
It would;d be nice to compare tmf to GLTF, as more and more models are being distributed in it. Also a GLTF->TMF converter would be nice

[FEATURE] Support for quads

I wanted to ask if this is in the to-do list. There is literally no way out there to compress quad meshes effectively right now. Converting to triangles means you lose all the precious clean quad topology.

[BUG] Triangle segment spilling has some weird issues.

Describe the bug
Triangle segment spiriting should not change the mesh in any way, besides reducing file size. Basic tests show that there are no issues with spilling segments, but when a real mesh(Suzanne) is saved/read and then exported, one triangle is always wrong.

To Reproduce
Read/save and the export a mesh.

Expected behavior
Exported mesh is the same as imported one.

[BUG] tmf Doesn't compile with the `fast_trig` feature enabled

I get a compilation error if I enable the fast_trig feature:

tmf = { version = "0.1.1", features = ["fast_trig"] }

When running cargo build I get the following compilation error:

error[E0412]: cannot find type `fprec` in this scope
  --> /github.com-1ecc6299db9ec823/tmf-0.1.1/src/normals.rs:98:27
   |
98 |     let x = fsin(asine as fprec) as FloatType;
   |                           ^^^^^ not found in this scope
   |
note: type alias `crate::utilis::fprec` exists but is inaccessible
  --> /github.com-1ecc6299db9ec823/tmf-0.1.1/src/utilis.rs:21:1
   |
21 | type fprec = f64;
   | ^^^^^^^^^^^^^^^^^ not accessible

I would expect tmf to compile regardless of the enabled features. tmf compiles correctly when fast_trig is not enabled.

[FEATURE] Tangent vertex attribute

I'd like to be able to store tangents in my tmf files.

Storing tangents in a serialized mesh format is a way to avoid expensive computation at runtime. Hence reducing load time. In fact glTF specifies tangents as a standard field for their model file format. Quoting https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#meshes-overview

XYZW vertex tangents where the XYZ portion is normalized, and the W component is a sign value (-1 or +1) indicating handedness of the tangent basis

Those look very much like a normal (with the one additional +1 -1 component).

Octahedron normals

Hey there!

Have you considered encoding normals using octahedron mapping? It's a neat way to map a direction vector into two components and it has a more uniform distribution than storing (angle, z).

I haven't profiled it against the current implementation, but here's some sample code:

code

use glam::{Vec2, Vec3};

/// Encode a 3d direction vector to a 2d vector using octahedron mapping.
/// The output vector is in the range [-1..1]. The input vector doesn't have to be normalized.
pub fn encode_oct(dir: Vec3) -> Vec2 {
    let norm = dir.x.abs() + dir.y.abs() + dir.z.abs();
    let nx = dir.x / norm;
    let ny = dir.y / norm;
    if dir.z.is_sign_positive() {
        Vec2::new(nx, ny)
    } else {
        // fold over negative z
        Vec2::new(
            (1.0 - ny.abs()) * nx.signum(),
            (1.0 - nx.abs()) * ny.signum(),
        )
    }
}

/// Decode an octahedron mapped direction vector back to the original one.
/// The output is normalized.
pub fn decode_oct(mut oct: Vec2) -> Vec3 {
    let z = 1.0 - oct.x.abs() - oct.y.abs();
    oct += oct.signum() * z.min(0.0);
    Vec3::new(oct.x, oct.y, z).normalize()
}

If you want to, I could open a PR to compare it to the current impl.

How does this compare to draco?

Draco is a mesh data compression scheme that is supported e.g. in glTF (with an extension that e.g. Blender supports out of the box) or as a standalone mesh format.

The README shows some comparisons against zipping up some common 3D formats, but based on the description the impression I got was that glb+draco would be a much closer alternative than a zipped fbx so that would be interesting to see.

Draco is not the smallest library so it might also be interesting to take a look at the code size & runtime speed of the different solutions.

EDIT: another similar library is meshoptimizer which, while the main focus seems to be on optimizing meshes for more efficient rendering, also does some tricks to reduce file size.

Question: Why have distinct index buffers per attribute?

The API exposes set_normal_triangles, set_uv_triangles, set_vertex_triangles. This allows user to use different index buffers per attribute.

In my experience, there is only a single set of indices per mesh. So all those foo_triangles arrays are duplicate of the same index buffer!

So why do they exist as separate entities? Shouldn't they be merged into a single one? It seems this could reduce memory usage as well.

[META] Preparations for 0.2: What is missing.

0.2 was planned to be released very soon.
I plan on releasing 0.2 in roughly the state it is in on 20 CET on 10 of June, because after this period I will be unable to work on the project for more than a week.
I am happy with the amount of features in this release. Is there anything missing?

Shared Index Segment Type[FEATURE]

Index segments currently make up most of the mesh size. Different approaches to reducing their size have yielded insufficient results. The previous attempts were small, universal improvements for saving all kinds of meshes. Some meshes can be however further compressed by increasing the size of one segment to shrink another. This is where SharedIndexSegment would come in.

A SharedIndexSegment is a segment that stores indices, which are identical, but would normally end up duplicated in completely different segments. A bitmask at the begging of the segment signals, which segments did the indices in the segment belong to.
This alone will not do a lot, since a large range of indices being shared across segments is very rare. This is where the cost in increasing size of some other segments comes in.

Let us imagine this hypothetical scenario:
We have a set of vertices:
[va,vb,vc,vd]
uvs:
[ua,ub,ub,ud]
combined into two triangles:
vertex index:
[0,1,2,3,1,2]
and uvs:
[1,2,3,0,2,3]
There are 5 unique combinations of vertex and uv indices:
[ (0,1) ,(1,2),(2,3),(3,0),(1,1)]
if we change the uv and vertex array to look like this:
[va,vb,vc,vd,vb]
and uv array to look like this:
[ub,uc,ud,ua,ub]
Each unique combination of uv and vertex data can be represented with a single index!
[0,1,2,3,4,2]
This has it's downsides:

1. Both the vertex and uv array now contain duplicate data.
2. The highest value in the unifed index array is now bigger.
3. Computing the new data will increase write times.
   Which is why it is not something that will fit each mesh, and should be done on a per-mesh basis. Additionally, reordering data may be not allowed for some user-generated meshes. Someone might not want their mesh data reordered. Which is why this will be a function: [unfiy_index_data].

This has the potential to drastically reduce the size of some meshes.

What is needed for this to work?

• Write the unify_index_data function.
• Find identical fragments of segments, before encoding them
• Add support for writing/reading shared index segments.

Delta encoding [FEATURE]

Currently, indices are by far the biggest (size-wise) parts of the final file(60%). While previously attempted remedies(e.g. splitting index arrays to allow lower indices to be saved with fewer bits) to help, the issue still persists. Potential good solution would be
delta encoding, modified to better fit this particular use case.

A potential approach could look something like that:

1. Delta-encoding gets assigned a compression-type (marked in the segment header).
2. A delta encoded segment will start with the following:
   2.a. A field describing the amount of elements in a segment(u64).
   2.b. Raw data precision bits(u8).
   2.c. Delta precision bits(u8)
   2.d. Delta min/max (UBA field, size of Raw data precision bits bits). IMPORTANT: preceded by sign!
3. After that, a continus array of indices, encoded as follows:
   3.a DeltaOrRaw (u1) - marks if next item is delta encoded(0) or raw(1)
   3.b.0 If delta, a Delta precision bits long number will be encoded, containing a number between Delta min and Delta max. This number should be added to the value of the last number.
   3.b.1 If Raw, a Raw data precision bits long number, encoding the value of the index.

This compression type will be selected during encoding, if it is beneficial and TMFPrecisonInfo field allow_delta_encoing is not set to false.

Possible issues:

1. This relies on neighbouring indices being mostly very similar, which is the case for most meshes (not all!).
2. May increase decode time. How much? At worst, I expect it to be ~2x slower than normal encoding. There is a slim chance it will be faster. But it is planned to be opt-out during encode, so eventual performance issues should not be a problem.

[FEATURE] Support arbitrary vertex attributes

TMF looks super cool!

But I'd not only want to store normal, positions and uvs data. Formats like glTF also store other attributes per vertex, such as: color, tangent, secondary uvs, skeletal animation bone weights.

Game engines do use more than normal position and uvs, and it seems reasonable to expect from a mesh serialization format that it supports additional attributes.

Implementation

I'm conscious this is not a trivial task, because each attribute can have an arbitrary representation. And it probably makes it hard to apply some useful heuristics. So I'm not adventuring to propose an implementation. But the most basic API I'd like is a method on TMFMesh that does:

fn set_attribute<T: VertexAttribute>(&mut self, attribute_id: usize, buffer: &[T]) {}
fn get_attribute<T: VertexAttribte>(&self, attribute_id: usize) -> Option<&[T]> {}

(this is an extremely primitive API that can fairly trivially be improved, not a template for a final API)

A method on VertexAttribute could provide to TMFMesh informations so that it can apply good compression heuristics, and specifically allow erasing the type (so that it could be internally stored as a Box<[u8]> and cast when accessed)

What is this for?

I'm maintaining the bevy_fbx crate and bevy is landing a new asset loader with a post-processing step. The FBX format suuuuucks and is wasteful when it comes to game assets (it even leaks private information), using an intermediary representation *that is fast to write and read from is a necessity tbh.

My fbx loader currently only supports normal positions and uvs, so I already can use TMF! However, storing tangents in the backed meshes would be super useful, it would allow not having to compute tangents at runtime, which can be fairly expensive.