data/objaverse_holodeck format - triangles, scales, orientation etc

[MichaelRabinovich]

data/objaverse_holodeck format - triangles, scales, orientation etc

[YueYANG1996]

We will address running the code on Windows soon after the holiday.
For your question about using your pipeline, it might be very challenging and time-consuming to make it work.
Here are some reasons:

1. The Objaverse assets can be very large and make it hard to load many such assets at the same time. Therefore, we did some mesh reduction to optimize the asset. That's why the faces, vertices, etc., differ from the original asset.
2. The scale and rotation of the original Objaverse assets are very noisy. We've tried human annotations on the assets, but they still have a lot of errors. We finally used GPT-4 to annotate the asset and get the processed assets in 09_23_combine_scale.
3. The annotations you saw in the JSON file like pose_z_rot_angle are outdated and not useful to you.
4. The asset format in 09_23_combine_scale is for AI2-Thor. I think you can recover the 3D information from the pickle file, but it can be very tricky.

So, I suggest waiting until I hear from the Thor team on how to run on Windows.

[MichaelRabinovich]

Thank you for the swift response. The windows issue is not my only reason to use the assets as is.
I have no interest in using the original objaverse and would just like to use your processed objaverse assets.
How do I access the triangles/faces of your processed meshes?

[YueYANG1996]

Okay, got it. I prompt GPT-4 to get a script to convert the pkl.gz into .obj file:

# Full code to convert the pkl.gz data to an .obj file

import gzip
import pickle
import numpy as np
import os

def load_pkl_gz(file_path):
    """Load a .pkl.gz file."""
    with gzip.open(file_path, 'rb') as f:
        return pickle.load(f)

def extract_vertices(vertices_data):
    """Extract vertices into a NumPy array from the given data format."""
    return np.array([[v['x'], v['y'], v['z']] for v in vertices_data])

def create_faces(triangles_data):
    """Create faces (triangles) from the given indices."""
    return triangles_data.reshape(-1, 3)

def save_to_obj(vertices, faces, file_path):
    """Save vertices and faces to an OBJ file."""
    with open(file_path, 'w') as file:
        for v in vertices:
            file.write(f"v {v[0]} {v[1]} {v[2]}\n")
        for f in faces:
            # OBJ files are 1-indexed
            file.write(f"f {f[0]+1} {f[1]+1} {f[2]+1}\n")

def convert_pkl_gz_to_obj(input_file_path, output_file_path):
    """Convert a .pkl.gz file to an .obj file."""
    # Load the .pkl.gz file
    data = load_pkl_gz(input_file_path)

    # Extracting vertices and triangles (faces) from the data
    vertices = np.array(data['vertices'])
    triangles = np.array(data['triangles'])

    # Process the data
    vertices_array = extract_vertices(vertices)
    faces_array = create_faces(triangles)

    # Saving to .obj file
    save_to_obj(vertices_array, faces_array, output_file_path)

    return output_file_path

# File paths
input_file_path = '/mnt/data/0a0be10ec4974c8f932818d0a7472702.pkl.gz'
output_file_path = '/mnt/data/visualized_mesh.obj'

# Convert and save the .obj file
converted_obj_path = convert_pkl_gz_to_obj(input_file_path, output_file_path)

Then you can visualize the asset in blender, for example:

Then, within the blender, you can map the textures (albedo, emission, normal) to the mesh.

For running on Windows, since AI2-THOR only supports macOS 10.9+ or Ubuntu 14.04+, you can try installing Ubuntu on Windows as an alternative.