eth-ait/aitviewer

A set of tools to visualize and interact with sequences of 3D data.

aitviewer's Issues

How do you display the difference in real time with different color levels?

Thanks for your great work, I now estimate five different ways to get the pose, trying to distinguish the gap size by color change. I found that the v.run() function generates the entire animation directly. If I want the color gap every frame I need to transport every frame. But v.run() is a loop and I don't know how to add color. Do you have any solution?

The effect I want is real-time animation output, with large color distinctions that are very different from GT. Something like this:


Thanks for your help and work!

Not compatible with imgui 2.0

imgui 2.0 introduced some breaking changes. Until we adapt to 2.0, we fix the version to be 1.4.1.

Position the camera to keep the whole motion in the field of view

Hi, this is a feature suggestion.

The headless rendering is convenient, but often the subject gets out of the camera field of view during the motion sequences.
It would be great to have the option to automatically position the camera such that the whole motion stays in the camera field.

Thanks for the great work :).

Text display

Hi, great works!
I would like to ask whether aitviewer support text display when rendering SMPL models?

Frame of Reference in Transformation

In the aitviewer.utils.utils.py
There is a method local_to_global

def local_to_global(poses, parents, output_format="aa", input_format="aa"):
    """
    Convert relative joint angles to global ones by unrolling the kinematic chain.
    :param poses: A tensor of shape (N, N_JOINTS*3) defining the relative poses in angle-axis format.
    :param parents: A list of parents for each joint j, i.e. parent[j] is the parent of joint j.
    :param output_format: 'aa' or 'rotmat'.
    :param input_format: 'aa' or 'rotmat'
    :return: The global joint angles as a tensor of shape (N, N_JOINTS*DOF).
    """
    assert output_format in ["aa", "rotmat"]
    assert input_format in ["aa", "rotmat"]
    dof = 3 if input_format == "aa" else 9
    n_joints = poses.shape[-1] // dof
    if input_format == "aa":
        local_oris = aa2rot(poses.reshape((-1, 3)))
    else:
        local_oris = poses
    local_oris = local_oris.reshape((-1, n_joints, 3, 3))
    global_oris = torch.zeros_like(local_oris)

    for j in range(n_joints):
        if parents[j] < 0:
            # root rotation
            global_oris[..., j, :, :] = local_oris[..., j, :, :]
        else:
            parent_rot = global_oris[..., parents[j], :, :]
            local_rot = local_oris[..., j, :, :]
            global_oris[..., j, :, :] = torch.matmul(parent_rot, local_rot)

    if output_format == "aa":
        global_oris = rot2aa(global_oris.reshape((-1, 3, 3)))
        res = global_oris.reshape((-1, n_joints * 3))
    else:
        res = global_oris.reshape((-1, n_joints * 3 * 3))
    return res

There is a two options for apply transformation matrix.

global_oris[..., j, :, :] = torch.matmul(parent_rot, local_rot) (1. multiply left)
global_oris[..., j, :, :] = torch.matmul(local_rot, parent_rot) (2. multiply right)

Normally, option2 is applied. but aitviewer uses the first convention.
Could you explain why? I think many dataset like AMASS uses the second convention.

Also, I found aitviewer uses roma as an angle-handling library which uses second convention.

Is it Possible to Save Videos Without Displaying in Viewer?

I am using the aitviewer for training and evaluating models with the SMPL sequence on a server that does not have a GUI. This setup prevents me from viewing or saving video outputs through the interactive interface. I am inquiring whether aitviewer supports off-screen video saving to overcome this limitation?

Real headless rendering

Awesome project! The rendering works great on the desktop.
But on the remote server, we may need a real headless rendering to avoid the error like:
(standalone) XOpenDistplay: cannot open display

error occurs: qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.

Hello, when I tried to install it to visualize the .ply file I got after training before, the following error appeared：

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

I tried to uninstall and install it several times, i tried: pip install aitviewer or
git clone [email protected]:eth-ait/aitviewer.git
cd aitviewer
pip install -e .
but it kept the same error, can you tell me how I can solve this error, thanks a lot !!

Some problems about load_AMASS

I found a problem after running the three programs load_AMASS.py, load_DIP_TC.py and load_DIP_IMU.py. Unlike the situation that the character models in load_DIP_TC.py and load_DIP_IMU.py are facing me, the models in load_AMASS.py in the initial stage are sideways to me, how can I improve this situation? What should be done so that the data in the amass dataset is displayed right to me as the mp4 file of renders in the website?Meanwhile, after deploying the actions in the DIP-IMU dataset in Unity first, then putting the actions in the AMASS dataset into Unity will result in a flipped situation.
What should I do to align the AMASS global framework with DIP? If I use AMASS dataset for training and DIP-IMU dataset for testing,what should I do with the AMASS dataset?

how to remove checkerboard

Thanks for your great work! I am able to render meshes successfully. While the rendered image contains the checkerboard. I wonder if there is a way to remove the checkerboard.

AssertionError： load_DIP_TC.py

when running examples\load_DIP_TC.py
reporting an error:
Traceback (most recent call last):
File "c:/Users/xxx/Desktop/Project_report/aitviewer-main/aitviewer-main/examples/load_DIP_TC.py", line 41, in
_, joints = smpl_layer(
File "D:\anaconda\envs\py3_8\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\anaconda\envs\py3_8\lib\site-packages\aitviewer\models\smpl.py", line 243, in forward
return self.fk(*args, **kwargs)
File "D:\anaconda\envs\py3_8\lib\site-packages\aitviewer\models\smpl.py", line 171, in fk
assert poses_body.shape[1] == self.bm.NUM_BODY_JOINTS * 3
AssertionError

But I can't find a way to solve it. What should I do

TRACE: Projects using the aitviewer

Dear Kaufmann, Manuel and Vechev, Velko and Mylonopoulos, Dario,

Thank you all for creating such a useful and fantastic visualization lib. As an old user of AitViewer, I'm really happy to see its new features in streaming, can't wait to try. I have recommanded AitViewer to many friends.

During developing our recent work :
TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments (CVPR2023)
project page: http://www.yusun.work/TRACE/TRACE.html
AitViewer helps a lot, especially in checking the dataset loading.
If possible, please consider to add TRACE to "Projects using the aitviewer".

Best regard,
Yu

Fail loading SMPL-H

I am trying to execute the script load_amass.py.

It expects .pkl models for SMPL-H, but the models SMPL-H for amass were released as .npz.

From MANO Download page, which models should I download, the first link or the second?
[1] Models & Code
[2] Extended SMPL+H model (used in AMASS project)

Whether I try to load the first version (.pkl) or the second version (.npz), I get the following error:

  File "visualize_seq.py", line 27, in <module>
    seq_amass = SMPLSequence.from_amass(
  File "/home/mkeller2/.pyenv/versions/skel_release_env/lib/python3.8/site-packages/aitviewer/renderables/smpl.py", line 154, in from_amass
    smpl_layer = SMPLLayer(model_type='smplh', gender=body_data['gender'].item(), device=C.device)
  File "/home/mkeller2/.pyenv/versions/skel_release_env/lib/python3.8/site-packages/aitviewer/models/smpl.py", line 56, in __init__
    self.bm = smplx.create(C.smplx_models, model_type=model_type,
  File "/home/mkeller2/.pyenv/versions/skel_release_env/lib/python3.8/site-packages/smplx/body_models.py", line 2405, in create
    return SMPLH(model_path, ext="npz", **kwargs)
  File "/home/mkeller2/.pyenv/versions/skel_release_env/lib/python3.8/site-packages/smplx/body_models.py", line 602, in __init__
    left_hand_components = data_struct.hands_componentsl[:num_pca_comps]
AttributeError: 'Struct' object has no attribute 'hands_componentsl'

Can I visualize stream data?

I have a smpl motion which comes frame by frame instead of a whole. Is there any way to visualize it in aitviewer?

qt.qpa.plugin: Could not load the Qt platform plugin "xcb"

Cannot launch the viewer.

Error:

QObject::moveToThread: Current thread (0x55963f7b4e10) is not the object's thread (0x55963fcbcfd0).
Cannot move to target thread (0x55963f7b4e10)

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/zfan/miniconda3/envs/aitviewer/lib/python3.8/site-packages/cv2/qt/plugins" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl.

Installation Issue on macOS with Apple Silicon

When trying to install aitviewer on a machine running macOS with AppleSilicon (M1 Pro), the PyQT5 dependency causes issues by seemingly being stuck in an infinite loop during installation, see Pip Install stuck on "Preparing Wheel metadata..." when trying to install PyQT5.

I have documented steps on how I was able to get around this issue and create a virtual environment with aitviewer installed in it:

1. First, install Python 3.8 and PyQT5 using Homebrew:

   brew install [email protected]
   brew install pyqt@5

   and ensure they necessary paths are discoverable:

   echo 'export PATH="/opt/homebrew/opt/qt@5/bin:$PATH"' >> ~/.zshrc
   echo 'export PATH="/opt/homebrew/opt/pyqt@5/5.15.4_1/bin:$PATH"' >> ~/.zshrc
   source ~/.zshrc
   

2. Then, create an environment with system site packages (for PyQT5):

   python3.8 -m venv --system-site-packages <ENV_PATH>

   Activate the environment with:

   source <ENV_PATH>/bin/activate

3. Install the pyqt5 manually by first upgrading pip:

   python -m pip install --upgrade pip

   N.B.: Some older versions of pip don't support some of the settings
   required below, hence the upgrade.

   Then install pyqt5 with:

   python -m pip install pyqt5 --config-settings --confirm-license= --verbose

   N.B.: This method avoids the issue of installation being stuck:
   Pip Install stuck on "Preparing Wheel metadata..." when trying to install PyQT5 - Stack Overflow

   N.B.: This step still may take up to 30 minutes to complete.

4. Install aitviewer:

   python -m pip install aitviewer
   

When I run this package on my video,have some error

Error is
QOpenGLContext::swapBuffers() called with non-exposed window,behavior is undefined
QOpenGLContext::swapBuffers() called with non-exposed window,behavior is undefined
and failed to update video frame information

Do i need to download smplx model?

       assert osp.exists(smplx_path), 'Path {} does not exist!'.format(
   AssertionError: Path ../data/smplx_models does not exist!

when i run quickstart.py i get this rerror.
I upload (https://smpl.is.tue.mpg.de/index.html) and install(https://github.com/vchoutas/smplx) smpl model but i still get this error
Do i need to download smplx model? and if so, where should it be placed?

How to release the memory after exporting the video in headless rendering?

Hi~ I encountered a memory problem when using headless rendering multiple times. The memory occupancy will accumulate till the system shuts down. I tried to reset the viewer and release the renderable nodes after saving the video. However, it does not work.

self.viewer = HeadlessRenderer()

def rendering_loop(self, ...):
     self.viewer.reset()
     obj = Skeletons(...)
     self.viewer.scene.add(obj)
     self.viewer.lock_to_node(obj)
     self.viewer.save_video(...)
     obj.release()

I checked line by line and found that the memory usage suddenly increased after the self.viewer.save_video.

Would you like to give me some suggestions to solve this?

Where can I find flame_static_embedding.pkl?

Now I can run the code examples/load_template.py with SMPL-X and MANO pkl data. But When I try to run FLAME data, I can download male, female, netrual from FLAME website, but I just can't find the related pkl data on the web. Here is the error:

No such file or directory: './smplx_models\flame\flame_static_embedding.pkl'

So the question is where can I find the releated FLAME file?

BUG: Compatible Issue

I have tried to compile aitviewer in windows and ubuntu 20.04, but encountered same problems:

1. Ubuntu: Anaconda; Python = 3.9.7;
   Error detail:
   File "/home/yingjun/anaconda3/envs/test/lib/python3.9/site-packages/_moderngl.py", line 344, in
   data = b''.join(struct.pack(self._fmt, *row) for row in value)
   TypeError: Value after * must be an iterable, not int

2. Windows 10. Anaconda; Python = 3.9.7
   Error Detail:
   File "H:\anaconda3\lib\site-packages\moderngl-5.7.2-py3.9-win-amd64.egg_moderngl.py", line 344, in
   data = b''.join(struct.pack(self._fmt, *row) for row in value)
   TypeError: Value after * must be an iterable, not int

Could you help on this and indicate which version I should use to solve this issue?

Question about 'updating the smplx_models parameter in the aitvconfig.yaml'

I'm setting up SMPL according to README.
Unable to reach 'the Configuration page' here.
Could you fix the link?

Missing pkl file for FLAME template demo

I am trying to run the load_template.py demo and get the following error:

  File "load_template.py", line 30, in <module>
    flame_template = SMPLSequence.t_pose(SMPLLayer(model_type='flame', gender='neutral', device=C.device),
  File "/home/kellerm/Code/aitviewer/aitviewer/models/smpl.py", line 56, in __init__
    self.bm = smplx.create(C.smplx_models, model_type=model_type,
  File "/home/kellerm/Code/aitviewer/aitviewer_env/lib/python3.8/site-packages/smplx/body_models.py", line 2408, in create
    return FLAME(model_path, **kwargs)
  File "/home/kellerm/Code/aitviewer/aitviewer_env/lib/python3.8/site-packages/smplx/body_models.py", line 1943, in __init__
    with open(landmark_bcoord_filename, 'rb') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/home/kellerm/Data/smplx_models/flame/flame_static_embedding.pkl'

For the Flame model, the models released on Flame website do not contain flame_static_embedding.pkl, where can I get this file? SMPLX github don't seem to give instructions about Flame.

Custom Background/Sky

Hello,

first of all, thanks for the amazing tool you developed!

Is there any easy way to customize the background (color etc), or possibly even use some custom background images/textures?

Thanks in advance!

How to render normal map?

Hi, how to render normal map with this tool?

Offset on SMPL

With the current code, the generated AMASS sequence has feet below zero and an offset compared to the mocap markers :

This is caused by the change of coordinate to get Y axis up :

aitviewer/aitviewer/renderables/smpl.py

Line 177 in 4aac6fd

# Transform root orientation and translation into our viewer's coordinate system (where Y is up).

I think the reason is that the rotation is applied to the root_joint of SMPL. Yet the root joint of SMPL is not the origin of SMPL (this detail is a common source of error). If you display SMPL in Tpose, the root joint is typically 20cm below the world origin.

For now, I removed those lines in the .from_amass method and instead apply the rotation directly on the vertices in fk()

    def fk(self):
        """Get joints and/or vertices from the poses."""
        verts, joints = self.smpl_layer(poses_root=self.poses_root,
                                        poses_body=self.poses_body,
                                        poses_left_hand=self.poses_left_hand,
                                        poses_right_hand=self.poses_right_hand,
                                        betas=self.betas,
                                        trans=self.trans)
        skeleton = self.smpl_layer.skeletons()['body'].T
        faces = self.smpl_layer.bm.faces.astype(np.int64)
        joints = joints[:, :skeleton.shape[0]]

        #Transform output vertices and joints into our viewer's coordinate system (where Y is up).
        to_y_up = torch.Tensor([[1, 0, 0], [0, 0, 1], [0, -1, 0]]).to(self.device)
        verts = torch.matmul(to_y_up.unsqueeze(0), verts.unsqueeze(-1)).squeeze()
        joints = torch.matmul(to_y_up.unsqueeze(0), joints.unsqueeze(-1)).squeeze()

This fixes the offset issue:

x11, opengl error

When I use aitviewer below error happened. it seems to be from openGL context:
I already saw previous issue and tried to fix it.

• I used python3.8.1

--------------------------------------------------------------log ---------------------------------------------------------------
Traceback (most recent call last):
File "scripts/visualize-SMPL.py", line 101, in
viewer = HeadlessRenderer()
File "/home/datasynthesizer/miniconda3/envs/simple_romp/lib/python3.8/site-packages/aitviewer/headless.py", line 38, in init
super().init(**kwargs)
File "/home/datasynthesizer/miniconda3/envs/simple_romp/lib/python3.8/site-packages/aitviewer/viewer.py", line 72, in init
self.window = base_window_cls(
File "/home/datasynthesizer/miniconda3/envs/simple_romp/lib/python3.8/site-packages/moderngl_window/context/headless/window.py", line 23, in init
self.init_mgl_context()
File "/home/datasynthesizer/miniconda3/envs/simple_romp/lib/python3.8/site-packages/moderngl_window/context/headless/window.py", line 33, in init_mgl_context
self._ctx = moderngl.create_standalone_context(require=self.gl_version_code)
File "/home/datasynthesizer/miniconda3/envs/simple_romp/lib/python3.8/site-packages/moderngl/context.py", line 1667, in create_standalone_context
ctx.mglo, ctx.version_code = mgl.create_context(glversion=require, mode=mode, **settings)
File "/home/datasynthesizer/miniconda3/envs/simple_romp/lib/python3.8/site-packages/glcontext/init.py", line 90, in create
return x11.create_context(**kwargs)
Exception: (standalone) cannot create context

Questions about using SMPL-H

I have downloaded the file (.pkl) from the website (https://mano.is.tue.mpg.de/download.php) as requested by Supported Body Models. and placed it in the correct location (... /data/smplx_models/smplh/SMPLH_FEMALE.pkl).However, when running load_AMASS.py in the examples folder, it still reports an error (I get the same error when running the official Quickstart provided):
Traceback (most recent call last):
File "/home/xxx/Download/amass-bml/aitviewer-main/examples/load_AMASS.py", line 30, in
seq_amass = SMPLSequence.from_amass(
File "/home/xxx/anaconda3/envs/ait/lib/python3.9/site-packages/aitviewer/renderables/smpl.py", line 220, in from_amass
smpl_layer = SMPLLayer(model_type="smplh", gender=body_data["gender"].item(), device=C.device)
File "/home/xxx/anaconda3/envs/ait/lib/python3.9/site-packages/aitviewer/models/smpl.py", line 63, in init
self.bm = smplx.create(
File "/home/xxx/anaconda3/envs/ait/lib/python3.9/site-packages/smplx/body_models.py", line 2402, in create
return SMPLH(model_path, **kwargs)
File "/home/xxx/anaconda3/envs/ait/lib/python3.9/site-packages/smplx/body_models.py", line 601, in init
left_hand_components = data_struct.hands_componentsl[:num_pca_comps]
AttributeError: 'Struct' object has no attribute 'hands_componentsl'

How to specify model_type for smplx ?

quickstart code

from aitviewer.renderables.smpl import SMPLSequence
from aitviewer.models.smpl import SMPLLayer
from aitviewer.viewer import Viewer
from aitviewer.configuration import CONFIG as C

C.update_conf({"smplx_models": '~/w/code/smplx/models'})

if __name__ == '__main__':
    v = Viewer()
    smpl_layer = SMPLLayer(model_type="smpl", gender="neutral")

    v.scene.add(SMPLSequence.t_pose(smpl_layer))
    v.run()

but return a error

$ python aitviewer_u/quickstart.py
Traceback (most recent call last):
  File "C:\Users\zyp\w\code\aitviewer_u\quickstart.py", line 11, in <module>
    smpl_layer = SMPLLayer(model_type="smpl", gender="neutral")
  File "C:\Users\zyp\w\sw\anaconda\envs\aitv\lib\site-packages\aitviewer\models\smpl.py", line 63, in __init__
    self.bm = smplx.create(
  File "C:\Users\zyp\w\sw\anaconda\envs\aitv\lib\site-packages\smplx\body_models.py", line 2410, in create
    raise ValueError(f'Unknown model type {model_type}, exiting!')
ValueError: Unknown model type models, exiting!

fyi, the tree output of ~/w/code/smplx/models

zyp@pc ~/w/code/smplx/models (main)
$ tree
.
|-- smpl
|   `-- SMPL_NEUTRAL.pkl
`-- smplx
    |-- SMPLX_FEMALE.npz
    |-- SMPLX_FEMALE.pkl
    |-- SMPLX_MALE.npz
    |-- SMPLX_MALE.pkl
    |-- SMPLX_NEUTRAL.npz
    |-- SMPLX_NEUTRAL.pkl
    |-- smplx_npz.zip
    `-- version.txt

Questions about design pattern for playing indefinite number of frames

Before opening the issue, I would really thank you for maintaining wonderful project I've ever seen. The codes are kindly commented and tutorials are nice!

I am developing some motion-generation projects and I need some helps with the design pattern.

I need to render indefinite numbers of frames. However, All of Node classes (renderable) especially Skeleton and SMPLSequence demands me to have a fixed number of frames before it instantiated.
But I am generating motions with autoregressive manner, that means the network receives the previous frame and outputs the next frame so that it can generate infinite number of frames.

I considered about generating 1000 frames before test and play the data, But, If I want to check further, re-generating frames is too cumbersome because I need to check my output many times.

You already made a nice suggestions
(#14 (comment))

Since I also need interactive gui in the Viewer class,
updating 1 length sequence in the Viewer.run() may seem plausible.

But, before implement in actual, could you give me any suggestions about the design pattern that I need?

Best regards,

How to disable the adptive ground level?

Hi! Thanks for developing this amazing visualization tool.

I met a problem when I tried to add a sphere to the scene. Here is my code:

sphere = trimesh.creation.icosphere(subdivisions=3, radius=1.0)
vertices = sphere.vertices
faces = sphere.faces
sphere_Meshes = Meshes(vertices, faces, gui_affine=False, color=(0.5, 0.5, 0.5, 1.0), name="obstacle")
v.scene.add(sphere_Meshes)

And here is my result:

The center of the sphere is supposed on the origin of the coordinate system. So the ground floor should cut the sphere into halves. However, the aitviewer seems to automatically help me to change the ground level to make sure all the objects are above the surface.

So how can I disable this function?

AttributeError: 'FFmpegWriter' object has no attribute '_proc'

I faced below error when I use aitviewer.
I set smplx_models but I could not find out the solutions...

C.update_conf({"smplx_models": "./data/SMPLX/"})

-------------------------------------------------------------- error log ---------------------------------------------------------------
/home/datasynthesizer/miniconda3/envs/ROMP/lib/python3.9/site-packages/aitviewer/viewer.py:1889: RuntimeWarning: divide by zero encountered in double_scalars
az_delta = np.radians(rotation_degrees) / frames
Rendering frames: 0it [00:00, ?it/s]
Traceback (most recent call last):
File "/home/datasynthesizer/Documents/IA_data/InstantAvatar/scripts/visualize-SMPL.py", line 141, in
viewer.save_video(video_dir=f"{args.path}/output.mp4", output_fps=args.fps)
File "/home/datasynthesizer/miniconda3/envs/ROMP/lib/python3.9/site-packages/aitviewer/headless.py", line 38, in save_video
self.export_video(
File "/home/datasynthesizer/miniconda3/envs/ROMP/lib/python3.9/site-packages/aitviewer/viewer.py", line 1957, in export_video
writer.close()
File "/home/datasynthesizer/miniconda3/envs/ROMP/lib/python3.9/site-packages/skvideo/io/ffmpeg.py", line 432, in close
if self._proc is None: # pragma: no cover
AttributeError: 'FFmpegWriter' object has no attribute '_proc'

Can not run multiple viewers sequencially

With the current code, the following crashes on the last line:

   v = Viewer()
   v.run()
   v = Viewer()
   v.run()

With the error:

Traceback (most recent call last):
  File "analysis/joint_projection_unposed.py", line 164, in <module>
    v.run()
  File "/home/kellerm/Code/aitviewer/aitviewer/viewer.py", line 274, in run
    self._init_scene()
  File "/home/kellerm/Code/aitviewer/aitviewer/viewer.py", line 260, in _init_scene
    self.scene.make_renderable(self.ctx)
  File "/home/kellerm/Code/aitviewer/aitviewer/scene/scene.py", line 149, in make_renderable
    r.make_renderable(self.ctx)
  File "/home/kellerm/Code/aitviewer/aitviewer/scene/node.py", line 359, in _decorator
    func(self, *args, **kwargs)
  File "/home/kellerm/Code/aitviewer/aitviewer/renderables/meshes.py", line 382, in make_renderable
    self.smooth_vao = ctx.vertex_array(self.smooth_prog,
  File "/home/kellerm/Code/skeleton_inference/skel_env/lib/python3.8/site-packages/moderngl/context.py", line 1140, in vertex_array
    return self._vertex_array(*args, **kwargs)
  File "/home/kellerm/Code/skeleton_inference/skel_env/lib/python3.8/site-packages/moderngl/context.py", line 1169, in _vertex_array
    res.mglo, res._glo = self.mglo.vertex_array(program.mglo, content, index_buffer_mglo,
moderngl.error.Error: the program belongs to a different context
Segmentation fault (core dumped)

The problem is in aitviewer/enderables/meshes.py : make_renderable : l.351, the shader program is not reloaded with the new context. The old shader program instance is used and it has the context of the first viewer, hence the mismatch.

I see that the shader program is not reloaded everytime l.351 is executed thanks to a cache @functools.lru_cache() in aitviewer/shaders.py. I guess this cache should be cleared in between two viewers creation.

This is the hack I have so far that fixes it:

def clear_shader_cache():
        # Import and list all the functions that need the cache to be cleared
        from aitviewer.shaders import get_smooth_lit_with_edges_program, get_flat_lit_with_edges_program, get_smooth_lit_texturized_program, get_simple_unlit_program, get_cylinder_program, get_screen_texture_program, get_chessboard_program
        for f in [get_smooth_lit_with_edges_program, get_flat_lit_with_edges_program, get_smooth_lit_texturized_program, get_simple_unlit_program, get_cylinder_program, get_screen_texture_program, get_chessboard_program]:
           # Try clearing the cache
            try:
                f.cache_clear()
                print(f'Clearing cache for {f}')
            except:
                pass

    #Viewer.py
    def run(self, *args, log=True):
        ...
        self.on_close()
        self.window.destroy()
        self.clear_shader_cache() # I add this line to clear the shader cache 
        ...
        
        
 With that, the following code works:
 

 for i in range(10):
    v = Viewer()
    v.run()
    # closing manually the window

       
   But still this tends to crash randomly with a segfault and no error message if I close the window too fast.

Posture generation color contrast

Thank you for such an experienced work, the visualization has tortured me to the point of unconsciousness!

I am currently working on joint prediction. I have seen comparisons between other people’s work and GT. The greater the difference, the more obvious the color. Do you have any examples of this?

Thank you so much!

Configuring font size

Hello guys,

I've noticed that the ImGui font can appear very blurry on high-resolution displays, so I dug into the code a bit and have two findings:

1. Currently, we cannot set the font scale in the configuration.
2. In the current implementation, the custom font is added without clearing the built-in fonts. Consequently, the font scale is only applied to the scene editor GUI and not to the playback GUI or menu items.

I've made a small tweak to make font size configurable and would be happy to share the code if you’d like. Please let me know if a pull request would be helpful.

Best, Tianjian

normalize_root DOES NOT WORK with SMPLLayer

Hi, Thank you for maintaining nice projects

normalize_root option is not properly working on the SMPLSequence class.

Because when calling self.smpl_layer.forward() in the SMPLSequence.fk()
normalize_root option does not passed to the smpl_layer so that
normalize_root is always the false in the SMPLLayer

aitviewer/aitviewer/renderables/smpl.py

Lines 400 to 407 in 65b38da

	verts, joints = self.smpl_layer(
	poses_root=poses_root,
	poses_body=poses_body,
	poses_left_hand=poses_left_hand,
	poses_right_hand=poses_right_hand,
	betas=betas,
	trans=trans,
	)

I think this code should changed like this

verts, joints = self.smpl_layer(
    poses_root=poses_root,
    poses_body=poses_body,
    poses_left_hand=poses_left_hand,
    poses_right_hand=poses_right_hand,
    betas=betas,
    trans=trans,
    normalize_root=self._normalize_root,
)

Thank you

AssertionError: Path ../data/smplx_models does not exist!

I have already pip install smplx [all], pip install aitviewer, but both Python 3.8 and Python 3.9 are trying 'Quickstart' from:
https://eth-ait.github.io/aitviewer/#quickstart
reporting an error:
Traceback (most recent call last):
File "/home/xxx/test/test/ait.py", line 6, in
v.scene.add(SMPLSequence.t_pose())
File "/home/xxx/anaconda3/envs/ait/lib/python3.9/site-packages/aitviewer/renderables/smpl.py", line 316, in t_pose
smpl_layer = SMPLLayer(model_type="smplh", gender="neutral")
File "/home/xxx/anaconda3/envs/ait/lib/python3.9/site-packages/aitviewer/models/smpl.py", line 63, in init
self.bm = smplx.create(
File "/home/xxx/anaconda3/envs/ait/lib/python3.9/site-packages/smplx/body_models.py", line 2404, in create
return SMPLX(model_path, **kwargs)
File "/home/xxx/anaconda3/envs/ait/lib/python3.9/site-packages/smplx/body_models.py", line 969, in init
assert osp.exists(smplx_path), 'Path {} does not exist!'.format(
AssertionError: Path ../data/smplx_models does not exist!

I also encountered almost the same issue when running the 'load_DIP_IMU. py' file in the examples folder. Is there a way for me to add SMPL and SMPLX .pkl files to the program? Could you give me an example?

Support for Multiple UV Maps

Hi,

Thanks for your work! I am using MANO with HTML to generate the hand texture map. The HTML UV map does not match the MANO vertices default mapping. It assumes 907 vertices.
This feature is supported in frameworks like Pytorch3d and Open3D but it looks like Pyrender does not support it.
Is there a way to render the hand texture in aitviewer?

Kind Regards,
Marsil.

some data cannot be seen

I successfully ran aitviewer, but it seems that the editor on the left side of the box cannot be dragged and some data cannot be seen

Exception: (detect) glXGetCurrentContext: cannot detect OpenGL context

It is just a script to open the viewer without any object inside but it doesn't seem to open openGL context:

➜  aitviewer git:(main) ✗ python test.py
libGL error: MESA-LOADER: failed to open iris (search paths /usr/lib/x86_64-linux-gnu/dri:\$${ORIGIN}/dri:/usr/lib/dri)
libGL error: failed to load driver: iris
libGL error: MESA-LOADER: failed to open iris (search paths /usr/lib/x86_64-linux-gnu/dri:\$${ORIGIN}/dri:/usr/lib/dri)
libGL error: failed to load driver: iris
libGL error: MESA-LOADER: failed to open swrast (search paths /usr/lib/x86_64-linux-gnu/dri:\$${ORIGIN}/dri:/usr/lib/dri)
libGL error: failed to load driver: swrast
Traceback (most recent call last):
  File "test.py", line 5, in <module>
    v = Viewer()
  File "/home/zfan/aitviewer/aitviewer/viewer.py", line 139, in __init__
    self.window = base_window_cls(
  File "/home/zfan/aitviewer/aitviewer/utils/pyqt5_window.py", line 107, in __init__
    self.init_mgl_context()
  File "/home/zfan/anaconda3/envs/instant/lib/python3.8/site-packages/moderngl_window/context/base/window.py", line 170, in init_mgl_context
    self._ctx = moderngl.create_context(require=self.gl_version_code)
  File "/home/zfan/anaconda3/envs/instant/lib/python3.8/site-packages/moderngl/context.py", line 2004, in create_context
    ctx.mglo, ctx.version_code = mgl.create_context(glversion=require, mode=mode, **settings)
  File "/home/zfan/anaconda3/envs/instant/lib/python3.8/site-packages/glcontext/__init__.py", line 90, in create
    return x11.create_context(**kwargs)
Exception: (detect) glXGetCurrentContext: cannot detect OpenGL context

Invalid DOI

Just for info, the doi you give in your bibtex on the readme does not send to a valid address : https://doi.org/10.5281/zenodo.1234