real-stanford/universal_manipulation_interface

Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots

universal_manipulation_interface's Issues

polymetis installation problems

After I configured the prerequisites according to the installation document, there was a version conflict between the python=3.8 version and the umi environment (python=3.9) when installing polymetis. How should I solve it?

What model is the robotic arm

Can it be replaced with others, and what are the performance requirements for the robotic arm?
thanks

Epson 6DoF robot arm

Hi @cheng-chi ,

We have a pair of Epson C4-A901S and we want to use it to assemble the UMI.
As far as I know, it need one more connection device which control the gripper, what the device is and how to order it? Thanks.

Can't download data/zarr_datasets

Hi, congratulations on the amazing work!

I tried to download the zarr_datasets, but it seems I don't have the access. Then I tried to checked it out in the website and it said I don't have the permission. Is that a problem of my network or just the dataset is not currently released to public?

Thanks

Robot control and Interface setup

Error encountered while executing "train.py"

This error appears to be an SSL error, but how can I fix it so that it runs without warning.

想花钱学习下

有国内的大神，出一个视频课程吗，在国内怎么买这些硬件，怎么跑起来这个程序，最后使用demo演示下
愿意花钱跟着学习下

Error encountered while executing 02_create_map.py

When I execute python run_slam_pipeline.py example_demo_session, I found that I encountered an error when executing 02_create_map.py, which prevented me from getting map_atlas.osa
The log is as follows:
slam_stdout.txt
slam_stderr.txt
I don’t know much about SLAM and I’m not sure what the problem is.

how session folders look like?

I have forked your project ,but I have no idea how to create a session,I'm already complete record my video data(mapping video,gripper calibration video, task video). How to name them and put them to the correct struct session folder?

about the example data

Hi @cheng-chi ,

I am going download the data using this command:
(umi)$ wget --recursive --no-parent --no-host-directories --cut-dirs=2 --relative --reject="index.html*" https://real.stanford.edu/umi/data/example_demo_session/

I'm having a network problem:

--2024-03-03 16:30:23--  https://real.stanford.edu/umi/data/example_demo_session/
Connecting to 127.0.0.1:7890... connected.
Unable to establish SSL connection.

Is there any other way to download?

Question about low dim keys

Good afternoon,

Firstly, I would like to thank you for your outstanding project and the well-documented code!

I have a query regarding the provided example for a single robot arm. I noticed that the key robot0_eef_rot_axis_angle_wrt_start in the shape meta represents the robot's end effector orientation relative to the episode's starting orientation. From the paper, it seems that you have used relative transforms. Should this key be removed when a relative trajectory is used?

Additionally, I observed in the code that when relative transforms are used, the last position obs_dict[f"robot{robot_id}_eef_pos"][-1] of the end effector in the history before normalization appears to be always set to an identity matrix. Is this an intentional design for compatibility with other representations, or could it be a bug?

Difference of training configs

Hi, thank you for sharing your inspiring work. I am wondering what is the primary policy difference between train_diffusion_unet_timm_umi_workspace and train_diffusion_unet_image_workspace? In my understanding, they both condition on visual and proprioception observations to predict robot actions. Aside from variations in training hyperparameters, are there any specific design features intended for the UMI task?

camera_trajectory.csv always 50 line?

When analyzing mp4 data, why does the generated camera_trajectory.csv always contain 50 entries? Did you perform sampling? How do I need to debug?

uvccamera.data assignment issue

Hello, thank you for your brilliant work,
But I was confused when I checked this code snippet. Is this a wrong assignment or just the naming prob.?

what is mean of dist_to_center ?

I get some erros .

Terminal output:

Traceback (most recent call last):
  File "/home/marcuswu/prj/universal_manipulation_interface/scripts/calibrate_slam_tag.py", line 117, in <module>
    main()
  File "/home/marcuswu/miniforge3/envs/umi/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/marcuswu/miniforge3/envs/umi/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/marcuswu/miniforge3/envs/umi/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/marcuswu/miniforge3/envs/umi/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/marcuswu/prj/universal_manipulation_interface/scripts/calibrate_slam_tag.py", line 94, in main
    all_slam_tag_pos = all_tx_slam_tag[:,:3,3]
IndexError: too many indices for array: array is 1-dimensional, but 3 were indexed

in file slam_stdout.txt:

Mask img size mismatch! Converting [2704 x 2028] to [960 x 720]

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular-Inertial
Loading settings from /ORB_SLAM3/Examples/Monocular-Inertial/gopro10_maxlens_fisheye_setting_v1_720.yaml
	-Loaded camera 1
	-Loaded image info
	-Loaded IMU calibration
	-Loaded ORB settings
	-Loaded viewer settings
	-Loaded Atlas settings
	-Loaded misc parameters
----------------------------------
SLAM settings: 
	-Camera 1 parameters (Kannala-Brandt): [ 282.907 282.907 480 360 -0.0174584 -0.0280825 0.0128327 -0.0045198 ]
	-Original image size: [ 960 , 720 ]
	-Current image size: [ 960 , 720 ]
	-Sequence FPS: 60
	-Gyro noise: 0.0015
	-Accelerometer noise: 0.017
	-Gyro walk: 5e-05
	-Accelerometer walk: 0.0055
	-IMU frequency: 200
	-Features per image: 1250
	-ORB scale factor: 1.2
	-ORB number of scales: 8
	-Initial FAST threshold: 20
	-Min FAST threshold: 7


Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Initialization of Atlas from file: /map/map_atlas.osa
Starting to read the save binary file
End to load the save binary file
Change to map with id: 0
Atlas loaded!
There are 1 cameras in the atlas
Camera 0 is fisheye
Discard points further than 20 m from current camera
Video opened using backend FFMPEG
There are 533 frames in total
video FPS 59.9401
line 1873 keyframes 3MapPoints302
line 1877
mlFrameTimes.size()= 0
KeyFrame::nNextId=6
Relocalized!!
INIT_RELOCALIZE success!
vpKFs.back()->mpImuPreintegrated0x562f1fe18410
pKFCur->mTimeStamp0
vpKFs.front()->mTimeStamp-0.283608
vpKFs.back()->mTimeStamp-0.016675
pKFcur->GetMapPointMatches().size()5006
1691 pKFcur->GetVelocity() 0
0
0
pKFcur->mnId6
Video FPS: 59.9401
ORB-SLAM 3 running at: 21.7047 FPS
cap.read failed!
Shutdown
mpLocalMapper is not finished
mpLoopCloser is not finished

Saving camera trajectory to /data/camera_trajectory.csv ...
There are 1 maps in the atlas
  Map 0 has 7 KFs

CSV camera trajectory saved!

I really need some help!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Error encountered while executing python run_slam_pipeline.py example_demo_session

############# 05_run_calibrations ###########
Traceback (most recent call last):
File "/home/zxlong10/ALOHA/universal_manipulation_interface/scripts_slam_pipeline/05_run_calibrations.py", line 67, in
main()
File "/opt/anaconda3/envs/zxl_umi/lib/python3.9/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/opt/anaconda3/envs/zxl_umi/lib/python3.9/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/opt/anaconda3/envs/zxl_umi/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/anaconda3/envs/zxl_umi/lib/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/zxlong10/universal_manipulation_interface/scripts_slam_pipeline/05_run_calibrations.py", line 33, in main
assert tag_path.is_file()
AssertionError
Traceback (most recent call last):
File "/home/zxlong10/universal_manipulation_interface/run_slam_pipeline.py", line 120, in
main()
File "/opt/anaconda3/envs/zxl_umi/lib/python3.9/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/opt/anaconda3/envs/zxl_umi/lib/python3.9/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/opt/anaconda3/envs/zxl_umi/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/anaconda3/envs/zxl_umi/lib/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/zxlong10/universal_manipulation_interface/run_slam_pipeline.py", line 106, in main
assert result.returncode == 0
AssertionError

Help reproducing cup arranging task

I'm trying to reproduce the cup arranging task using the pre-trained checkpoint. When I run it my arm starts to go nuts (writhe in all directions).

To debug it, I looked at the outputs of policy for the first observation. Here is the output of the policy that's been passed through get_real_umi_action:

action = 
[[ 0.1914, -0.0103,  0.0675, -0.158 ,  0.4563,  0.0026,  0.0903],
 [ 0.332 , -0.0674,  0.2605, -0.1588,  1.7837,  0.9145,  0.0882],
 [ 0.106 ,  0.0006,  0.3121, -0.7326,  0.67  ,  0.0385,  0.0906],
 [ 0.1506, -0.0715,  0.3616, -0.0239,  0.5017,  0.2986,  0.0906],
 [ 0.5219,  0.031 ,  0.0285, -0.3815,  0.1622, -0.2435,  0.0841],
 [ 0.6206, -0.0691,  0.5133, -0.9307, -0.3093, -0.2785,  0.0885],
 [ 0.4292,  0.0545,  0.2291,  0.0067,  0.9565, -0.3452,  0.0906],
 [ 0.3204,  0.137 ,  0.3674,  0.3146,  0.5091,  0.4602,  0.0906],
 [ 0.4759,  0.015 ,  0.227 ,  0.3432,  0.8916, -0.4273,  0.0897],
 [ 0.2003, -0.1353,  0.1784, -0.1041,  1.6625,  0.6456,  0.0808],
 [ 0.1624,  0.0373,  0.1323,  0.4304,  0.989 , -0.0285,  0.0836],
 [ 0.1347, -0.0015, -0.157 ,  0.3469,  2.1637, -0.4349,  0.0859],
 [ 0.2741, -0.1414,  0.3928,  0.2504,  0.8465,  0.2287,  0.0906],
 [ 0.2959,  0.1048,  0.2066,  0.786 ,  0.3266,  0.3725,  0.0906],
 [ 0.0924,  0.0105,  0.1356,  0.53  ,  0.884 ,  0.0471,  0.0903],
 [ 0.0216,  0.0544,  0.1702,  0.5112,  1.0929, -0.1559,  0.0905]]

If I understand the format of action correctly, it represents (x, y, z, rx, ry, rz, gripper pos) in meters/rads in EE space, since the relative offset has already been applied to the base pose. To me the actions explain why the arm is writhing, since if you interpret the data that way it's a really large change in pose from frame to frame and it doesn't seem to be following much of a trajectory except for the gripper pos. The rest of the values look somewhat random.

As for the observation I'm passing to the policy, I'm giving it:

obs['robot0_eef_pos'] = [
  [0.3079, 0.002 , 0.2051],
  [0.3079, 0.002 , 0.2051]]
obs['robot0_eef_rot_axis_angle'] = [
  [0.0118, 0.8268, 0.0094],
  [0.0118, 0.8268, 0.0094]]
obs['camera0_rgb] = image below

I will keep digging but I was wondering if anything stood out to you as wrong. I'm not using one of the supported arms (ViperX), so there might be an issue with my code downstream of the policy, but since I wasn't able to make sense of the policy output I wanted to check here first in case I'm misunderstanding something.

P.S. thanks for the advice on the IK filtering earlier. I realize now my issue was with how I was using EE positioning with my robot arm

The installation height of the mechanical arm

We are trying to deploy the mechanical arm for testing, and then we don't know how to place the height of the mechanical arm and the distance between the two mechanical arms to fit the activity range of people. Can everyone help me?

Gripper for Franka

Hi @cheng-chi, this work is incredible!
I would like to ask if the UMI is designed to be used with the Schunk WSG-50 gripper?
I want to reproduce UMI on a Franka robot, does that mean I have to buy a WSG-50 gripper and replace Franka's original gripper with the WSG-50?

CondaEnvException: Pip failed

配置conda环境报错：
Successfully built spnav robosuite
Failed to build egl-probe
Installing collected packages: ur-rtde, spnav, py-spy, opencensus-context, nvidia-ml-py, monotonic, glfw, fastcrc, egl-probe, distlib, colorful, wrapt, werkzeug, v4l2py, threadpoolctl, termcolor, tensorboardX, tensorboard-data-server, rpds-py, python-xlib, PyExifTool, pydantic, pyasn1, prometheus-client, platformdirs, imageio-ffmpeg, imageio, imagecodecs, h5py, grpcio, googleapis-common-protos, fasteners, evdev, cachetools, blessed, absl-py, virtualenv, smart-open, rsa, referencing, pynput, pyasn1-modules, markdown, gpustat, free-mujoco-py, atomics, tensorboard, robosuite, jsonschema-specifications, google-auth, aiohttp-cors, robomimic, jsonschema, google-api-core, ray, opencensus
Running setup.py install for egl-probe: started
Running setup.py install for egl-probe: finished with status 'error'

Pip subprocess error:
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [356 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-39
creating build/lib.linux-x86_64-cpython-39/egl_probe
copying egl_probe/get_available_devices.py -> build/lib.linux-x86_64-cpython-39/egl_probe
copying egl_probe/init.py -> build/lib.linux-x86_64-cpython-39/egl_probe
running egg_info
writing egl_probe.egg-info/PKG-INFO
writing dependency_links to egl_probe.egg-info/dependency_links.txt
writing top-level names to egl_probe.egg-info/top_level.txt
reading manifest file 'egl_probe.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'egl_probe.egg-info/SOURCES.txt'
/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad' is absent from the packages configuration.
!!

          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad' to be distributed and are
          already explicitly excluding 'egl_probe.glad' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.EGL' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.EGL' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.EGL' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.EGL' to be distributed and are
          already explicitly excluding 'egl_probe.glad.EGL' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.KHR' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.KHR' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.KHR' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.KHR' to be distributed and are
          already explicitly excluding 'egl_probe.glad.KHR' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.X11' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.X11' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.X11' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.X11' to be distributed and are
          already explicitly excluding 'egl_probe.glad.X11' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.X11.extensions' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.X11.extensions' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.X11.extensions' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.X11.extensions' to be distributed and are
          already explicitly excluding 'egl_probe.glad.X11.extensions' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.glad' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.glad' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.glad' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.glad' to be distributed and are
          already explicitly excluding 'egl_probe.glad.glad' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  copying egl_probe/CMakeLists.txt -> build/lib.linux-x86_64-cpython-39/egl_probe
  copying egl_probe/query_devices.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe
  copying egl_probe/test_device.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad
  copying egl_probe/glad/egl.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe/glad
  copying egl_probe/glad/gl.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe/glad
  copying egl_probe/glad/glx_dyn.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe/glad
  copying egl_probe/glad/linmath.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/EGL
  copying egl_probe/glad/EGL/eglplatform.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/EGL
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/KHR
  copying egl_probe/glad/KHR/khrplatform.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/KHR
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/X.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/XKBlib.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xatom.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xfuncproto.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xfuncs.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xlib.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/XlibConf.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xlibint.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xmd.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xosdefs.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xproto.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xprotostr.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xutil.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/cursorfont.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/keysym.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/keysymdef.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/XKB.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/XKBstr.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/XShm.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/Xext.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/extutil.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/shape.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/shapeconst.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/shm.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/glad
  copying egl_probe/glad/glad/egl.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/glad
  copying egl_probe/glad/glad/gl.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/glad
  copying egl_probe/glad/glad/glx.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/glad
  running build_ext
  Traceback (most recent call last):
    File "/tmp/pip-install-lrh9tyyi/egl-probe_b98446fc758045bd9144862418e416d6/setup.py", line 21, in run
      out = subprocess.check_output(['cmake', '--version'])
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/subprocess.py", line 424, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/subprocess.py", line 505, in run
      with Popen(*popenargs, **kwargs) as process:
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/subprocess.py", line 951, in __init__
      self._execute_child(args, executable, preexec_fn, close_fds,
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/subprocess.py", line 1837, in _execute_child
      raise child_exception_type(errno_num, err_msg, err_filename)
  FileNotFoundError: [Errno 2] No such file or directory: 'cmake'
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-lrh9tyyi/egl-probe_b98446fc758045bd9144862418e416d6/setup.py", line 50, in <module>
      setup(
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 368, in run
      self.run_command("build")
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-lrh9tyyi/egl-probe_b98446fc758045bd9144862418e416d6/setup.py", line 23, in run
      raise RuntimeError("CMake must be installed.")
  RuntimeError: CMake must be installed.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for egl-probe
error: subprocess-exited-with-error

× Running setup.py install for egl-probe did not run successfully.
│ exit code: 1
╰─> [371 lines of output]
running install
/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

          ********************************************************************************
          Please avoid running ``setup.py`` directly.
          Instead, use pypa/build, pypa/installer or other
          standards-based tools.
  
          See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
          ********************************************************************************
  
  !!
    self.initialize_options()
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-39
  creating build/lib.linux-x86_64-cpython-39/egl_probe
  copying egl_probe/get_available_devices.py -> build/lib.linux-x86_64-cpython-39/egl_probe
  copying egl_probe/__init__.py -> build/lib.linux-x86_64-cpython-39/egl_probe
  running egg_info
  writing egl_probe.egg-info/PKG-INFO
  writing dependency_links to egl_probe.egg-info/dependency_links.txt
  writing top-level names to egl_probe.egg-info/top_level.txt
  reading manifest file 'egl_probe.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file 'egl_probe.egg-info/SOURCES.txt'
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad' to be distributed and are
          already explicitly excluding 'egl_probe.glad' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.EGL' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.EGL' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.EGL' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.EGL' to be distributed and are
          already explicitly excluding 'egl_probe.glad.EGL' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.KHR' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.KHR' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.KHR' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.KHR' to be distributed and are
          already explicitly excluding 'egl_probe.glad.KHR' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.X11' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.X11' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.X11' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.X11' to be distributed and are
          already explicitly excluding 'egl_probe.glad.X11' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.X11.extensions' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.X11.extensions' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.X11.extensions' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.X11.extensions' to be distributed and are
          already explicitly excluding 'egl_probe.glad.X11.extensions' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'egl_probe.glad.glad' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'egl_probe.glad.glad' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'egl_probe.glad.glad' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'egl_probe.glad.glad' to be distributed and are
          already explicitly excluding 'egl_probe.glad.glad' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  copying egl_probe/CMakeLists.txt -> build/lib.linux-x86_64-cpython-39/egl_probe
  copying egl_probe/query_devices.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe
  copying egl_probe/test_device.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad
  copying egl_probe/glad/egl.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe/glad
  copying egl_probe/glad/gl.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe/glad
  copying egl_probe/glad/glx_dyn.cpp -> build/lib.linux-x86_64-cpython-39/egl_probe/glad
  copying egl_probe/glad/linmath.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/EGL
  copying egl_probe/glad/EGL/eglplatform.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/EGL
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/KHR
  copying egl_probe/glad/KHR/khrplatform.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/KHR
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/X.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/XKBlib.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xatom.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xfuncproto.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xfuncs.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xlib.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/XlibConf.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xlibint.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xmd.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xosdefs.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xproto.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xprotostr.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/Xutil.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/cursorfont.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/keysym.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  copying egl_probe/glad/X11/keysymdef.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/XKB.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/XKBstr.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/XShm.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/Xext.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/extutil.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/shape.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/shapeconst.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  copying egl_probe/glad/X11/extensions/shm.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/X11/extensions
  creating build/lib.linux-x86_64-cpython-39/egl_probe/glad/glad
  copying egl_probe/glad/glad/egl.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/glad
  copying egl_probe/glad/glad/gl.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/glad
  copying egl_probe/glad/glad/glx.h -> build/lib.linux-x86_64-cpython-39/egl_probe/glad/glad
  running build_ext
  Traceback (most recent call last):
    File "/tmp/pip-install-lrh9tyyi/egl-probe_b98446fc758045bd9144862418e416d6/setup.py", line 21, in run
      out = subprocess.check_output(['cmake', '--version'])
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/subprocess.py", line 424, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/subprocess.py", line 505, in run
      with Popen(*popenargs, **kwargs) as process:
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/subprocess.py", line 951, in __init__
      self._execute_child(args, executable, preexec_fn, close_fds,
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/subprocess.py", line 1837, in _execute_child
      raise child_exception_type(errno_num, err_msg, err_filename)
  FileNotFoundError: [Errno 2] No such file or directory: 'cmake'
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-lrh9tyyi/egl-probe_b98446fc758045bd9144862418e416d6/setup.py", line 50, in <module>
      setup(
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/command/install.py", line 79, in run
      return orig.install.run(self)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 697, in run
      self.run_command('build')
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/home/zdx/anaconda3/envs/umi/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-lrh9tyyi/egl-probe_b98446fc758045bd9144862418e416d6/setup.py", line 23, in run
      raise RuntimeError("CMake must be installed.")
  RuntimeError: CMake must be installed.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> egl-probe

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

failed

CondaEnvException: Pip failed

Dataset Structure Probelm

Hello dear authors,

I've run the SLAM pipeline and got the generated dataset for diffusion policy training in the form of the following six folders:

1. camera0_rgb
2. robot0_demo_start_pose
3. robot0_demo_end_pose
4. robot0_eef_pose
5. robot0_eef_rot_axis_angle
6. robot0_gripper_width

I'm confused about the contents of the six folders for most of them are binary files, and her are my questions:

1. For robot0_demo_start_pose and robot0_demo_end_pose, what's the structure of the pose information, and what's the definition of 'start' and 'end' ? Are there any specific timestamps?
2. Is robot0_eef_pose in the form of 3D Cartesian Coordinate and robot0_eef_rot_axis_angle in the form of quarternion?
3. Do camera0_rgb, robot0_demo_start_pose and robot0_demo_end_pose act as At, and robot0_eef_pose, robot0_eef_rot_axis_angleandrobot0_gripper_widthact asOt` in the input of the diffusion policy?
4. What's the exact structure of the input and output of the SLAM pipeline?
5. What's the function of the marker in the Mapping Video step in data collection? Is it only for the scale calibration or it also acts as a reference for global localization of the end effector?

Thank you so much for your patience!

Link to SLAM marker access restricted

I'm trying to find the data collection instructions (https://swanky-sphere-ad1.notion.site/UMI-Data-Collection-Tutorial-4db1a1f0f2aa4a2e84d9742720428b4c?pvs=4) and it has a link to a aruco marker to help calibrate (https://file.notion.so/f/f/9bd858e8-8338-4b7d-9a7e-e005b4302446/53114102-4cdd-407c-a10a-d230b1adfbbd/table_marker.pdf?id=245a0cf6-9d3a-4c3d-874e-a53e124bc6be&table=block&spaceId=9bd858e8-8338-4b7d-9a7e-e005b4302446&expirationTimestamp=1708552800000&signature=2jjsgMCRq3SjR0vJyZIMz9dfBqo947ISo0-NnSDUQQc&downloadName=table_marker.pdf). When I click this link I get the following error:

Module produces "ones"

Hello,

It seems that this module produces "ones" for any input. Please check the following white board test:

And I think it is not reasonable.

Thanks

code files

Hello! I have read your paper and code and am very interested in them. Your README file is quite concise; could you possibly produce a more detailed version of the README, or release a video explaining the code files? Thank you for your efforts.

no camera_trajectory.csv

I have recorded 100 videos, and 54 of them have error messages: no camera_trajectory.csv. What is the cause of this error? Is it the chaotic environment?

Question: Hardware Execution Latencies

From the associated paper, it seems that hardware latencies can play a big part in the overall performance of the policy. I noticed that in eval_real.py there is a action_exec_latency variable that is used for filtering actions inferenced from the policy. At the same time, there is a separate latency value robot_action_latency, that is used in bimanul_umi_env.py. What is the difference between these two latency values? Should they be the same value?

How to the control the movement of the gripper when the gripper is mounted to different robot arms ？

Hi, it is a wonderful work! One thing I want to know is how to control the movement of the gripper when the gripper is mounted to different robot arms. Is there any other control signal? When collecting data it was driving by hand, how about mounting to different robot arms?

Problem on 'cam_to_tip_offset'

Hello I have some problenms on the cam_to_tip_offset in https://github.com/real-stanford/universal_manipulation_interface/blob/main/scripts_slam_pipeline/06_generate_dataset_plan.py#L105,

As shown in the code, the cam_to_tip_offset = cam_to_mount_offset + tcp_offset , but in my opinon, the cam_to_tip_offset = tcp_offset - cam_to_mount_offset, shown in the image below

looking forward the response.

How can I deploy umi on other robots？ like jaka robot.

Streaming at 2.7K during inference

Dear authors,

I want to stream using the capture card+media mod at 2.7K (same as the training data), because I need to rectify the images using the provided config file.
But I can only get 1920x1080 images using the capture card at the moment. Could you help?

Thanks!

polymetis server issue

running the file scripts_real/launch_franka_interface_server.py on the NUC returns the following the error:

raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.CANCELLED
        details = "Robot context not valid when calling GetRobotClientMetadata!"
        debug_error_string = "{"created":"@1712241739.450975495","description":"Error received from peer ipv4:127.0.0.1:50051","file":"src/core/lib/surface/call.cc","file_line":952,"grpc_message":"Robot context not valid when calling GetRobotClientMetadata!","grpc_status":1}"

Grippers Supported

@cheng-chi will the WSG controller code support out of the box this schunk EGK gripper

What other grippers can be supported out of the box ( i.e robotiq?)

Error when generating the zarr.zip

Hello, thanks a lot for such great work !
I have run the command

python run_slam_pipeline.py example_demo_session

However it error when generating the dataset, the errors are shown below:

with av.open(vid_args[0][0]) as container:
IndexError: list index out of range

Any solution?

By the way, I find that the Python code style (both in UMI and Diffusion Policy) you write is so fancy and is hard for me to read (though I also always program by Python), I am curious about what class or book you have learned to write such brilliant code style?

Spacemouse deadzone values

I noticed that the spacemouse dead zone values are set to zeros only. Is this a design choice? since its very hard to have exactly zero values when the spacemouse is not moving!

Any method that I can visualize the ORB SLAM results?

This would be very helpful and more straightforward than pure CSV and other text file formats.

Calibrating GoPro with non standard lenses.

I am using a GoPro with non-standard fisheye lens. So now I need to calibrate the camera to get better SLAM performance.

• Which process do you follow to calibrate the camera using OpenICC?
• And how/where is that information used in the UMI pipeline?

Thanks!

About output

Hello,I want know what is the output after running the example. I found a folder called checkpoints with two zip packs, but I don't know how to use it.The input data is video, but what is the output data?Can you answer it for me？ Thank you

Hardware requirements

Hi there,

first of all thank you for sharing this amazing work!

I was wondering what is the minimum required setup in terms of GPUs/RAM to train the model?

I saw in your paper you guys are using 4/8 A100, is there any chance to train the model using consumer GPUs?

Thanks

Help with kinematic based data filtering

I'm trying to replicate this paper on my own hardware, which is a ViperX 300 6 DoF. I'm running into a problem where my policy is generating a lot of singularities, which I imagine is partly due to the ViperX having more of them than the robots you used in the paper.

However in the paper you mentioned that you used kinematics based data filtering to train the policy. I didn't see any place to incorporate the ViperX's kinematics into the code, so I don't think I did any filtering when I trained my policy. Could you give me any pointers on how you achieved this?

gopro camera QR code setting.

I conducted an experiment using your GoPro Hero 9 Black. I installed GoPro Labs and scanned the QR code mentioned in the git. However, I found that the captured videos cannot be parsed correctly, and an error message 'An error occurred Track not found' appeared. Eventually, I discovered that the captured videos did not contain a 'gpmd' track. Additionally, the GoPro set up with the QR code did not capture scenes of the 'fisheye' type.

GoPro Media Mod

Hi @cheng-chi,
Why do I need a GoPro Media Mod when deploying on the real robot? It seems that the GoPro in the video on your website does not have GoPro Media Mod installed.

More Detailed Documentation Coming Soon!

Hi!
I noticed in your README that you mentioned 'More Detailed Documentation Coming Soon!' Could you please provide an estimated timeframe for when this detailed documentation might be released? Thanks!

GoPro Hero10 replace GoPro Hero 9

Hi @cheng-chi ,

Thanks for sharing the amazing work!
Can we use GoPro Hero 10 replace GoPro Hero 9 since we can not order GoPro Hero 9?

Thanks

Train and deploy with different action frequency

Say I want to use 20Hz instead of 10Hz. What do I need to do for this?
I see the --frequency option in eval_real.py which is straightforward but what do I need to do for training?

Flange position and rotation settings

I'm wondering how to measure these values

Provide `tx_slam_tag` file for the in the wild cup experiment

Hi UMI authors,

In

universal_manipulation_interface/scripts_slam_pipeline/06_generate_dataset_plan.py

Line 110 in 298776c

# SLAM map origin to table tag transform

tx_slam_tag is used to convert camera in the SLAM frame to the aruco tag frame.

# SLAM map origin to table tag transform
  if tx_slam_tag is None:
      path = demos_dir.joinpath('mapping', 'tx_slam_tag.json')
      assert path.is_file()
      tx_slam_tag = str(path)
  tx_slam_tag = np.array(json.load(
      open(os.path.expanduser(tx_slam_tag), 'r')
      )['tx_slam_tag']
  )

Question: Can we have the tx_slam_tag.json file for data in https://real.stanford.edu/umi/data/zarr_datasets/cup_in_the_wild.zarr.zip?

It is easier to use data defined in the SLAM frame so that
(1) I can convert the data to other frames
(2) run models from codebase that uses a different frame.

Thank you!

questions of orbslam3 yaml file

It's really an amazing job, but after 3 weeks exploring I still not start this project correctlly.Some problems as follows:

1. why you choose gopro10 replace gopro9?
2. why put the whole slam project in docker? including yaml file?
3. is it possible to use my own yaml file from openicc? i did so,but I got a very low rate of success when I check camera trajectory from my own datasets.

thank you for your reply