Comments (7)
Hi, Yuzhe, thanks for your time and great explanation.
Yes, I see the weight variable in the Optimizer.py.
I still have a question about the rendering in Sapien.
Through visualize_hand_object.py, the retargeting results are shown in Sapien.
Sometimes, I find that the robot's fingertips overlap with the object, and sometimes, the object is not even touched by the human hand or the robot hand. However, the object is still moving stably with the hands.
In the robot hand and the object overlapping cases, how do you keep a stable grasp?
In the untouching cases, how do you prevent the object from falling?
Thanks a lot.
from dex-retargeting.
When performing offline retargeting with an object in hand, position-based retargeting is preferable to vector or DexPilot methods.
The visualize_hand_object.py script disables SAPIEN's physical simulation, functioning solely as a visualizer. Consequently, objects move regardless of contact accuracy.
Physically, directly replaying a retargeted trajectory doesn't produce a valid result. Instead, researchers often use retargeted results as references for motion imitation or as training data for reinforcement learning combined with imitation learning.
from dex-retargeting.
hi @Smilels
The DexPilotOptimizer is a variant of the VectorOptimizer, inspired by the DexPilot paper. It incorporates a fingertip prior to the original vector retargeting objective. For grasping tasks, the DexPilotOptimizer is recommended. For other tasks, the VectorOptimizer remains a suitable choice.
By the way, thank you for your excellent work on TeachNet, which has been a valuable inspiration for our teleoperation research.
from dex-retargeting.
hi there, thanks for your explanation.
With your code, I don't experience an obvious frequency difference between position-based retargeting and vector-based retargeting. Why do you suggest using vector or DexPilot to perform real-time retargeting?
from dex-retargeting.
Hi @Smilels
Great question I refer vector retargeting as real-time retargeting not because it is faster. Compared with position retargeting, which target at handling offline hand-object dataset, vector retargeting is used for online tasks like teleoperation.
The difference between them are not running speed, but target applications.
from dex-retargeting.
The objective of position retargeting and vector retargeting is different. Vector retargeting try to be more intuitive for close-loop control while position retargeting try to keep the finger tip position. It is better to use different retargeting optimizer for different applications.
from dex-retargeting.
thanks a lot. That's clear.
Thanks again for your sharing
from dex-retargeting.
Related Issues (20)
- How To do position retarget like AnyTeleop HOT 6
- hand detection retarget not accurate HOT 6
- Issue running "pip3 install -e ." HOT 1
- Import the new manipulator's URDF HOT 4
- How To do position retarget in real world? HOT 19
- Expand the retargeting algorithm to 2 arm-hand system HOT 1
- teleop on physic simulator to collect data HOT 1
- Error when using isaacgym HOT 4
- AttributeError
- Curobo implementation
- retargetting problem with laptop's built-in camera HOT 7
- some mistake when I install dex-retargeting HOT 1
- Basic Mathmatics about calculating the joint_pos variable in SingleHandDetector.py HOT 2
- install error HOT 3
- How can I retargeting with my hand keypoints as input? HOT 1
- Deployment on physical inspire dexterous hand HOT 1
- Recommended Python Version HOT 2
- position retargeting demo fails when only `*_glb.urdf` is available HOT 3
- using `warm_start` with position optimizer HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dex-retargeting.