Comments (12)
Thanks,yuzhe!
from dex-retargeting.
Regarding your query about the out-of-bound value, it seems likely that the problem arises from the last_qpos
input in self.optimize(objective_fn, last_qpos)
being beyond the allowable range for that particular joint. This could result from an unsuccessful optimization process in the previous iteration, potentially due to alterations in the Jacobian calculation within the original codebase.
from dex-retargeting.
Consider this library as an opaque utility that carries out the necessary retargeting calculations for your application. Once you receive the retargeting data from this library, you can readily apply it to simulations or visualizations within IsaacSim. There's no need for you to delve into the computational details managed by SAPIEN.
Regarding the Jacobian, I'm not certain of the specific definition used in Isaac Sim. The function compute_single_link_local_jacobian
calculates the spatial twist Jacobian for a given link as defined in the URDF. It produces a matrix with the dimensions 7*n
, where n
represents the degrees of freedom (DOF) of the robot.
from dex-retargeting.
And I always have this problem in the optimization process:
bounds 2 fail 1e-06 <= 0.785397 <= 0.785397
File "/home/wsj/.local/share/ov/pkg/retarget/retarget/retarget_optimizer.py", line 80, in optimize
qpos = self.opt.optimize(last_qpos)
File "/home/wsj/.local/share/ov/pkg/retarget/retarget/retarget_optimizer.py", line 177, in retarget
return self.optimize(objective_fn, last_qpos)
File "/home/wsj/.local/share/ov/pkg/retarget/retarget/retarget_optimizer.py", line 224, in retarget
qpos = self.optimizer.retarget(
File "/home/wsj/.local/share/ov/pkg/retarget/main.py", line 105, in main
qpos = retargeting.retarget(ref_value) # qpos 对应的kinematics_solver.get_joint_names的index
File "/home/wsj/.local/share/ov/pkg/retarget/main.py", line 116, in
main(dexycb_dir="/home/wsj/.local/share/ov/pkg/retarget/dexycb_datasets/datasets",
ValueError: bounds 2 fail 1e-06 <= 0.785397 <= 0.785397
It feels like the optimization is out of bounds.How do I avoid it well?
from dex-retargeting.
Hi @SshiJwu ,
I'm not deeply versed in IsaacSim, yet from my understanding and previous exposure to IsaacGym, I suspect that the discrepancy you're observing between SAPIEN and IsaacSim stems from their distinct interpretations of the Jacobian. In SAPIEN, the Jacobian is related to the spatial twist, whereas in Isaac, it pertains to cartesian velocity combined with angular velocity. Typically, when rotational dynamics are negligible, these Jacobsian representations might be seen as interchangeable. However, complications can arise due to the varying joint sequencing as interpreted by different URDF parsers. It's crucial to ensure that you maintain a consistent joint order across your various applications.
Moreover, I'm intrigued as to why there's a necessity to recode for IsaacSim. Should you be utilizing IsaacSim for simulations, it's feasible to apply our existing codebase without alterations, despite the embedded SAPIEN simulator. The retargeting code we've developed is capable of calculating the necessary joint angles, which you can then input directly into the IsaacSim environment. I'm of the opinion that redeveloping this codebase to employ an alternative Jacobian computation offers no clear advantage.
from dex-retargeting.
Yes. The order of the joints returned by the Isaac SIM is inconsistent with that of SAPIEN.
In Isaac Sim, it is ['WRJ1', 'FFJ4', 'LFJ5', 'MFJ4', 'RFJ4', 'THJ5', 'WRJ2', 'FFJ3', 'LFJ4', 'MFJ3', 'RFJ3', 'THJ4', 'FFJ2', 'LFJ3', 'MFJ2', 'RFJ2', 'THJ3', 'FFJ1', 'LFJ2', 'MFJ1', 'RFJ1', 'THJ2', 'LFJ1', 'THJ1'], but ['WRJ2', 'WRJ1', 'FFJ4', 'FFJ3', 'FFJ2', 'FFJ1', 'MFJ4', 'MFJ3', 'MFJ2', 'MFJ1', 'RFJ4', 'RFJ3', 'RFJ2', 'RFJ1', 'LFJ5','LFJ4','LFJ3','LFJ2','LFJ1','THJ5','THJ4','THJ3','THJ2','THJ1'] in SAPIEN.
How can I get the same results as SAPIEN in ISAAC SIM? Is there a solution available?
Can't I get the same Jacobi as SAPIEN in ISAAC SIM?
The api compute_single_link_local_jacobian in SAPIEN,What is returned? About position or orientation?
from dex-retargeting.
I also have a question about the left and right hands. What is the difference between the left and right dexterous hand URDF? Can I retarget the data from the left hand to the URDF of the right hand.
from dex-retargeting.
I've found that if I retarget the data from the left hand to URDF of the right hand, I get a big shift between the hand and the object.
from dex-retargeting.
Why did I change the hand.type of DexYCB to left? How can the relative position of the hand and the object be so different after rendering?
from dex-retargeting.
It does not make sense to retarget the motion from right hand to left hand due to the huge morphology gap. Always retargeting left human hand motion to a left hand robot and vice versa.
from dex-retargeting.
There is another question that has been bothering me for a long time, in order to allow the dexterous hand to move freely in space, you added six free joints. The translation_range is (-1,1),and the rotation_range is (-pi,pi). Why is the range of translation this? Since space is infinite, I can use my dexterous hands to reach anywhere in the space. Can I consider the range of rotation to be Euler angles?
from dex-retargeting.
You can tune this value based on your applications. For example, you can use a very large limit for translation to make the hand more freely inside the space. The (-1, 1)
is an example of motion range, there are no specific meaning of these numbers,
from dex-retargeting.
Related Issues (20)
- Faild to load Leap Hand model HOT 6
- Retargeting of example allegro inconsistent with video in project website? HOT 1
- How to relax restriction on collision between links? HOT 5
- Plan to add dummy joints under Pin Model? HOT 1
- error during visualizing robot hand-object Interaction HOT 1
- question about mano conver HOT 2
- How To do position retarget like AnyTeleop HOT 6
- hand detection retarget not accurate HOT 6
- Issue running "pip3 install -e ." HOT 1
- Import the new manipulator's URDF HOT 4
- How To do position retarget in real world? HOT 20
- Expand the retargeting algorithm to 2 arm-hand system HOT 1
- teleop on physic simulator to collect data HOT 1
- Error when using isaacgym HOT 4
- AttributeError
- Curobo implementation
- Difference bettwenn VectorOptimizer and DexPilotOptimizer HOT 7
- retargetting problem with laptop's built-in camera HOT 4
- some mistake when I install dex-retargeting HOT 1
- Basic Mathmatics about calculating the joint_pos variable in SingleHandDetector.py HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dex-retargeting.