Giter Site home page Giter Site logo

Comments (12)

SshiJwu avatar SshiJwu commented on August 14, 2024 2

Thanks,yuzhe!

from dex-retargeting.

yzqin avatar yzqin commented on August 14, 2024 1

@SshiJwu

Regarding your query about the out-of-bound value, it seems likely that the problem arises from the last_qpos input in self.optimize(objective_fn, last_qpos) being beyond the allowable range for that particular joint. This could result from an unsuccessful optimization process in the previous iteration, potentially due to alterations in the Jacobian calculation within the original codebase.

from dex-retargeting.

yzqin avatar yzqin commented on August 14, 2024 1

Consider this library as an opaque utility that carries out the necessary retargeting calculations for your application. Once you receive the retargeting data from this library, you can readily apply it to simulations or visualizations within IsaacSim. There's no need for you to delve into the computational details managed by SAPIEN.

Regarding the Jacobian, I'm not certain of the specific definition used in Isaac Sim. The function compute_single_link_local_jacobian calculates the spatial twist Jacobian for a given link as defined in the URDF. It produces a matrix with the dimensions 7*n, where n represents the degrees of freedom (DOF) of the robot.

from dex-retargeting.

SshiJwu avatar SshiJwu commented on August 14, 2024

And I always have this problem in the optimization process:
bounds 2 fail 1e-06 <= 0.785397 <= 0.785397
File "/home/wsj/.local/share/ov/pkg/retarget/retarget/retarget_optimizer.py", line 80, in optimize
qpos = self.opt.optimize(last_qpos)
File "/home/wsj/.local/share/ov/pkg/retarget/retarget/retarget_optimizer.py", line 177, in retarget
return self.optimize(objective_fn, last_qpos)
File "/home/wsj/.local/share/ov/pkg/retarget/retarget/retarget_optimizer.py", line 224, in retarget
qpos = self.optimizer.retarget(
File "/home/wsj/.local/share/ov/pkg/retarget/main.py", line 105, in main
qpos = retargeting.retarget(ref_value) # qpos 对应的kinematics_solver.get_joint_names的index
File "/home/wsj/.local/share/ov/pkg/retarget/main.py", line 116, in
main(dexycb_dir="/home/wsj/.local/share/ov/pkg/retarget/dexycb_datasets/datasets",
ValueError: bounds 2 fail 1e-06 <= 0.785397 <= 0.785397
It feels like the optimization is out of bounds.How do I avoid it well?

from dex-retargeting.

yzqin avatar yzqin commented on August 14, 2024

Hi @SshiJwu ,

I'm not deeply versed in IsaacSim, yet from my understanding and previous exposure to IsaacGym, I suspect that the discrepancy you're observing between SAPIEN and IsaacSim stems from their distinct interpretations of the Jacobian. In SAPIEN, the Jacobian is related to the spatial twist, whereas in Isaac, it pertains to cartesian velocity combined with angular velocity. Typically, when rotational dynamics are negligible, these Jacobsian representations might be seen as interchangeable. However, complications can arise due to the varying joint sequencing as interpreted by different URDF parsers. It's crucial to ensure that you maintain a consistent joint order across your various applications.

Moreover, I'm intrigued as to why there's a necessity to recode for IsaacSim. Should you be utilizing IsaacSim for simulations, it's feasible to apply our existing codebase without alterations, despite the embedded SAPIEN simulator. The retargeting code we've developed is capable of calculating the necessary joint angles, which you can then input directly into the IsaacSim environment. I'm of the opinion that redeveloping this codebase to employ an alternative Jacobian computation offers no clear advantage.

from dex-retargeting.

SshiJwu avatar SshiJwu commented on August 14, 2024

Yes. The order of the joints returned by the Isaac SIM is inconsistent with that of SAPIEN.
In Isaac Sim, it is ['WRJ1', 'FFJ4', 'LFJ5', 'MFJ4', 'RFJ4', 'THJ5', 'WRJ2', 'FFJ3', 'LFJ4', 'MFJ3', 'RFJ3', 'THJ4', 'FFJ2', 'LFJ3', 'MFJ2', 'RFJ2', 'THJ3', 'FFJ1', 'LFJ2', 'MFJ1', 'RFJ1', 'THJ2', 'LFJ1', 'THJ1'], but ['WRJ2', 'WRJ1', 'FFJ4', 'FFJ3', 'FFJ2', 'FFJ1', 'MFJ4', 'MFJ3', 'MFJ2', 'MFJ1', 'RFJ4', 'RFJ3', 'RFJ2', 'RFJ1', 'LFJ5','LFJ4','LFJ3','LFJ2','LFJ1','THJ5','THJ4','THJ3','THJ2','THJ1'] in SAPIEN.
How can I get the same results as SAPIEN in ISAAC SIM? Is there a solution available?
Can't I get the same Jacobi as SAPIEN in ISAAC SIM?
The api compute_single_link_local_jacobian in SAPIEN,What is returned? About position or orientation?

from dex-retargeting.

SshiJwu avatar SshiJwu commented on August 14, 2024

I also have a question about the left and right hands. What is the difference between the left and right dexterous hand URDF? Can I retarget the data from the left hand to the URDF of the right hand.

from dex-retargeting.

SshiJwu avatar SshiJwu commented on August 14, 2024

I've found that if I retarget the data from the left hand to URDF of the right hand, I get a big shift between the hand and the object.

2023-12-24 23-37-53屏幕截图

from dex-retargeting.

SshiJwu avatar SshiJwu commented on August 14, 2024

Why did I change the hand.type of DexYCB to left? How can the relative position of the hand and the object be so different after rendering?
2023-12-25 12-45-53屏幕截图

from dex-retargeting.

yzqin avatar yzqin commented on August 14, 2024

It does not make sense to retarget the motion from right hand to left hand due to the huge morphology gap. Always retargeting left human hand motion to a left hand robot and vice versa.

from dex-retargeting.

SshiJwu avatar SshiJwu commented on August 14, 2024

There is another question that has been bothering me for a long time, in order to allow the dexterous hand to move freely in space, you added six free joints. The translation_range is (-1,1),and the rotation_range is (-pi,pi). Why is the range of translation this? Since space is infinite, I can use my dexterous hands to reach anywhere in the space. Can I consider the range of rotation to be Euler angles?

from dex-retargeting.

yzqin avatar yzqin commented on August 14, 2024

You can tune this value based on your applications. For example, you can use a very large limit for translation to make the hand more freely inside the space. The (-1, 1) is an example of motion range, there are no specific meaning of these numbers,

from dex-retargeting.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.