Issue with running example file.

[Huangruopeng]

Issue with running example file.

[JenniferBuehler]

I can confirm that I am experiencing the same. I'm on Ubuntu 22.04, python 3.10.12, and downloaded Isaac 2023.1.1 just yesterday from Omniverse. I get the same for 2023.0.1-hotfix as well.

For examples/isaac_sim/motion_gen_reacher.py I get what you do:

Traceback (most recent call last):
  File "/home/jenny/omni/curobo/examples/isaac_sim/motion_gen_reacher.py", line 437, in <module>
    main()
  File "/home/jenny/omni/curobo/examples/isaac_sim/motion_gen_reacher.py", line 213, in main
    motion_gen_config = MotionGenConfig.load_from_robot_config(
  File "/home/jenny/omni/curobo/src/curobo/wrap/reacher/motion_gen.py", line 642, in load_from_robot_config
    ik_solver_cfg = IKSolverConfig.load_from_robot_config(
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/kit/python/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/jenny/omni/curobo/src/curobo/wrap/reacher/ik_solver.py", line 316, in load_from_robot_config
    arm_rollout_mppi = ArmReacher(cfg)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/kit/python/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/jenny/omni/curobo/src/curobo/rollout/arm_reacher.py", line 167, in __init__
    ArmBase.__init__(self)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/kit/python/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/jenny/omni/curobo/src/curobo/rollout/arm_base.py", line 233, in __init__
    self._init_after_config_load()
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/kit/python/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/jenny/omni/curobo/src/curobo/rollout/arm_base.py", line 324, in _init_after_config_load
    self.null_convergence = DistCost(self.convergence_cfg.null_space_cfg)
  File "/home/jenny/omni/curobo/src/curobo/rollout/cost/dist_cost.py", line 343, in __init__
    self._l2_dof_kernel = make_l2_kernel(self.dof)
  File "/home/jenny/omni/curobo/src/curobo/rollout/cost/dist_cost.py", line 227, in make_l2_kernel
    return wp.Kernel(forward_l2_loop_warp)
TypeError: Kernel.__init__() missing 2 required positional arguments: 'key' and 'module'

And for collision_checker.py it's actually a different one (right after hitting play):

  File "/home/jenny/omni/curobo/examples/isaac_sim/collision_checker.py", line 205, in <module>
    main()
  File "/home/jenny/omni/curobo/examples/isaac_sim/collision_checker.py", line 179, in main
    d, d_vec = model.get_collision_vector(x_sph)
  File "/home/jenny/omni/curobo/src/curobo/wrap/model/robot_world.py", line 244, in get_collision_vector
    d = self.collision_cost.forward(x_sph, env_query_idx=env_query_idx)
  File "/home/jenny/omni/curobo/src/curobo/rollout/cost/primitive_collision_cost.py", line 179, in discrete_fn
    dist = self.coll_check_fn(
  File "/home/jenny/omni/curobo/src/curobo/geom/sdf/world_mesh.py", line 435, in get_sphere_distance
    d = self._get_sdf(
  File "/home/jenny/omni/curobo/src/curobo/geom/sdf/world_mesh.py", line 361, in _get_sdf
    d = SdfMeshWarpPy.apply(
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.pip.torch-2_0_1-2.0.2+105.1.lx64/torch-2-0-1/torch/autograd/function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/jenny/omni/curobo/src/curobo/geom/sdf/warp_primitives.py", line 538, in forward
    wp.launch(
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/context.py", line 3358, in launch
    if not module.load(device):
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/context.py", line 1388, in load
    builder = ModuleBuilder(self, self.options)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/context.py", line 1041, in __init__
    self.build_function(f)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/context.py", line 1087, in build_function
    func.adj.build(self)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 551, in build
    raise e
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 542, in build
    adj.eval(adj.tree.body[0])
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 1766, in eval
    return emit_node(adj, node)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 1054, in emit_FunctionDef
    adj.eval(f)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 1766, in eval
    return emit_node(adj, node)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 1633, in emit_Assign
    rhs = adj.eval(node.value)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 1766, in eval
    return emit_node(adj, node)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 1479, in emit_Call
    out = adj.add_call(func=func, args=args, kwds=kwds, templates=templates, min_outputs=min_outputs)
  File "/home/jenny/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/codegen.py", line 779, in add_call
    raise Exception(
Exception: Error while parsing function "mesh_query_point_fn" at /home/jenny/omni/curobo/src/curobo/geom/sdf/warp_primitives.py:25:
    collide_result = wp.mesh_query_point(idx, point, max_distance)
Couldn't find function overload for 'mesh_query_point' that matched inputs with types: [uint64, vec3f, float32]

One thing to note is that I could not have installed cuRobo without installing usd-core first, as it was not installed in the isaac python environment. I got an error message with the installation omni_python -m pip install -e .[isaacsim] --no-build-isolation that it was a missing dependency, and then installed it, but I had to take a wild guess at the version number, which had to be < 24.00 so I picked the release before that:

omni-python pip install usd-core==23.11

Did you have to do that too?
If I would guess (I'm new to this code base so it's just a guess really), that sounds like it could potentially relate to the second error, at least it's calling a file that contains the word core ;)

I have unfortunately only very limited time this week and I will probably have to move past the examples, but if I hit the same issues in my other use case, I will report here that it is an issue beyond just the examples (and the fix if I find it).

[JenniferBuehler]

Confirm that this is not limited to the examples, at least the error you also reported.

It happens in load_from_rorot_config but ONLY if the SimulationApp instance has previously been created. Very simple code to reproduce:

# isaac
# -----------

# >>>>>>>>>> Set this to True to get the exception!!!
load_sim = False

if load_sim:
    from omni.isaac.kit import SimulationApp
    headless = True
    simulation_app = SimulationApp(
        {
            "headless": headless,
            "width": "1920",
            "height": "1080",
        }
    )
        
# cuRobo
# -----------
from curobo.wrap.reacher.motion_gen import MotionGenConfig
from curobo.util_file import load_yaml

# in this simple example, I just have a config in the current working directory
robot_cfg = load_yaml("<path-to-any-yaml>")["robot_cfg"]
robot_cfg["kinematics"]["external_asset_path"] = "." 
            
motion_gen_config = MotionGenConfig.load_from_robot_config(
    robot_cfg
)   

Run:

omni-python <file-with-above-code>.py

[JenniferBuehler]

I had a closer look and NVidia warp is installed twice, and the Kernel class is different in these two.

HOME/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/context.py
HOME/.local/share/ov/pkg/isaac-sim-2023.1.1/kit/python/lib/python3.10/site-packages/warp/context.py

It's the one in extscache that gets used, as it expects key and module arguments. That's the file installed by IsaacSim:

# caches source and compiled entry points for a kernel (will be populated after module loads)
class Kernel:
    def __init__(self, func, key, module, options=None, code_transformers=[]):
        self.func = func

And the one that would be expected by curobo is the other one:

# caches source and compiled entry points for a kernel (will be populated after module loads)
class Kernel:
    def __init__(self, func, key=None, module=None, options=None, code_transformers=None):
        self.func = func

The first one, which causes the issue, seems odd to me in general (but I'm also absolutely new to this codebase). Why? In the original repo for release 1.0.0 beta, the parameters have default values too. Therefore, to test what's up with this, I just removed the whole package (I know, it might fly in my face at another occasion):

mv HOME/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp* ~/isaacbackups

I kept a backup of course, just in case I need to move it back. And: no more exception. It also solves the other exception in collision_checker.py that I reported above.

The motion_gen_reacher.py still doesn't load the robot visuals, but that's an entirely different problem. When I move the cube I can see in the XForms that the arm technically is following the cube, so the curobo part seems to be working. The collision checker works just as expected.

I will report this in the isaac forum as well, maybe someone there knows what's up with this warp beta version. Maybe it's a fork from the original which was modified.

P.S. the duplicated warp package has nothing to do with the installation of usd-core as mentioned above (confirm with omni-python -m pip show usd-core -f)

[chiawenwang]

I had a closer look and NVidia warp is installed twice, and the Kernel class is different in these two.

HOME/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp.core-1.0.0-beta.2/warp/context.py
HOME/.local/share/ov/pkg/isaac-sim-2023.1.1/kit/python/lib/python3.10/site-packages/warp/context.py

It's the one in extscache that gets used, as it expects key and module arguments. That's the file installed by IsaacSim:

# caches source and compiled entry points for a kernel (will be populated after module loads)
class Kernel:
    def __init__(self, func, key, module, options=None, code_transformers=[]):
        self.func = func

And the one that would be expected by curobo is the other one:

# caches source and compiled entry points for a kernel (will be populated after module loads)
class Kernel:
    def __init__(self, func, key=None, module=None, options=None, code_transformers=None):
        self.func = func

The first one, which causes the issue, seems odd to me in general (but I'm also absolutely new to this codebase). Why? In the original repo for release 1.0.0 beta, the parameters have default values too. Therefore, to test what's up with this, I just removed the whole package (I know, it might fly in my face at another occasion):

mv HOME/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp* ~/isaacbackups

I kept a backup of course, just in case I need to move it back. And: no more exception. It also solves the other exception in collision_checker.py that I reported above.

The motion_gen_reacher.py still doesn't load the robot visuals, but that's an entirely different problem. When I move the cube I can see in the XForms that the arm technically is following the cube, so the curobo part seems to be working. The collision checker works just as expected.

I will report this in the isaac forum as well, maybe someone there knows what's up with this warp beta version. Maybe it's a fork from the original which was modified.

P.S. the duplicated warp package has nothing to do with the installation of usd-core as mentioned above (confirm with omni-python -m pip show usd-core -f)

I just follow your advice by：
mv HOME/.local/share/ov/pkg/isaac-sim-2023.1.1/extscache/omni.warp* ~/isaacbackups

But it still doesn't work.

I think this problem is link to the version of Isaac sim? and my version is 2023.1.1 in Ubuntu20.04. I also try to add --robot ur5e.yml. but not work too.

do you have more process on this issue?

[JenniferBuehler]

Silly question, but I guess you added the $ to the HOME env that I forgot to add in the text above ;)

Can you do a sudo updatedb && locate context.py | grep warp and see if there are several context.py files (which is where the Kernel class lives)?

Make sure that you use the python.sh from your isaac package as well, which sets the python envorinment variables. Having several isaac versions is not a problem then, I have several installed.

[balakumar-s]

We fixed this issue in the latest main commit. cuRobo now also works with older warp versions. Please re-open if it still doesn't work for you.

[JenniferBuehler]

Great news, thank you!