Giter Site home page Giter Site logo

disect's Introduction

DiSECt: Differentiable Simulator for Robotic Cutting

Potato slicing

DiSECt is a simulator for the cutting of deformable materials. It uses the Finite Element Method (FEM) to simulate the deformation of the material, and leverages a virtual node algorithm to introduce springs between the two halves of the mesh being cut. These cutting springs are weakened in proportion to the knife forces acting on the material, yielding a continuous model of deformation and crack propagation. By leveraging source code transformation, the back-end of DiSECt automatically generates CUDA-accelerated kernels for the forward simulation and the gradients of the simulation inputs. Such gradient information can be used to optimize the simulation parameters to achieve accurate knife force predictions, optimize cutting actions, and more.

Prerequisites

  • Python 3.6 or higher
  • PyTorch 1.4.0 or higher
  • Pixar USD lib (optional, for advanced visualization)

Pre-built USD Python libraries can be downloaded from https://developer.nvidia.com/usd, once they are downloaded you should follow the corresponding instructions to add them to your PYTHONPATH environment variable. Besides using the provided basic visualizer implemented using pyvista, DiSECt can generate USD files for rendering, e.g., in NVIDIA Omniverse™ or usdview.

Using the built-in backend

By default, the simulation back-end uses the built-in PyTorch cpp-extensions mechanism to compile auto-generated simulation kernels.

  • Windows users should ensure they have Visual Studio 2019 installed

Installation

Dataset

To set up our dataset of meshes, simulated knife forces and nodal motion fields we recorded in the ANSYS LS-DYNA simulator, download this zip file (96 MB) and extract it in the project folder, such that the folder dataset is at the top level.

We provide a README.md file with more details on the contents of this dataset in the dataset folder. The dataset is released under the Creative Commons Attribution-NonCommercial 4.0 International License.

Python dependencies

Next, set up the Python dependencies listed in requirements.txt via

pip install -r requirements.txt

Mesh processing library

See meshing/README.md for instructions on how to install the recommended C++-based mesh cutting library that DiSECt relies on to process meshes.

Mesh discretization

For the mesh discretization we provide an example script in cutting/tetrahedralization.py based on the Wildmeshing Python API that can be used to generate a tetrahedral mesh from a triangle surface mesh, which allows it to be used in the FEM simulator.

Examples

The following demos are provided and can be executed via python examples/<example_name>.py.

Example Description
basic_cutting Cutting a prism shape with a knife following a slicing motion, running in the interactive pyvista 3D visualizer
render_usd Demonstrates how to generate a USD file from the simulation
optimize_slicing Constrained optimization via MDMM to find a slicing motion of the knife that minimizes force while adhering to blade length and knife height constraints
parameter_inference Optimizes simulation parameters to match a knife force profile from one of the measurements in our dataset

Citation

@INPROCEEDINGS{heiden2021disect,
    AUTHOR    = {Eric Heiden AND Miles Macklin AND Yashraj S Narang AND Dieter Fox AND Animesh Garg AND Fabio Ramos},
    TITLE     = {{DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting}},
    BOOKTITLE = {Proceedings of Robotics: Science and Systems},
    YEAR      = {2021},
    ADDRESS   = {Virtual},
    MONTH     = {July},
    DOI       = {10.15607/RSS.2021.XVII.067}
}

Contributions

We do not accept outside code contributions in the form of pull requests, but you are welcome to open issues which we may address in future updates to this work.

License

Copyright © 2021, NVIDIA Corporation. All rights reserved.

This work is made available under the NVIDIA Source Code License.

disect's People

Contributors

eric-heiden avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

disect's Issues

Error on `examples/render_usd.py` with leaf `Variable` and gradients

I have installed DiSECt on my system:

  • Ubuntu 18.04
  • NVIDIA GeForce RTX 3090 GPU
  • Conda environment with Python 3.7 (find the conda list here)
  • nvcc --version gives me CUDA 11.2.
  • The dataset/ folder is located in the home directory.

Three of the example scripts seem to be running without errors or warnings.

python examples/basic_cutting.py
python examples/optimize_slicing.py
python examples/parameter_inference.py

The exception is this fourth example:

(disect) seita@takeshi:~/DiSECt (main) $ python examples/render_usd.py 
Using cached kernels
Using log folder at "/home/seita/DiSECt/log".
Converted Young's modulus 43000.0 and Poisson's ratio 0.49 to Lame parameters mu = 14429.530201342282 and lambda = 707046.9798657711
PyANSYS MAPDL Result file object
Title       : Cutting_v5--Static Structural (B5)
Units       : User Defined
Version     : 20.2
Cyclic      : False
Result Sets : 1
Nodes       : 797
Elements    : 3562


Available Results:
ENS : Nodal stresses
ENG : Element energies and volume
EEL : Nodal elastic strains
EUL : Element euler angles
EPT : Nodal temperatures
NSL : Nodal displacements
RF  : Nodal reaction forces

ANSYS Mesh
  Number of Nodes:              797
  Number of Elements:           3562
  Number of Element Types:      2
  Number of Node Components:    1
  Number of Element Components: 0

Loaded mesh with 797 vertices and 3472 tets.
Creating free-floating knife
cut_meshing_cpp took 2.58 ms
224 cut springs have been inserted.
/home/seita/DiSECt/dflex/model.py:2223: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  /opt/conda/conda-bld/pytorch_1634272178570/work/torch/csrc/utils/tensor_new.cpp:201.)
  m.shape_transform = torch.tensor(transform_flatten_list(self.shape_transform), dtype=torch.float32, device=adapter)
self.cut_edge_indices: (448, 2)
self.cut_spring_indices: (224, 2)
self.cut_virtual_tri_indices: (790, 3)
self.cut_edge_indices: (448, 2)
self.cut_spring_indices: (224, 2)
self.cut_virtual_tri_indices: (790, 3)
render_demo:   0%|                                                                             | 0/40 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "examples/render_usd.py", line 54, in <module>
    sim.simulate(render=True)
  File "/home/seita/DiSECt/cutting/cutting_sim.py", line 708, in simulate
    self.simulation_step()
  File "/home/seita/DiSECt/cutting/cutting_sim.py", line 650, in simulation_step
    update_mass_matrix=False)
  File "/home/seita/DiSECt/dflex/sim.py", line 2912, in forward
    state_in.joint_qdd.zero_()
RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.
(disect) seita@takeshi:~/DiSECt (main) $ 

This seems to be a PyTorch error (e.g., https://discuss.pytorch.org/t/leaf-variable-was-used-in-an-inplace-operation/308) but are we supposed to have other information stored or loaded?

Error when reproduce demo on DiSECt

Hi, Heiden
Thank you for sharing such awesome work of DiSECt, where the codebase is solid and cool.
But I meet some bugs when try to python examples/basic_cutting.py

The following is my bug info. It seems that cannot find kernels.so
` python examples/basic_cutting.py
Rebuilding kernels
/home/anabur/anaconda3/envs/robot/lib/python3.6/site-packages/torch/utils/cpp_extension.py:298: UserWarning:

                           !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (clang++-9) is not compatible with the compiler Pytorch was
built with for this platform, which is g++ on linux. Please
use g++ to to compile your extension. Alternatively, you may
compile PyTorch from source using clang++-9, and then you can also use
clang++-9 to compile your extension.

See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
with compiling PyTorch from source.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                          !! WARNING !!

platform=sys.platform))
Detected CUDA files, patching ldflags
Emitting ninja build file /media/anabur/E/robot_similation/DiSECt/dflex/kernels/build.ninja...
Building extension module kernels...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
1.10.2.git.kitware.jobserver-1
Loading extension module kernels...
Traceback (most recent call last):
File "examples/basic_cutting.py", line 20, in
from cutting import load_settings, SlicingMotion, CuttingSim
File "/media/anabur/E/robot_similation/DiSECt/cutting/init.py", line 10, in
from .cutting_sim import *
File "/media/anabur/E/robot_similation/DiSECt/cutting/cutting_sim.py", line 25, in
from cutting.urdf_loader import load_urdf
File "/media/anabur/E/robot_similation/DiSECt/cutting/urdf_loader.py", line 12, in
import dflex as df
File "/media/anabur/E/robot_similation/DiSECt/dflex/init.py", line 15, in
kernel_init()
File "/media/anabur/E/robot_similation/DiSECt/dflex/sim.py", line 47, in kernel_init
kernels = df.compile()
File "/media/anabur/E/robot_similation/DiSECt/dflex/adjoint.py", line 1934, in compile
with_pytorch_error_handling=False)
File "/home/anabur/anaconda3/envs/robot/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1285, in load_inline
keep_intermediates=keep_intermediates)
File "/home/anabur/anaconda3/envs/robot/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1362, in _jit_compile
return _import_module_from_library(name, build_directory, is_python_module)
File "/home/anabur/anaconda3/envs/robot/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1752, in _import_module_from_library
module = importlib.util.module_from_spec(spec)
ImportError: /media/anabur/E/robot_similation/DiSECt/dflex/kernels/kernels.so: cannot open shared object file: No such file or directory
`

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.