v-wewei/dexterity

Software and tasks for dexterous multi-fingered hand manipulation, powered by MuJoCo.

The MuJoCo Dexterity Suite (alpha-release)

Software and tasks for dexterous multi-fingered hand manipulation, powered by MuJoCo.

dexterity builds on dm_control and provides a collection of modular components that can be used to define rich Reinforcement Learning environments for dexterous manipulation. It also comes with a set of standardized tasks that can serve as a performance benchmark for the research community.

An introductory tutorial is available as a Colab notebook:

Installation

PyPI (Recommended)

The recommended way to install this package is via PyPI:

pip install dexterity

Source

We provide a Miniconda environment with Python 3.8 for development. To create it and install dependencies, run the following steps:

git clone https://github.com/kevinzakka/dexterity
cd dexterity
conda env create -f environment.yml  # Creates a dexterity env.
conda activate dexterity
pip install .

Overview

The MuJoCo dexterity suite is composed of the following core components:

• models: MuJoCo models for dexterous hands and PyMJCF classes for dynamically customizing them.
• inverse_kinematics: Inverse kinematics library for multi-fingered hands.
• effectors: Interfaces for controlling hands and defining action spaces.

These components, in conjunction with dm_control, allow you to define and customize rich environments for reinforcement learning. We facilitate this process by providing the following:

• task: Wrappers over composer.Task that simplify the creation of generic dexterous tasks as well as goal-reaching based tasks (e.g., successive object reorientation).
• manipulation: A library of pre-defined, benchmark RL environments geared towards dexterous manipulation. For an overview of the available tasks, see the task library.

Our hope is to grow the benchmark over time with crowd-sourced contributions from the research community -- PR contributions are welcome!

Acknowledgements

A large part of the design and implementation of dexterity is inspired by the MoMa library in dm_robotics.