lge-ros2/cloisim

Unity3D based multi-robot simulator can bring-up simulation environment by SDF(Simulation Description File) easily. And it supports to connect ROS2 through cloisim_ros.

CLOiSim : Multi-Robot Simulator

Happy to announce CLOiSim. It is a new multi-robot simulator that uses an SDF file containing 3d world environemnts and robot descriptions.

The simulator is based on Unity 3D. It may look similar to Gazebo, where, unfortunately, we encountered performance problems while loading multiple robots equipped with multiple sensors. Hence, CLOiSim.

This project consists of

• SDF Parser for C#
• SDF Robot Implementation for Unity -> Visual / Collision / Sensor / Physics for joints
• SDF Plugins for Unity3D
• UI modules -> Module for on-screen information
• Network modules -> Module for transporting sensor data or control signal
• Web service -> Module for controling simulation through a web interface

video link

Notices

Working branch policy has been updated. Previous latest main branch which is corresponding to release-3.2.0 branch is no longer available. So please check release-3.2.0 branch if you want to follow the history of CLOiSim-3.x.x. develop branch shall be merged into main branch every released.

Features

Sensors

The current release includes the features only for marked items in the list below. Other sensor models are work in progress. Here are the list of items that is implemented(marked) or planned to be implemented.

• Joint models
  • 2-Wheeled Motor driving
  • Joint control
  • Joint Pose
• Sensor models
  • LiDAR Sensor
    • 2D
    • 3D
  • Sonar sensor
  • IMU
  • Contact
  • Camera
    • Camera intrinsic parameter
    • Depth Camera
    • Multi-camera
    • RealSense (RGB + IR1 + IR2 + Depth)
    • Semantic Segmentation Camera
  • GPS sensor
  • Sensor noise models
    • Gaussian
      • GPS, IMU
      • Lidar
      • Camera
    • Custom
• Physics
  • Support all physics parameters in SDF specification
  • Support <Joint type="revolute2">
• Worlds
  • Actors
    • interpolate_x in <animation>
  • Lights
    • supporting <specular>, <attenuation/linear>, <attenuation/contant>, <attenuation/quadratic>, <spot/falloff>
  • Spherical Coordinates
  • Heightmap (DEM)
    • Support GeoTIFF
    • Texture Blend
  • Road

Plus, SDF file basically targeting and supporting version 1.6 and works on the essential elements such as <model>, <link>, <visual>, <collision>, <joint>, etc. It does not support optional elmenets like <wind>, <audio>, <state>, <atmosphere>, <magnetic_field>, <scene>, <road>, <population>.

There is problem with <pose> in <joint> since introduction of articulation body model. Therefore, plaese

Currently, geometry mesh type is supporting only 'Wavefront(.obj) with material', 'Collada(.dae) including animation' and 'STL(.stl)'. <ambient> elements in <materal> and ambient properies in mesh files are not support in CLOiSim.

If you're trying to connect <link> of <model>, it needs to specify unique name in link name in joint/parent or joint/child within same root model.

video link

Sensor Plugins

It called 'CLOiSimPlugin'. And below plugins could be utilized through write an element on SDF.

Plugin name should be written in filename attribute and it's case sensitive.

For example,

<plugin name="actor_plugin" filename="libActorPlugin.so" />

More details and usages for configuration/parameters in here.

Model Specific

• LaserPlugin: help to publish 2D or 3D lidar data
• CameraPlugin: help to publish 2D color image data or depth image data
• SegmentationCameraPlugin: help to publish semantic segmentation image data and label info
• MultiCameraPlugin: help to publish multiple color image data
• RealSensePlugin: can handle ir1(left), ir2(right), depth, color
• MicomPlugin: control micom(differential drive) input/output(sensor)
• GpsPlugin: gps position in world
• JointControlPlugin: can control joints and help to publish joints status.
• ActorPlugin: add actor control functionality using AI(Unity) components
• ImuPlugin: help to publish IMU sensor data
• SonarPlugin: help to publish Sonar range data
• ParticleSystemPlugin: Enable particla system of Unity

World Specific

• ElevatorSystemPlugin: control(lifting, cal) elevators
• GroundTruthPlugin: retrieve all information(position, size, velocity) for objects
• ActorControlPlugin: controls actor using AI(Unity) components(actor which loaded ActorPlugin)
• MowingPlugin: plant grass and enable to mow

How it works

Refer to core codes in 'Assets/Scripts'.

• Load SDF file -> Parse SDF(simulation description) -> Implement and realize description

Shaders are also used to get depth buffer information in a few sensor model.

Default physics engine 'Nvidia PhysX' is used for physics. And it retrieves some of physics parameters from <ode> in sdf. 'SDFPlugins' help physics tricky handling for jointing <link> ojbects by <joint> element.

We've deceided to change a solver type of physics engine since new solver "TGS(Temporal Gauss Seidel)" is intorduced recently(PhysX 4.1).

So there is NO more constaints for rigidbodies by PGS(Projected Gauss Seidel) solver type since latest version(CLOiSim-1.11.0).

For the performance in terms of collision handling, designing collision geometry properly may important.

an aspect of rendering

if <name> element of <script> element in <material> element contains "tree" words, CLOiSim applies "URP/Nature/SpeedTree" Shader as a rendering material.

Getting Started

Minimum requirement

• Processor: testing and looking for the minimum
• Memory: testing and looking for the minimum
• Graphics: testing and looking for the minimum

Tested environement (latest)

• Unity Editor Version: '2022.3.40f1 (LTS)'.

• Linux Machine

  • OS: Ubuntu 22.04.4 LTS
  • Processor: AMD® Ryzen 9 5950x 16-core processor × 32
  • Memory: 128.0 GiB
  • Graphics: NVIDIA Corporation [GeForce RTX 3090]

• Windows Machine

  • OS: Windows 10 20H2
  • Processor: AMD® Ryzen 9 5900HS 8-core processor x 16
  • Memory: 32GB
  • Graphics: NVIDIA GeForce RTX3060 Laptop GPU

Release version

If you don't want to build a project, just USE a release binary(Download linux version). And just refer to 'Usage' section.

In terms of branch, 'main' is release(stable) version, and 'develop' is used for development(on-going).

If you want to build a project

Please visit here build guide in Wiki pages.

Usage

Any problem during launching the simultion, please refer to Wiki Page/Usage' section.

Run 'CLOiSim'

Set environment path like below. You can find the sample resources here

Multiple path can be set by :(colon).

export CLOISIM_FILES_PATH="/home/Unity/cloisim/sample_resources/media"
export CLOISIM_MODEL_PATH="/home/Unity/cloisim/sample_resources/models:/home/Unity/cloisim/another_resources/models"
export CLOISIM_WORLD_PATH="/home/Unity/cloisim/sample_resources/worlds"

Run CLOiSim

 ./run.sh cloisim.world

or you can execute binary file directly.

./CLOiSim.x86_64 -world lg_seocho.world

Run 'cloisim_ros' after running CLOiSim

• You need to run this package in order to publish sensor data in ROS2.

• Run bringup node in 'cloisim_ros' ros2 packages

• Latest version of CLOiSim will support only humble version of ROS2

• That's it. Have fun!!!

Debugging log

tail -f ~/.config/unity3d/LGE.CTO.AdvancedRoboticsLab/CLOiSim/Player.log

Control and external UI service

CLOiSim supports web-based control service through websocket as an external interface.

websocket service path: ws://127.0.0.1:8080/{service-name}

You can add markers like line, text, box, or sphere point and reset simulation by just sending a request data as a JSON format.

Read detail guide in Wiki pages.

CLOiSim + nav2

video link

Future Plan

New features or functions shall be developed on demand.

• Fully support to keep up with the latest version of 'SDF specifiaction'

  • Considering wrapper class for SDF c++ library.

• Add new sensor models and enhance sensor performance

• introduce programmable c++ plugin

• Performance optimization for sensors (Use DOT by unity?)

• Upgrade quality of graphical elements

• If you have any troubles or issues, please don't hesitate to create a new issue on 'Issues'. https://github.com/lge-ros2/cloisim/issues

감사합니다. Thank you

cloisim's Issues

Clarification on build process

Hello,
I found this gazebo replacement just recently and I see great potential in it.

However, I did not succeed in building cloisim myself following this guide, but this may be due to the fact that I am a novice in unity.

I proceeded as following:

• installed vulcan and other dependencies as described in guide
• installed unity 2020.3.13f1
• cloned cloisim, cloisim_ros and cloi_common_interfaces
• cloned sample_resources into cloisim
• built and sourced the ros packages
• opened unity
• added the cloisim folder to projects
  
• opened the project by clicking on it
• did NOT receive a popup about TMP imports, so I importet TMP like this
• get a blank window in unity, which results in a blank application when built
  

My directory structure is the following:

├── cloi_common_interfaces
│   ├── build
│   ├── install
│   ├── log
│   └── src
├── cloisim
│   ├── Assets
│   ├── Docker
│   ├── Library
│   ├── Logs
│   ├── Packages
│   ├── ProjectSettings
│   ├── sample_resources
│   ├── Temp
│   └── UserSettings
├── cloisim_ros
│   ├── build
│   ├── install
│   ├── log
│   └── src

I expected to see the shopping hall and a robot.
Any idea where I am going wrong?
How is the development workflow like, for instance if I want to use custom maps and robots?

Thanks in advance and best regards,

Max

Gazebo's World import[Question]

Hi, I've read that you support sdf. Is it possible to load gazebo world files with objects/meshes[for example AWS's world environments]?

[Question] How can I make the cloi navigate over the seocho world?

A clear and concise description of what is your question.
I've done to run the cloisim in a docker container and display the lg_seocho.world(and other worlds too), and to run the cloisim_ros in another docker container, but the cloi has never moved.
I manipulated it with teleop_twist_keyboard but still don't know how to apply your project like README demos.
I'm working on Ubuntu 20.04

My poor cloi is standing alone on the freezing lg seocho world.

Please gimme some advice to let cloi run freely!

A quick start example

I downloaded the sample-resource and the simple-device. Could you explain steps after <After run 'CLOiSim'> of Usage to run any simple example? I fail to locate any launch files to run ROS.

video and images

Unity_lidar_ros2.light.mp4

Unity_nav2_ros2.light.mp4

unity_multirobot.light.mp4

[Request] Front camera capture

Is your feature request related to a problem? Please describe.
I cannot get images from this simulator.

Describe the solution you'd like
I want to get images from the multi-robot-simulator. These images should be acquired from the cameras of (virtual) robots. Additionally, each image should synchronize with the robot's position.

Describe alternatives you've considered
Image capturing should be possible at any time. I want to adjust a frame rate and image resolution too.

Additional context