Comments (6)
-
GIGA focuses on generating grasps from a fixed single view. Although we have done some randomization on the camera view, GIGA has limited generalizability to arbitrary view. So I would suggest first trying aligning your camera view with the one we used in the simulation as accurate as possible and then test the pretrained ckpt.
-
As mentioned above, aligning the camera view should help improve the result. Our real-world depth is also very noisy, I think it should be fine as long as objects are correctly captured.
-
We didn't use any planner. The collision avoidance is actually learned implicitly through data prior. Because in the data generation, a grasp would be labelled as fail if it collides with other objects before getting to the target grasp pose. So our grasp predictor always prioritize the grasps that would not cause any collision.
from giga.
Following your advice, we set the camera pose as the same as in the simulation.
But the quality of the generated grasp is still low.
With the packed checkpoint you provide, I can only realize 20% grasp success rate.
Can you give me some advice what to do next?
from giga.
Hi, I think I know the issue. During real-robot experiment of GIGA, we found we were not able to align camera view perfectly, so we additionally trained a model with randomized side view (basically adding noise on the camera view when generating data). The resulting model is more robust to the misalignment of the camera view. I also uploaded those models here (https://utexas.box.com/s/47po84j62g7zgwogpr453sl3ch64vm6q).
Actually we kinda reproduce the GIGA on the real robot about a year ago and these models work fine. Only thing we need to fix is tuning an offset and add that to the predicted grasp, because the predicted grasp is with respect to the root of the gripper, while our real gripper is a bit different from the simulated one. You might also need to go through this.
Please let me know if this helps!
from giga.
Hi, I use your checkpoint but the accuracy is still low.
May I ask what camera are you using? And what configuration did you set (resolution, etc.)
from giga.
I think I found the bug. In the simulation, there is always a 5cm table in the workspace.
In order to generate a collision-free grasp, the model will tend to generate grasps much higher than 5cm. Besides, the grasps below 5.5cm will be filtered.
So I lower the task frame and everything works fine now.
from giga.
Sorry I was planning to look into this today. But glad you found the issue and GIGA works out now!
from giga.
Related Issues (20)
- New dataset HOT 3
- Visualizing on custom dataset HOT 9
- Visualization of data generation HOT 2
- visual HOT 1
- How to use GIGA on real robot? HOT 2
- how to query at a higher resolution of 60×60×60 HOT 4
- Installation error: " LINK : fatal error LNK1181: can not open input file“m.lib” " HOT 4
- About NVIDIA Driver in WSL2 HOT 5
- Scene Descriptor HOT 2
- [Question] How to get the 3d reconstruction at inference time? HOT 6
- No module named vgn HOT 2
- libmesh failed! HOT 2
- question about GIGA(HR) HOT 1
- Train GIGA HOT 1
- Can't log _aff.obj when running sim_grasp_multiple.py HOT 1
- Some confusion in the paper HOT 2
- The program that generates data gets stuck in the first loop HOT 2
- The time to generate the training set HOT 1
- Did you consider trying to avoid using the grasp data on the wrong voxels? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from giga.