jonygao621/affordance-language

Code for RSS 2020 paper: Robot Object Retrieval with Contextual Natural Language Queries

Robot Object Retrieval with Contextual Natural Language Queries

• Read the paper: http://www.roboticsproceedings.org/rss16/p080.pdf
• Watch the virtual conference presentation: https://youtu.be/HOYvL5AwX38
• To evaluate the trained model, run eval.py with the desired mode (YCB, robot, eval)
• The annotated dataset of 655 verb-object pairs can be found at data/annotated-vo.csv (a valid verb-object pair is annotated with label '1')
• We also release a smaller dataset of 88 verb-object pairs over a subset of verbs from our original dataset and objects from the YCB object set, this dataset can be found at data/ycb-verb-object.csv
• To train and evaluate your own model on our dataset, run run.py (you will first need to run resnet.py to generate embeddings for your objects, and gen_data.py to generate train and test data for the model)
• If you find the dataset or code useful, please cite:

@article{nguyen2020robot,
  title={Robot Object Retrieval with Contextual Natural Language Queries},
  author={Nguyen, Thao and Gopalan, Nakul and Patel, Roma and Corsaro, Matt and Pavlick, Ellie and Tellex, Stefanie},
  journal={arXiv preprint arXiv:2006.13253},
  year={2020}
}