ioujenliu/afk

Asking for Knowledge : Training RL Agents to Query External Knowledge Using Language

[Project Website]

The repository contains Pytorch implementation of the Q-BabyAI, Q-TextWorld environments, and Asking for knowledge (AFK) agent.

If you used this code for your experiments or found it helpful, please consider citing the following paper:

@inproceedings{AFK2022,
  author = {Iou-Jen Liu$^\ast$ and Xingdi Yuan$^\ast$ and Marc-Alexandre C\^{o}t\'{e}$^\ast$ and Pierre-Yves Oudeyer and Alexander G. Schwing},
  title = {Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language},
  booktitle = {arXiv.},
  year = {2022},
  note = {$^\ast$ equal contribution}
}

Please see README.md in afk-q-babyai and afk-q-textworld/text_agent folder for detailed instruction for running the code.

Acknowledgement

The Q-babyAI environments and the RL agent are based on the BabyAI of https://github.com/mila-iqia/babyai

License

AFK is licensed under the MIT License

afk's Issues

BrokenPipeError

Hello, I am a student in Japan.
Your research is very interesting.
I actually tried to run your prepared code , but
I got the following error
Is there any solution for this?

"Function rng.randint(low, [high, size, dtype]) is marked as deprecated "
-= Games #[799572887, 1811408612, 1493662463, 1791150409, 2078953035] =-
Traceback (most recent call last):
File "test_cooking_afk_batch.py", line 136, in
main()
File "test_cooking_afk_batch.py", line 117, in main
assert word in "{} {} {}".format(obs_, inv_, desc_), word
AssertionError: cook
Exception ignored in: <function AsyncBatchEnv.del at 0x7f2029906e60>
Traceback (most recent call last):
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 196, in del
self.close()
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 189, in close
env.call("close")
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 63, in call
self._pipe.send(("call", method, args))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <function _ChildEnv.del at 0x7f20299069e0>
Traceback (most recent call last):
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 87, in del
self.call_sync("close")
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 75, in call_sync
self.call(*args)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 63, in call
self._pipe.send(("call", method, args))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <function _ChildEnv.del at 0x7f20299069e0>
Traceback (most recent call last):
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 87, in del
self.call_sync("close")
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 75, in call_sync
self.call(*args)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 63, in call
self._pipe.send(("call", method, args))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <function _ChildEnv.del at 0x7f20299069e0>
Traceback (most recent call last):
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 87, in del
self.call_sync("close")
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 75, in call_sync
self.call(*args)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 63, in call
self._pipe.send(("call", method, args))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <function _ChildEnv.del at 0x7f20299069e0>
Traceback (most recent call last):
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 87, in del
self.call_sync("close")
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 75, in call_sync
self.call(*args)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 63, in call
self._pipe.send(("call", method, args))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <function _ChildEnv.del at 0x7f20299069e0>
Traceback (most recent call last):
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 87, in del
self.call_sync("close")
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 75, in call_sync
self.call(*args)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/site-packages/textworld/envs/batch/batch_env.py", line 63, in call
self._pipe.send(("call", method, args))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
self._send(header + buf)
File "/home/yoneda/anaconda3/envs/afk/lib/python3.7/multiprocessing/connection.py", line 368, in _send
n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

oracle knowledge source

I read your paper I would like to thank you for oroviding the code.
However, I cant see what and were is the queryable oracle knowledge source in the cofe