sheng-j/dom-q-net

Graph-based Deep Q Network for Web Navigation

DOM-Q-NET: Grounded RL on Structured Language

"DOM-Q-NET: Grounded RL on Structured Language" International Conference on Learning Representations (2019). Sheng Jia, Jamie Kiros, Jimmy Ba. [arxiv] [openreview]

Demo

Trained multitask agent: https://www.youtube.com/watch?v=eGzTDIvX4IY
Facebook login: https://www.youtube.com/watch?v=IQytRUKmWhs&t=2s

Requirement

Need to download selenium & install chrome driver for selenium..

Installation

1. Clone this repo
2. Download MiniWoB++ environment from the original repo https://github.com/stanfordnlp/miniwob-plusplus
   and copy miniwob-plusplus/html folder to miniwob/html in this repo
   
3. In fact, this html folder could be stored anywhere, but remember to perform one of the following actions:
   

• Set environment variable "WOB_PATH" to
  file://"your-path-to-miniwob-plusplus"/html/miniwob
  E.g. "your-path-to-miniwob-plusplus" is "/h/sheng/DOM-Q-NET/miniwob
• Directly modify the base_url on line 33 of instance.py to
  "your-path-to-miniwob-plusplus"/html/miniwob
  In my case, base_url='file:///h/sheng/DOM-Q-NET/miniwob/html/miniwob/'
  

Run experiment

Experiment launch files are stored under runs For example,

cd runs/hard2medium9tasks/
sh run1.sh

will launch a 11 multi-task (social-media search-engine login-user enter-password click-checkboxes click-option enter-dynamic-text enter-text email-inbox-delete click-tab-2 navigation-tree) experiment.

Multitask Assumptions

State & Action restrictions

Item	Maximum number of items
DOM tree leaves (action space)	160
DOM tree	200
Instruction tokens	16

Attribute embeddings & vocabulary

Attribute	max vocabulary	Embedding dimension
Tag	100	16
Text (shared with instructions)	600	48
Class	100	16

• UNKnown tokens
  These are assigned to a random vector such that the cosine distance with the text attribute can yield 1.0 for the direct alignment.

Acknowledgement

Credit to Dopamine for the implementation of prioritized replay used in dstructs/dopamine_segtree.py

dom-q-net's Issues

Trying to reproduce facebook login

Hi,
I am trying to reproduce facebook login demo by running the hard2medium9tasks and facing the following challenges -

• Linking the code to live browser instance and sending commands from terminal: I came across demo.py file and it looks like it runs on the miniWob examples. It will be great if you can provide a demo file which can be used to test on any website through command input via terminal.
• Generalization accuracy for facebook (or other websites) login - I trained the hard2medium9tasks model for 30000 iterations and tried running login user scenario for miniWob websites and the average reward is consistently 1. But when I used this model on facebook webpage (passed the static webpage content through miniwob), the reward is -1. I can see the model able to detect the input fields but not able to paste the right username and password fields. I would like to know if I have to run for 150000 iterations or I can see better results at a lower number of iterations.

Question about PR - It will be great to know if your repository is open for PRs and open-source contributions.