thudm/autowebglm

An LLM-based Web Navigating Agent (KDD'24)

autowebglm's Issues

Please consider cite rft

I find you use RFT for further fine-tuning, please consider cite Scaling relationship on learning mathematical reasoning with large language models

Will you release your SFT/DPO data/Model?

Thank you for your excellent work. Will you release your SFT/DPO data? Thank you

Pillow==9.2.0 Pillow==10.1.0 重复了

miniwob++ 的 requirements.txt 里有2个版本

Typo in regex pattern in action parsing - fails to parse goto actions

Hi, awesome work, I really like the work you did. I just noticed that jump_to/goto actions do not work and I discovered that there is a small typo in the actions.py on line 1714 where instead of (\d+) is (d+)

如何使用，为什么代码里默认是GPT3.5,而不是chatglm

怎么使用呢？没看懂

MiniWoB 报错

运行python main.py [cudas] [test-amount] [model-path] [result-path]
显示以下错误，但是能运行
Setting eos_token is not supported, use the default one.
Setting pad_token is not supported, use the default one.
Setting unk_token is not supported, use the default one.

部分结果，基本都是0，是不是还需要配置什么？
后来增加了https://storage.googleapis.com/chrome-for-testing-public/126.0.6478.126/linux64/chromedriver-linux64.zip之后就不能跑了。
2024-07-04 23:27:27,399 - INFO - email-inbox-forward-nl 0.00
2024-07-04 23:27:27,399 - INFO - email-inbox-forward-nl-turk 0.00
2024-07-04 23:27:27,399 - INFO - email-inbox-nl-turk 0.00
2024-07-04 23:27:27,399 - INFO - enter-date 0.00
2024-07-04 23:27:27,399 - INFO - enter-password 0.00
2024-07-04 23:27:27,399 - INFO - enter-text 0.00
2024-07-04 23:27:27,399 - INFO - enter-text-dynamic 0.00
2024-07-04 23:27:27,399 - INFO - enter-time 0.00
2024-07-04 23:27:27,399 - INFO - focus-text 0.00
2024-07-04 23:27:27,399 - INFO - focus-text-2 0.00
2024-07-04 23:27:27,399 - INFO - grid-coordinate 0.00
2024-07-04 23:27:27,399 - INFO - guess-number 0.00
2024-07-04 23:27:27,399 - INFO - identify-shape 0.00
2024-07-04 23:27:27,399 - INFO - login-user 0.00
2024-07-04 23:27:27,400 - INFO - login-user-popup 0.00
2024-07-04 23:27:27,400 - INFO - multi-layouts 0.00
2024-07-04 23:27:27,400 - INFO - multi-orderings 0.00
2024-07-04 23:27:27,400 - INFO - navigate-tree 0.00
2024-07-04 23:27:27,400 - INFO - search-engine 0.00
2024-07-04 23:27:27,400 - INFO - social-media 0.00
2024-07-04 23:27:27,400 - INFO - social-media-all 0.00
2024-07-04 23:27:27,400 - INFO - social-media-some 0.00
2024-07-04 23:27:27,400 - INFO - tic-tac-toe 0.00
2024-07-04 23:27:27,400 - INFO - use-autocomplete 0.00
2024-07-04 23:27:27,400 - INFO - use-spinner 0.00
2024-07-04 23:27:27,400 - INFO - all 0.002

这个库干啥的，半天没看懂？

Several problems noticed

I followed the webarena instructions.
The first thing I notice is the gitlab docker image on the AWI somehow went into some issue, I reloaded it again using the .tar file and it works fine. (Also, there is a space issue with the sql-related code in this part, but it's easy to fix)

Seems like the webarena file on the AWI is not the same version as this the one in this repo. I used this repo version.

The "context_length" field was declared in line 140 in openai_utils.py, but not between line 149-156. Also in line 41 - 48 of utils.py should add a line of "context_length=lm_config.gen_config["context_length"],".

Then I met an issue with the list of stop_tokens. I checked that the list always be empty, so I just comment out line 43 in lm_config.py, and according places.

For the suggested command to run :
python run.py
--instruction_path agent/prompts/jsons/new_action_prompt.json \ # this is the reasoning agent prompt we used in the paper
--model gpt-3.5-turbo
--mode completion
--observation_type html
--action_set_tag id_html_nasc_tree
--result_dir <your_result_dir>
--test_start_idx 0
--test_end_idx 1
notice that if anyone wants to use gpt-3.5-turbo, he/she should change the mode to chat instead of completion. If you want to use the completion choice, you need to change to model to gpt-3.5-instruct. (Details:)

I used gpt-3.5-instruct, but the output shows that the task failed. When I tried to use chat option, the program somehow crashed saying there is an unhandled error.
Did not know what excatly happened, hope someone could tell me in the future, and apologies if I made mistakes in my statements.

Instructions how to use in production

Dear AutoWebGLM developers

Thank you for all your efforts

We want to use AutoWebGLM for extracting data from websites, is there any instructions how to do it, which python methods to call?

OpenAI Key in Code

The miniwob++ gpt4 llm provider (miniwob++/llms/providers/gpt4.py) contains what looks like a real OpenAI key. I would recommend removing it.

Issue in WebArena Browser Observation

Hi, great work!

I found a bug in the observation processing where it actually does not show the contents of input elements at all. So the model does not have enough information to complete some tasks. I thought it would be nice to report it. See the included image below.

The real web page has valuable information in the input boxes but autowebglm fails to show them to the model.