Giter Site home page Giter Site logo

minedojo / voyager Goto Github PK

View Code? Open in Web Editor NEW
5.4K 62.0 492.0 5.15 MB

An Open-Ended Embodied Agent with Large Language Models

Home Page: https://voyager.minedojo.org/

License: MIT License

Python 33.54% JavaScript 66.46%
large-language-models embodied-learning open-ended-learning minecraft

voyager's Introduction

Voyager: An Open-Ended Embodied Agent with Large Language Models

We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention. Voyager consists of three key components: 1) an automatic curriculum that maximizes exploration, 2) an ever-growing skill library of executable code for storing and retrieving complex behaviors, and 3) a new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement. Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning. The skills developed by Voyager are temporally extended, interpretable, and compositional, which compounds the agent’s abilities rapidly and alleviates catastrophic forgetting. Empirically, Voyager shows strong in-context lifelong learning capability and exhibits exceptional proficiency in playing Minecraft. It obtains 3.3× more unique items, travels 2.3× longer distances, and unlocks key tech tree milestones up to 15.3× faster than prior SOTA. Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch, while other techniques struggle to generalize.

In this repo, we provide Voyager code. This codebase is under MIT License.

Installation

Voyager requires Python ≥ 3.9 and Node.js ≥ 16.13.0. We have tested on Ubuntu 20.04, Windows 11, and macOS. You need to follow the instructions below to install Voyager.

Python Install

git clone https://github.com/MineDojo/Voyager
cd Voyager
pip install -e .

Node.js Install

In addition to the Python dependencies, you need to install the following Node.js packages:

cd voyager/env/mineflayer
npm install -g npx
npm install
cd mineflayer-collectblock
npx tsc
cd ..
npm install

Minecraft Instance Install

Voyager depends on Minecraft game. You need to install Minecraft game and set up a Minecraft instance.

Follow the instructions in Minecraft Login Tutorial to set up your Minecraft Instance.

Fabric Mods Install

You need to install fabric mods to support all the features in Voyager. Remember to use the correct Fabric version of all the mods.

Follow the instructions in Fabric Mods Install to install the mods.

Getting Started

Voyager uses OpenAI's GPT-4 as the language model. You need to have an OpenAI API key to use Voyager. You can get one from here.

After the installation process, you can run Voyager by:

from voyager import Voyager

# You can also use mc_port instead of azure_login, but azure_login is highly recommended
azure_login = {
    "client_id": "YOUR_CLIENT_ID",
    "redirect_url": "https://127.0.0.1/auth-response",
    "secret_value": "[OPTIONAL] YOUR_SECRET_VALUE",
    "version": "fabric-loader-0.14.18-1.19", # the version Voyager is tested on
}
openai_api_key = "YOUR_API_KEY"

voyager = Voyager(
    azure_login=azure_login,
    openai_api_key=openai_api_key,
)

# start lifelong learning
voyager.learn()
  • If you are running with Azure Login for the first time, it will ask you to follow the command line instruction to generate a config file.
  • For Azure Login, you also need to select the world and open the world to LAN by yourself. After you run voyager.learn() the game will pop up soon, you need to:
    1. Select Singleplayer and press Create New World.
    2. Set Game Mode to Creative and Difficulty to Peaceful.
    3. After the world is created, press Esc key and press Open to LAN.
    4. Select Allow cheats: ON and press Start LAN World. You will see the bot join the world soon.

Resume from a checkpoint during learning

If you stop the learning process and want to resume from a checkpoint later, you can instantiate Voyager by:

from voyager import Voyager

voyager = Voyager(
    azure_login=azure_login,
    openai_api_key=openai_api_key,
    ckpt_dir="YOUR_CKPT_DIR",
    resume=True,
)

Run Voyager for a specific task with a learned skill library

If you want to run Voyager for a specific task with a learned skill library, you should first pass the skill library directory to Voyager:

from voyager import Voyager

# First instantiate Voyager with skill_library_dir.
voyager = Voyager(
    azure_login=azure_login,
    openai_api_key=openai_api_key,
    skill_library_dir="./skill_library/trial1", # Load a learned skill library.
    ckpt_dir="YOUR_CKPT_DIR", # Feel free to use a new dir. Do not use the same dir as skill library because new events will still be recorded to ckpt_dir. 
    resume=False, # Do not resume from a skill library because this is not learning.
)

Then, you can run task decomposition. Notice: Occasionally, the task decomposition may not be logical. If you notice the printed sub-goals are flawed, you can rerun the decomposition.

# Run task decomposition
task = "YOUR TASK" # e.g. "Craft a diamond pickaxe"
sub_goals = voyager.decompose_task(task=task)

Finally, you can run the sub-goals with the learned skill library:

voyager.inference(sub_goals=sub_goals)

For all valid skill libraries, see Learned Skill Libraries.

FAQ

If you have any questions, please check our FAQ first before opening an issue.

Paper and Citation

If you find our work useful, please consider citing us!

@article{wang2023voyager,
  title   = {Voyager: An Open-Ended Embodied Agent with Large Language Models},
  author  = {Guanzhi Wang and Yuqi Xie and Yunfan Jiang and Ajay Mandlekar and Chaowei Xiao and Yuke Zhu and Linxi Fan and Anima Anandkumar},
  year    = {2023},
  journal = {arXiv preprint arXiv: Arxiv-2305.16291}
}

Disclaimer: This project is strictly for research purposes, and not an official product from NVIDIA.

voyager's People

Contributors

cmaranes avatar dribnet avatar ep3p avatar follgad avatar jesserweigel avatar nathan-gage avatar romechenko avatar swen128 avatar theblazehen avatar wangguanzhi avatar xieleo5 avatar youmustfight avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

voyager's Issues

ERR_CONNECTION_REFUSED.

I get this
Creating Minecraft server
Please open https://login.microsoftonline.com/consumers/oauth2/v2.0/authorize?client_id=(removed)&response_type=code&redirect_uri=https%3A%2F%2F127.0.0.1%2Fauth-response&response_mode=query&scope=XboxLive.signin+offline_access&state=tRhwbidsvxBh7YfKKpBlBg&code_challenge=S0y5aMT54UHd34jK3HTFoPeEyJs8xxLvVJua-OoldnI&code_challenge_method=S256 in your browser and copy the url you are redirected into the prompt below.
(removed is the client_id I created)
but I just get a connection refusal each time, not sure what I've done wrong

tonen access problem but now is 271 line

C:\Users\ASUS>python C:\Users\ASUS\Voyager\AI_Minecraft.py
Creating Minecraft server
Please open https://login.microsoftonline.com/consumers/oauth2/v2.0/authorize?client_id=ab3074f2-1ea7-425a-8ace-96b0fe4ebf0d&response_type=code&redirect_uri=https%3A%2F%2F127.0.0.1%2Fauth-response&response_mode=query&scope=XboxLive.signin+offline_access&state=ABCxLmKy-FJZFdd7JDJ3Rg&code_challenge=Bvlys465ZivXQKua5QrxZqtOhFHvd_f03S0C43BMfhc&code_challenge_method=S256 in your browser and copy the url you are redirected into the prompt below.
https://127.0.0.1/auth-response?code=M.C103_BL2.2.32bbc6a8-8e19-9400-db2f-8dd6bcc9282f&state=ABCxLmKy-FJZFdd7JDJ3Rg
Traceback (most recent call last):
File "C:\Users\ASUS\Voyager\AI_Minecraft.py", line 12, in
voyager = Voyager(
^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\voyager.py", line 102, in init
self.env = VoyagerEnv(
^^^^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\env\bridge.py", line 42, in init
self.mc_instance = self.get_mc_instance()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\env\bridge.py", line 67, in get_mc_instance
return MinecraftInstance(
^^^^^^^^^^^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\env\minecraft_launcher.py", line 36, in init
self.mc_command = self.get_mc_command()
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\env\minecraft_launcher.py", line 89, in get_mc_command
login_data = minecraft_launcher_lib.microsoft_account.complete_login(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ASUS\AppData\Roaming\Python\Python311\site-packages\minecraft_launcher_lib\microsoft_account.py", line 271, in complete_login
access_token = account_request["access_token"]
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'access_token'

I don't think I fixed that .. line 271.. what this mean because I totally stuck in access_code

ConnectionError aborted

raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) Stopping mineflayer

What might be the reason for this? I am using Azure Login

resume=True?

since i get this error when re-running:

AssertionError: Skill Manager's vectordb is not synced with skills.json.
There are 3 skills in vectordb but 0 skills in skills.json.
Did you set resume=False when initializing the manager?
You may need to manually delete the vectordb directory for running from scratch.

do i just set resume=True when i call

python3 voyagerscript.py resume=True

or do I place the resume arg somewhere else?

Evaluation scripts used in voyager

Hi, thanks for sharing and this project is really inspiring.

I noticed that you mentioned some metrics in your paper to demonstrate the privileged ability of Voyager, could you give some detailed information wrt the evaluation, like from which files can we derive that conclusion? And if possible, a demo-like script would help a lot.

Bests

Issues with building and running on Minecraft Launcher

I used gradle build command (not gradlew build as the readme said) and i keep getting an error around

Build file '/Users/XYZ/Downloads/better-respawn-1.19/fabric/build.gradle' line: 155

which is the classifier 'shadow-dev' down below

shadowJar {
    configurations = [project.configurations.shadow]
    classifier 'shadow-dev'
    relocate 'de.maxhenkel.configbuilder', "de.maxhenkel.betterrespawn.configbuilder"
}

How do i solve this?

Also, is there a way to adapt the installation file to account for Minecraft Launcher and not azure:

from voyager import Voyager

# you can also use mc_port instead of azure_login, but azure_login is highly recommended
azure_login = {
    "client_id": "YOUR_CLIENT_ID",
    "redirect_url": "https://127.0.0.1/auth-response",
    "secret_value": "[OPTIONAL] YOUR_SECRET_VALUE",
    "version": "fabric-loader-0.14.18-1.19", # the version Voyager is tested on
}
openai_api_key = "YOUR_API_KEY"

voyager = Voyager(
    azure_login=azure_login,
    openai_api_key=openai_api_key,
)

# start lifelong learning
voyager.learn()

Can my checkpoint be published as an example?

Hi, I think for many people spending $50 will seem like a pretty big deal. I have already spent and would like to share my checkpoint at 160it.

Is there any way to give my checkpoint as an example so people can play without spending a lot of money?
I can't spend much time working on the project myself, but it will be interesting what people can do.

looks like that it is some problem with UnicodeDecodeError,i'm completely unfamiliar with it

PS D:\test\mine\Voyager> pip install -e .
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Obtaining file:///D:/test/mine/Voyager
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "D:\test\mine\Voyager\setup.py", line 38, in
long_description=_read_file("README.md"),
File "D:\test\mine\Voyager\setup.py", line 16, in _read_file
return fp.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xa5 in position 2098: illegal multibyte sequence
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

UserWarning: Subprocess mineflayer failed to start.

Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 37572.
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 6988.
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 38292.
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 33320.
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 17008.

and this runs on loop with different PID values....

What should I do?

TimeoutError: timed out

Bot has joined the game, I don't see any errors so I assume its working, but seems to be timing out.

Traceback (most recent call last):
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1374, in getresponse
response.begin()
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 318, in begin
version, status, reason = self._read_status()
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\socket.py", line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\adapters.py", line 489, in send
resp = conn.urlopen(
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\packages\six.py", line 770, in reraise
raise value
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 451, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 340, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='127.0.0.1', port=3000): Read timed out. (read timeout=600)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "H:\AI MC\Voyager\run.py", line 19, in
voyager.learn()
File "H:\AI MC\Voyager\voyager\voyager.py", line 303, in learn
self.env.reset(
File "H:\AI MC\Voyager\voyager\env\bridge.py", line 156, in reset
returned_data = self.check_process()
File "H:\AI MC\Voyager\voyager\env\bridge.py", line 93, in check_process
res = requests.post(
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\adapters.py", line 578, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='127.0.0.1', port=3000): Read timed out. (read timeout=600)

Not able to find 'YOUR_MINECRAFT_GAME_LOCATION/config/better-respawn' to edit

In the fabric mods installation step 4:

After you launch the game, go to YOUR_MINECRAFT_GAME_LOCATION/config/better-respawn, and modify the properties file with ...

I am unable to find the file. Do I need to run the script and then edit it, or launch the game once before and then this file is generated?
I cannot see any config folder currently in my .minecraft folder location.

Deoplyed to real world? How far are we from AGI?

1, Maybe a stupid question is skill library the deterministic factor that excel Voyager framework?
2, We can replace GPT4 with any other LLM? What least required specific skills should that LLM have? Meaning what field should that LLM be especially trained on (coding, reasoning)
3, How is minecraft's reward feed back loop compare to real-world environment, is there any research on that? Like, killing a pig and heal is not simply two java api call. What if the feedback long, hard and negative, and more importantly, partially informative.

https://127.0.0.1/auth-response connection refused

I ran the python script in README.md
and this showed up

Please open https://login.microsoftonline.com/consumers/oauth2/v2.0/authorize?client_id=... in your browser and copy the url you are redirected into the prompt below.

but whenever I click this link after I logged in microsoft account, I constantly get connection refused from 127.0.0.1

I'm not sure if this is a particular problem of this project, but I have no place to ask

I've done with disabling firewall and flushing DNS but that doesn't seem to solve my issue

Inquiry Regarding Resource Utilization and Associated Costs for GPT-4 and GPT-3.5

Dear Team,

First and foremost, I'd like to express my deep appreciation for the excellent work you have accomplished in this project. The level of professionalism and detail in your work is truly commendable.

As I'm looking forward to running your code, I am trying to estimate the associated costs. I was wondering if you could provide a rough estimation for the computational cost of running, for instance, 150 prompting iterations on both the GPT-4 and GPT-3.5 models?

This information would be highly beneficial in planning for my resource allocation. I understand the cost might vary based on several factors, however, any approximation or range would be greatly appreciated.

Thank you for your attention and assistance with this matter.

Best Regards.

Game crashes on final step after entering newly created world

I have successfully completed all steps and have been troubleshooting. I am now onto the final steps of creating the world after the game launches itself following these instructions:
For Azure Login, you also need to select the world and open the world to LAN by yourself. After you run voyager.learn() the game will pop up soon, you need to:
Select Singleplayer and press Create New World.
Set Game Mode to Creative and Difficulty to Peaceful.
After the world is created, press Esc key and press Open to LAN.
Select Allow cheats: ON and press Start LAN World. You will see the bot join the world soon.

The game then launches and I am able to create a world, I then press esc once I am inside the world and the game crashes before I have anytime to click Open to Lan and Select Allow Cheats On. I then look at my command prompt and this error appears:

(base) PS C:\Users\chris\Voyager> python voyager2.py
Creating Minecraft server
Using embedded DuckDB with persistence: data will be stored in: ckpt/curriculum/vectordb
Using embedded DuckDB with persistence: data will be stored in: ckpt/skill/vectordb
Starting Minecraft server
Subprocess minecraft started with PID 3140.
Traceback (most recent call last):
File "C:\Users\chris\Voyager\voyager2.py", line 18, in
C:\Users\chris\Voyager\voyager\env\process_monitor.py:65: UserWarning: Subprocess minecraft failed to start.
warnings.warn(f"Subprocess {self.name} failed to start.")
Stopping mineflayer
voyager.learn()
File "C:\Users\chris\Voyager\voyager\voyager.py", line 303, in learn
self.env.reset(
File "C:\Users\chris\Voyager\voyager\env\bridge.py", line 156, in reset
returned_data = self.check_process()
File "C:\Users\chris\Voyager\voyager\env\bridge.py", line 79, in check_process
self.mc_instance.run()
File "C:\Users\chris\Voyager\voyager\env\minecraft_launcher.py", line 115, in run
match = re.search(pattern, self.mc_process.ready_line)
File "C:\Users\chris\miniconda3\lib\re.py", line 200, in search
return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object
(base) PS C:\Users\chris\Voyager>

Does anyone know what may be causing this issue or if i am the only one? I will also attach an image of my command prompt so its easier to read the error.

Thanks

--Christian

Creating Minecraft server ERROR

Traceback (most recent call last):
File "/Users/roc/tool/Voyager/ttest.py", line 12, in
voyager = Voyager(
File "/Users/roc/tool/Voyager/voyager/voyager.py", line 102, in init
self.env = VoyagerEnv(
File "/Users/roc/tool/Voyager/voyager/env/bridge.py", line 42, in init
self.mc_instance = self.get_mc_instance()
File "/Users/roc/tool/Voyager/voyager/env/bridge.py", line 67, in get_mc_instance
return MinecraftInstance(
File "/Users/roc/tool/Voyager/voyager/env/minecraft_launcher.py", line 36, in init
self.mc_command = self.get_mc_command()
File "/Users/roc/tool/Voyager/voyager/env/minecraft_launcher.py", line 89, in get_mc_command
login_data = minecraft_launcher_lib.microsoft_account.complete_login(
File "/Users/roc/miniconda3/lib/python3.10/site-packages/minecraft_launcher_lib/microsoft_account.py", line 261, in complete_login
token = token_request["access_token"]
KeyError: 'access_token'

Bot leaves and joins immediately. How do I give the bot commands, get the bot to stay? Is this an error?

Hey everyone so I am able to do everything the tutorial asks and the bot joins successfully, it worked entirely.

HOWEVER, right after joining the bot leaves. I will be attaching my logs to my mineflayer, as well as minecraft and then a screenshot of the console output below! if anyone has any advice for what it may be the help is much appreciated!

Mineflayer - 20230530_111719.log
Minecraft - 20230530_111719.log

Captureee

Items not being added to inventory

It seems like the bot is not recognizing items in it's inventory. I see it mining but the console output does not register the new items.

Is this a known issue? Would could be causing it? I am trying to go through the code to see what the issue might be but I haven't found anything so far.

Inventory not updating stuck mining wood endlessly

The bot seems to be mining wood logs but nothing seems to register in the inventory. I've been running it for about an hour.

Here is the output message

Chat log: No wood log found in inventory. Mining a wood log...
Collect finish!
Wood log mined.
Inventory: No wood log found.

Biome: plains

Time: day

Nearby blocks: grass_block, grass, oak_leaves, oak_log, dirt, stone

Nearby entities (nearest to farthest): bee, sheep

Health: 20.0/20

Hunger: 20.0/20

Position: x=1.5, y=68.0, z=23.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Craft 4 wooden planks

Context: Question: How to craft 4 wooden planks in Minecraft?
Answer: To craft 4 wooden planks in Minecraft, you need to place one wooden log in any square of the crafting grid. This will give you 4 wooden planks.

Critique: None

Action Agent ai message
Explain: The code failed to find the mined wood log in the inventory after mining it. This might be due to the bot not waiting long enough to pick up the dropped item.

Plan:

  1. Increase the waiting time after mining the wood log to ensure the bot picks up the dropped item.
  2. Check the inventory for the wood log again after waiting.
  3. If the wood log is still not found, mine another wood log and repeat the process.
  4. Craft 4 wooden planks from the wood log.

Code:

async function mineWoodLog(bot) {
  // Find a wood log block nearby
  const logBlock = bot.findBlock({
    matching: block => {
      return block.name.endsWith("_log");
    },
    maxDistance: 32
  });
  if (!logBlock) {
    bot.chat("No wood log found nearby. Exploring...");
    await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
      const logBlock = bot.findBlock({
        matching: block => {
          return block.name.endsWith("_log");
        },
        maxDistance: 32
      });
      return logBlock;
    });
  }

  // Mine the wood log block
  await mineBlock(bot, logBlock.name, 1);
  bot.chat("Wood log mined.");

  // Move to the position where the wood log was mined and wait for a moment to pick up the dropped item
  await bot.pathfinder.goto(new GoalNear(logBlock.position.x, logBlock.position.y, logBlock.position.z, 1));
  await bot.waitForTicks(200); // Wait for 10 seconds (200 ticks)

  // Check the inventory for the mined wood log
  let logItem = bot.inventory.items().find(item => item.name === logBlock.name);

  // Chat the inventory information
  if (logItem) {
    bot.chat(`Inventory: ${logItem.name} x${logItem.count}`);
  } else {
    bot.chat("Inventory: No wood log found.");
  }
}

async function craftWoodenPlanks(bot) {
  // Check if there is a wood log in the inventory
  let woodLog = bot.inventory.items().find(item => item.name.endsWith("_log"));

  // If not, mine a wood log
  while (!woodLog) {
    bot.chat("No wood log found in inventory. Mining a wood log...");
    await mineWoodLog(bot);
    woodLog = bot.inventory.items().find(item => item.name.endsWith("_log"));
  }

  // Craft 4 wooden planks from the wood log
  const logName = woodLog.name;
  const plankName = logName.replace("_log", "_planks");
  bot.chat(`Crafting 4 ${plankName} from ${logName}...`);
  await craftItem(bot, plankName, 1);
  bot.chat(`Crafted 4 ${plankName}.`);
}

ERR_CONNECTION_REFUSED when I use browser to access the created minecraft server

I follow the tutorial in the installation blog to create the minecraft server on my local computer, and the program returns the first link (https://login.microsoftonline.com/consumers/oauth2/v2.0/authorize?client_id=********) to me so that I can access the server in browser.

By clicking the link, I jump to a website where I was required to loggin with my microsoft account.

After loggin, the website address turns to the second link (https://127.0.0.1/auth-response?code=*******) and raises an error: ERR_CONNECTION_REFUSED.

How can I solve this error? Asking for help

always "Connection aborted" never success.

Before submitting an issue, make sure you read the FAQ.md

Briefly describe your issue

"""
voyager = Voyager(
azure_login=azure_login,
openai_api_key=openai_api_key,
)

start lifelong learning

voyager.learn()
"""
always throw this error. many time

image

...

Please provide your python, nodejs, Minecraft, and Fabric versions here

image

Minecraft:1.19
Fabric: fabric-loader-0.14.18-1.19
OS: Macos 12.6.5

...

[If applicable] Please provide the Minefalyer and Minecraft logs, you can find the log under logs folder

Minefalyer: 20230601_133536.log.zip
Minecraft: 20230601_133537.log.zip

[If applicable] Please provide the GPT conversations that are printed each round.

""""""""""""
glennxu@xugaorendeMacBook-Pro Voyager % /usr/bin/python3 run_voyager.py
Creating Minecraft server
Starting Minecraft server
Subprocess minecraft started with PID 5119.
The mc server is listening on port 60470
Server started on port 60470
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5163.
Server started on port 3000

Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5176.
Server started on port 3000

Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round

Execution error: No error

Chat log: None

Biome: plains

Time: sunrise

Nearby blocks: grass_block, dirt, stone, grass, iron_ore, dandelion, cornflower, poppy

Nearby entities (nearest to farthest): rabbit

Health: 20.0/20

Hunger: 20.0/20

Position: x=-31.5, y=95.0, z=-31.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: None

Action Agent ai message
Explain: No explanation needed as this is the first task.

Plan:

  1. Find a wood log block (oak, birch, spruce, jungle, acacia, dark oak, or mangrove) using exploreUntil.
  2. Mine the wood log block using mineBlock.

Code:

async function mineWoodLog(bot) {
  const woodLogNames = [
    "oak_log",
    "birch_log",
    "spruce_log",
    "jungle_log",
    "acacia_log",
    "dark_oak_log",
    "mangrove_log",
  ];

  // Find a wood log block
  const woodLogBlock = await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
    const woodLog = bot.findBlock({
      matching: (block) => woodLogNames.includes(block.name),
      maxDistance: 32,
    });
    return woodLog;
  });

  if (woodLogBlock) {
    // Mine the wood log block
    await mineBlock(bot, woodLogBlock.name);
    bot.chat("Wood log mined.");
  } else {
    bot.chat("Could not find a wood log.");
  }
}

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5198.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5208.
Server started on port 3000

Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round

Execution error: No error

Chat log: None

Biome: plains

Time: day

Nearby blocks: grass_block, dirt, grass, stone

Nearby entities (nearest to farthest): None

Health: 20.0/20

Hunger: 20.0/20

Position: x=-15.5, y=95.0, z=-31.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: None

Action Agent ai message
Explain: No explanation needed as this is the first task.

Plan:

  1. Find a wood log block nearby.
  2. Mine the wood log block.

Code:

async function mineWoodLog(bot) {
  // Find a wood log block nearby
  const logBlock = bot.findBlock({
    matching: (block) => {
      return (
        block.name === "oak_log" ||
        block.name === "birch_log" ||
        block.name === "spruce_log" ||
        block.name === "jungle_log" ||
        block.name === "acacia_log" ||
        block.name === "dark_oak_log" ||
        block.name === "mangrove_log"
      );
    },
    maxDistance: 32,
  });

  if (!logBlock) {
    // If no wood log block is found, explore the area
    await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
      const logBlock = bot.findBlock({
        matching: (block) => {
          return (
            block.name === "oak_log" ||
            block.name === "birch_log" ||
            block.name === "spruce_log" ||
            block.name === "jungle_log" ||
            block.name === "acacia_log" ||
            block.name === "dark_oak_log" ||
            block.name === "mangrove_log"
          );
        },
        maxDistance: 32,
      });
      return logBlock;
    });
  }

  // Mine the wood log block
  await mineBlock(bot, logBlock.name, 1);
}

mineWoodLog(bot);

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5245.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5261.
Server started on port 3000

Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round

Execution error: No error

Chat log: None

Biome: plains

Time: day

Nearby blocks: stone, dirt, copper_ore, grass_block, water

Nearby entities (nearest to farthest): None

Health: 20.0/20

Hunger: 20.0/20

Position: x=-15.5, y=90.0, z=-47.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: None

Action Agent ai message
Explain: No explanation needed as this is the first task.

Plan:

  1. Find a wood log block nearby.
  2. Mine the wood log block.

Code:

async function mineWoodLog(bot) {
  // Find a wood log block nearby
  const logBlock = bot.findBlock({
    matching: (block) => {
      return (
        block.name === "oak_log" ||
        block.name === "birch_log" ||
        block.name === "spruce_log" ||
        block.name === "jungle_log" ||
        block.name === "acacia_log" ||
        block.name === "dark_oak_log" ||
        block.name === "mangrove_log"
      );
    },
    maxDistance: 32,
  });

  if (!logBlock) {
    // If no wood log block is found, explore the area
    await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
      const logBlock = bot.findBlock({
        matching: (block) => {
          return (
            block.name === "oak_log" ||
            block.name === "birch_log" ||
            block.name === "spruce_log" ||
            block.name === "jungle_log" ||
            block.name === "acacia_log" ||
            block.name === "dark_oak_log" ||
            block.name === "mangrove_log"
          );
        },
        maxDistance: 32,
      });
      return logBlock;
    });
  }

  // Mine the wood log block
  await mineBlock(bot, logBlock.name, 1);
}

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5305.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5314.
Server started on port 3000

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5327.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5338.
Server started on port 3000

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5352.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5360.
Server started on port 3000

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5387.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5400.
Server started on port 3000

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5419.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5673.
Server started on port 3000

Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round

Execution error: No error

Chat log: None

Biome: plains

Time: noon

Nearby blocks: water, dirt, grass_block, stone, coal_ore, grass, copper_ore

Nearby entities (nearest to farthest): squid

Health: 20.0/20

Hunger: 20.0/20

Position: x=16.5, y=63.0, z=-44.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: None

Action Agent ai message
Explain: There is no code from the last round, so we will start from scratch.

Plan:

  1. Explore until finding a wood log (oak, birch, spruce, jungle, acacia, dark oak, or mangrove).
  2. Mine 1 wood log.

Code:

async function mineWoodLog(bot) {
  // Explore until finding a wood log
  const logTypes = [
    "oak_log",
    "birch_log",
    "spruce_log",
    "jungle_log",
    "acacia_log",
    "dark_oak_log",
    "mangrove_log",
  ];
  const log = await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
    const foundLog = bot.findBlock({
      matching: (block) => logTypes.includes(block.name),
      maxDistance: 32,
    });
    return foundLog;
  });

  if (log) {
    // Mine 1 wood log
    await mineBlock(bot, log.name, 1);
    bot.chat("Mined 1 wood log.");
  } else {
    bot.chat("Could not find a wood log.");
  }
}

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5823.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5838.
Server started on port 3000

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5869.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5883.
Server started on port 3000

Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5911.
Server started on port 3000

Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5935.
Server started on port 3000

Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round

Execution error: No error

Chat log: None

Biome: plains

Time: noon

Nearby blocks: stone, dirt, grass_block, coal_ore

Nearby entities (nearest to farthest): squid, salmon

Health: 20.0/20

Hunger: 20.0/20

Position: x=10.5, y=67.0, z=-31.5

Equipment: [None, None, None, None, None, None]

Inventory (0/36): Empty

Chests: None

Task: Mine 1 wood log

Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.

Critique: None
""""""""""

openai request rate

I let it run about an hour and it got itself rate limited with openai as well as cost ~$20 in that hour in api calls.

It made 214 api requests in that hour to gpt-4-0314. It didn't complete any tasks. Tons of failed mine wood attempts in the log.

is this expected behavior?

ubuntu 22.04, MC 1.19, fabric 14.18

Bot fails to deposit items into chest, claims "no chest at coordinates, it is chest"

Before submitting an issue, make sure you read the FAQ.md

I have read the FAQ.

Briefly describe your issue

Bot fails to deposit items into chest.

Error occurs In your program code: throw new Error(
No chest at (62.5, 52, -47.470345346655634), it is chest
at line 17:await depositItemIntoChest(bot, position, uselessItems); in your code

Please provide your python, nodejs, Minecraft, and Fabric versions here

Python 3.10.11, node.js v16.14.0, Minecraft 1.19, fabric-0.14.18-1.19 I have all the exact mods, except better respawn as I want my bots to be able to handle death.

[If applicable] Please provide the Mineflayer and Minecraft logs, you can find the log under logs folder

Important lines from mineflayer logs:
2023-05-31 21:36:53,884 - mineflayer - INFO - Error: No chest at (62.5, 52, -47.470345346655634), it is chest
2023-05-31 21:36:53,884 - mineflayer - INFO - at moveToChest (eval at evaluateCode Voyager\voyager\env\mineflayer\index.js:256:19), :1382:15)
2023-05-31 21:36:53,885 - mineflayer - INFO - at depositItemIntoChest (eval at evaluateCode Voyager\voyager\env\mineflayer\index.js:256:19), :1331:11)

[If applicable] Please provide the GPT conversations that are printed each round.

This has happened across so many different attempts at this point through multiple manual critiques, but I recall this problem ending my first fully auto run.

"No functions found" errors are not fed back to the prompt.

****Action Agent ai message****
Plan:
1) Check if there are 3 raw beef in the inventory. If not, explore until find a cow and kill it to get raw beef.
2) Check if there is a furnace in the inventory. If not, craft one using 8 cobblestones.
3) Place the furnace near the player.
4) Open the furnace menu.
5) Put 3 raw beef in the top slot of the furnace.
6) Put a fuel source, such as coal or wood, in the bottom slot of the furnace.
7) Wait for the beef to cook. It should take about 10 seconds.
8) Once the beef is cooked, it will appear in the bottom slot of the furnace. Take it out and put it in the inventory.

Code:
```javascript
async
****Recorder message: 3790 ticks have elapsed****
****Recorder message: 106 iteration passed****
Error parsing action response (before program execution): No functions found Trying again!

I noticed that the code generated by GPT has been reviewed by the Action agent first, but when there are errors in the code, they cannot be fed back to the prompt, resulting in the same error in the following code.

For example, the above code has an error of "No functions found", the attribute "Execution error" of following prompt is No error

Is it a bug ?

****Action Agent human message****
Code from the last round: No code in the first round

Execution error: No error

Chat log: None

Biome: forest

Time: night

Question about `control_primitives` and `control_primitives_context`

There seem to be functions of the same name under the two folders, one folder is used for skill manager agent while the other is used for action agent, I'm wondering what is the reason two function implementations of the same name is needed? Could anyone help me understand?

Having issues running Gradlew Build with better-respawn

It seems I am unable to run the code gradlew build after removing forge from the string next to common and fabric. Upon entering it says an error, ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.

Please set the JAVA_HOME variable in your environment to match the
location of your Java installation.

I have set my environment variables quite a few times and still am unable to run the code after resetting, etc

KeyError: 'access_token' when initiating

I've inserted the link on the browser to the prompt after running voyager, but then the following issue occurred

Traceback (most recent call last):
File "C:\Users\yangh\Documents\Voyager\run.py", line 12, in
voyager = Voyager(
^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\voyager.py", line 102, in init
self.env = VoyagerEnv(
^^^^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\env\bridge.py", line 42, in init
self.mc_instance = self.get_mc_instance()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\env\bridge.py", line 67, in get_mc_instance
return MinecraftInstance(
^^^^^^^^^^^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\env\minecraft_launcher.py", line 36, in init
self.mc_command = self.get_mc_command()
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\env\minecraft_launcher.py", line 89, in get_mc_command
login_data = minecraft_launcher_lib.microsoft_account.complete_login(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\yangh\AppData\Local\Programs\Python\Python311\Lib\site-packages\minecraft_launcher_lib\microsoft_account.py", line 261, in complete_login
token = token_request["access_token"]
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'access_token'

run.py is the one I made (strictly based on README.md) for running voyager

from voyager import Voyager

# you can also use mc_port instead of azure_login, but azure_login is highly recommended
azure_login = {
    "client_id": "[I've filled this section]",
    "redirect_url": "https://127.0.0.1/auth-response",
    "secret_value": "[this one too]",
    "version": "fabric-loader-0.14.21-1.19", # the version here has changed
}
openai_api_key = "[and this one either]"

voyager = Voyager(
    azure_login=azure_login,
    openai_api_key=openai_api_key,
)

# start lifelong learning
voyager.learn()

warnings.warn(f"Subprocess {self.name} failed to start.")

as title.
perplexity and gpt3 can't help with this
\Voyager\venv_Voyager\lib\site-packages\requests\adapters.py", line 519, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /start (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000016038136400>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
code:

from voyager import Voyager

voyager = Voyager(
    # azure_login=azure_login,
    mc_port = 65243,
    openai_api_key=openai_api_key
    # action_agent_model_name = 'gpt-3.5-turbo',
    # curriculum_agent_model_name = 'gpt-3.5-turbo'

    # curriculum_agent_qa_model_name = 'gpt-3.5-turbo',
    # critic_agent_model_name = 'gpt-3.5-turbo',
    # skill_manager_model_name = 'gpt-3.5-turbo',
)
voyager.learn()

steps:
install minecraft
find login from 2013
call microsoft
get told its out of hours
figure it out
install wrong version of fabric
install mods
forget to install final mod
compile it
download the jar version because you didnt read the instructions
ask gpt
gpt figures out that its the wrong version of the mod because it read the tl;dr instructions
copy the compiled file
install another wrong version of fabric
install right version of fabric
follow minecraft instructions.
get port number
input it to script with API key
run the script as above

tl;dr STEPS TO REPRODUCE
====================
follow minecraft instructions to compile and install mods
get port number from game
input it to script with API key
run the script w/ 2 parameters as above
voyager.learn()
mineflayer crashes.
====================

RuntimeError: Minecraft server reply with code 400

I've managed to boot up the MC instance and created the world, but it cannot create bots with error "Minecraft server reply with code 400"

Below are the logs:

Creating Minecraft server
Starting Minecraft server
Subprocess minecraft started with PID 27212.
The mc server is listening on port 54705
Server started on port 54705
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 6944.
Server started on port 3000

Traceback (most recent call last):
returned_data = self.check_process()
^^^^^^^^^^^^^^^^^^^^
File "c:\work\python\git\voyager\voyager\env\bridge.py", line 100, in check_process
raise RuntimeError(
RuntimeError: Minecraft server reply with code 400
Stopping mineflayer

Will a learned checkpoint be added to the repo?

This is a very interesting experiment! Really like the idea of accumulating skills as executable codes.

I can't find any checkpoint in this repo, and want to take a quick look at the learned skill codes. Will any checkpoint be added to this repo in the near future?

Thanks.

First person view is possible, at least with mc_port

In single player open to lan with cheats and survival, use the port for mc_port, once the bot joins. Then run /gamemode spectator command in in-game text, from the player in single player, then run /spectate bot command in in-game text.

This should work until the bot leaves and rejoins. Then you just need to rerun the /spectate bot command. There is definitely a way to automate this with repeating command blocks and chains, but idk enough about command blocks to be able to figure it out right now.

How to use proxy for testing this code?

I have to use the vpn for testing this code. When I open vpn for test.

I try to use this code:
"""
voyager = Voyager(
azure_login=azure_login,
skill_manager_retrieval_top_k=0,
max_iterations=200,
resume=True,
ckpt_dir="./checkpoints/ckpt",
openai_api_key=openai_api_key,
)

voyager.learn()
"""

image image image

always have many errors.
I need help!!!

Did you do any tests with an additional refactoring agent?

From my experiences gpt-4 is quite good at refactoring code for reusability. Your current set of agents does influence reusability with the prompt action_template.txt and the third code criteria:

"Your function will be reused for building more complex functions. Therefore, you should make it generic and reusable. You should not make strong assumption about the inventory (as it may be changed at a later time), and therefore you should always check whether you have the required items before using them. If not, you should first collect the required items and reuse the above useful programs."

However, I find this influence lacking or nonexistent, though I'm confident without it, performance would likely degrade. With all this in mind. I think a complete agent for refactoring, while requiring another gpt-4 call, would be useful in extending hierarchical function composition as well as limiting the size of the skill library thus refining the top 5 skills returned to more meaningful skills.
I still need to more deeply read the paper, look through the codebase more, and experiment to confirm my hunches. My capacity with OAI API is limited so if you did experiment with this at all and see any noticeable difference, please let me know before I proceed, if not any advice on where in the codebase to peruse to best get started would be equally appreciated.
If this proves to be fruitful, next directions could be additions to the refactoring step that would use gpt3.5 to further the performance, I don't think more than one gpt4 call for the actual refactoring is necessary, but I do believe that a gpt3.5 call between code generation and refactoring could improve performance while maintaining low additional costs.

General issues with Cheats not enabled

Main issue:
Bot leaves after task and does not rejoin.

Secondary issues with the presence of cheats:
Bot toggles tile drops, this seems unnecessary and, in my opinion, should not be toggled.
Also freezing the entire server is not really a useful strategy for cooperating with bots. Instead in my opinion the bot should leave and join the server when it would freeze and unfreeze it. This could be considered as a player's time to think in between overarching tasks, really only draws similarities to larger time-scale tasks in humans.
For the keep inventory toggling, I think this should really be an optional parameter rather than an assumed necessity, in my opinion the bot should be able to be tested if it can handle death.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.