minedojo / voyager Goto Github PK
View Code? Open in Web Editor NEWAn Open-Ended Embodied Agent with Large Language Models
Home Page: https://voyager.minedojo.org/
License: MIT License
An Open-Ended Embodied Agent with Large Language Models
Home Page: https://voyager.minedojo.org/
License: MIT License
I used gradle build
command (not gradlew build as the readme said) and i keep getting an error around
Build file '/Users/XYZ/Downloads/better-respawn-1.19/fabric/build.gradle' line: 155
which is the classifier 'shadow-dev' down below
shadowJar {
configurations = [project.configurations.shadow]
classifier 'shadow-dev'
relocate 'de.maxhenkel.configbuilder', "de.maxhenkel.betterrespawn.configbuilder"
}
How do i solve this?
Also, is there a way to adapt the installation file to account for Minecraft Launcher and not azure:
from voyager import Voyager
# you can also use mc_port instead of azure_login, but azure_login is highly recommended
azure_login = {
"client_id": "YOUR_CLIENT_ID",
"redirect_url": "https://127.0.0.1/auth-response",
"secret_value": "[OPTIONAL] YOUR_SECRET_VALUE",
"version": "fabric-loader-0.14.18-1.19", # the version Voyager is tested on
}
openai_api_key = "YOUR_API_KEY"
voyager = Voyager(
azure_login=azure_login,
openai_api_key=openai_api_key,
)
# start lifelong learning
voyager.learn()
There seem to be functions of the same name under the two folders, one folder is used for skill manager agent while the other is used for action agent, I'm wondering what is the reason two function implementations of the same name is needed? Could anyone help me understand?
In my chrome browser after login i have ERR_CONNECTION_REFUSED
I have read the FAQ.
Bot fails to deposit items into chest.
Error occurs In your program code: throw new Error(
No chest at (62.5, 52, -47.470345346655634), it is chest
at line 17:await depositItemIntoChest(bot, position, uselessItems); in your code
Python 3.10.11, node.js v16.14.0, Minecraft 1.19, fabric-0.14.18-1.19 I have all the exact mods, except better respawn as I want my bots to be able to handle death.
logs
folderImportant lines from mineflayer logs:
2023-05-31 21:36:53,884 - mineflayer - INFO - Error: No chest at (62.5, 52, -47.470345346655634), it is chest
2023-05-31 21:36:53,884 - mineflayer - INFO - at moveToChest (eval at evaluateCode Voyager\voyager\env\mineflayer\index.js:256:19), :1382:15)
2023-05-31 21:36:53,885 - mineflayer - INFO - at depositItemIntoChest (eval at evaluateCode Voyager\voyager\env\mineflayer\index.js:256:19), :1331:11)
This has happened across so many different attempts at this point through multiple manual critiques, but I recall this problem ending my first fully auto run.
1, Maybe a stupid question is skill library the deterministic factor that excel Voyager framework?
2, We can replace GPT4 with any other LLM? What least required specific skills should that LLM have? Meaning what field should that LLM be especially trained on (coding, reasoning)
3, How is minecraft's reward feed back loop compare to real-world environment, is there any research on that? Like, killing a pig and heal is not simply two java api call. What if the feedback long, hard and negative, and more importantly, partially informative.
In single player open to lan with cheats and survival, use the port for mc_port, once the bot joins. Then run /gamemode spectator
command in in-game text, from the player in single player, then run /spectate bot
command in in-game text.
This should work until the bot leaves and rejoins. Then you just need to rerun the /spectate bot
command. There is definitely a way to automate this with repeating command blocks and chains, but idk enough about command blocks to be able to figure it out right now.
I want someone can create a Discord to talk about Voyager. because this project is very useful. please!!!
I've managed to boot up the MC instance and created the world, but it cannot create bots with error "Minecraft server reply with code 400"
Below are the logs:
Creating Minecraft server
Starting Minecraft server
Subprocess minecraft started with PID 27212.
The mc server is listening on port 54705
Server started on port 54705
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 6944.
Server started on port 3000
Traceback (most recent call last):
returned_data = self.check_process()
^^^^^^^^^^^^^^^^^^^^
File "c:\work\python\git\voyager\voyager\env\bridge.py", line 100, in check_process
raise RuntimeError(
RuntimeError: Minecraft server reply with code 400
Stopping mineflayer
PS D:\test\mine\Voyager> pip install -e .
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Obtaining file:///D:/test/mine/Voyager
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "D:\test\mine\Voyager\setup.py", line 38, in
long_description=_read_file("README.md"),
File "D:\test\mine\Voyager\setup.py", line 16, in _read_file
return fp.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xa5 in position 2098: illegal multibyte sequence
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Hi there, I followed the instructions in README to copy the url from browser, the url is "https://127.0.0.1/auth-response?code=M.CXXXXXXXXXX&state=XXXXXXXXXX", and it led me to the following
Already followed the intruction to change the app to web
, still not working :(
I want to download mc-mods for fabric-api, but I find the link which is 404 (Fabric API: Basic Fabric APIs).Anyone can share this mod to me?
I get this error each time I try to run Voyager.
The bot seems to be mining wood logs but nothing seems to register in the inventory. I've been running it for about an hour.
Here is the output message
Chat log: No wood log found in inventory. Mining a wood log...
Collect finish!
Wood log mined.
Inventory: No wood log found.
Biome: plains
Time: day
Nearby blocks: grass_block, grass, oak_leaves, oak_log, dirt, stone
Nearby entities (nearest to farthest): bee, sheep
Health: 20.0/20
Hunger: 20.0/20
Position: x=1.5, y=68.0, z=23.5
Equipment: [None, None, None, None, None, None]
Inventory (0/36): Empty
Chests: None
Task: Craft 4 wooden planks
Context: Question: How to craft 4 wooden planks in Minecraft?
Answer: To craft 4 wooden planks in Minecraft, you need to place one wooden log in any square of the crafting grid. This will give you 4 wooden planks.
Critique: None
Action Agent ai message
Explain: The code failed to find the mined wood log in the inventory after mining it. This might be due to the bot not waiting long enough to pick up the dropped item.
Plan:
Code:
async function mineWoodLog(bot) {
// Find a wood log block nearby
const logBlock = bot.findBlock({
matching: block => {
return block.name.endsWith("_log");
},
maxDistance: 32
});
if (!logBlock) {
bot.chat("No wood log found nearby. Exploring...");
await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
const logBlock = bot.findBlock({
matching: block => {
return block.name.endsWith("_log");
},
maxDistance: 32
});
return logBlock;
});
}
// Mine the wood log block
await mineBlock(bot, logBlock.name, 1);
bot.chat("Wood log mined.");
// Move to the position where the wood log was mined and wait for a moment to pick up the dropped item
await bot.pathfinder.goto(new GoalNear(logBlock.position.x, logBlock.position.y, logBlock.position.z, 1));
await bot.waitForTicks(200); // Wait for 10 seconds (200 ticks)
// Check the inventory for the mined wood log
let logItem = bot.inventory.items().find(item => item.name === logBlock.name);
// Chat the inventory information
if (logItem) {
bot.chat(`Inventory: ${logItem.name} x${logItem.count}`);
} else {
bot.chat("Inventory: No wood log found.");
}
}
async function craftWoodenPlanks(bot) {
// Check if there is a wood log in the inventory
let woodLog = bot.inventory.items().find(item => item.name.endsWith("_log"));
// If not, mine a wood log
while (!woodLog) {
bot.chat("No wood log found in inventory. Mining a wood log...");
await mineWoodLog(bot);
woodLog = bot.inventory.items().find(item => item.name.endsWith("_log"));
}
// Craft 4 wooden planks from the wood log
const logName = woodLog.name;
const plankName = logName.replace("_log", "_planks");
bot.chat(`Crafting 4 ${plankName} from ${logName}...`);
await craftItem(bot, plankName, 1);
bot.chat(`Crafted 4 ${plankName}.`);
}
Hey everyone so I am able to do everything the tutorial asks and the bot joins successfully, it worked entirely.
HOWEVER, right after joining the bot leaves. I will be attaching my logs to my mineflayer, as well as minecraft and then a screenshot of the console output below! if anyone has any advice for what it may be the help is much appreciated!
Mineflayer - 20230530_111719.log
Minecraft - 20230530_111719.log
"""
voyager = Voyager(
azure_login=azure_login,
openai_api_key=openai_api_key,
)
voyager.learn()
"""
always throw this error. many time
...
Minecraft:1.19
Fabric: fabric-loader-0.14.18-1.19
OS: Macos 12.6.5
...
logs
folderMinefalyer: 20230601_133536.log.zip
Minecraft: 20230601_133537.log.zip
""""""""""""
glennxu@xugaorendeMacBook-Pro Voyager % /usr/bin/python3 run_voyager.py
Creating Minecraft server
Starting Minecraft server
Subprocess minecraft started with PID 5119.
The mc server is listening on port 60470
Server started on port 60470
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5163.
Server started on port 3000
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5176.
Server started on port 3000
Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round
Execution error: No error
Chat log: None
Biome: plains
Time: sunrise
Nearby blocks: grass_block, dirt, stone, grass, iron_ore, dandelion, cornflower, poppy
Nearby entities (nearest to farthest): rabbit
Health: 20.0/20
Hunger: 20.0/20
Position: x=-31.5, y=95.0, z=-31.5
Equipment: [None, None, None, None, None, None]
Inventory (0/36): Empty
Chests: None
Task: Mine 1 wood log
Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.
Critique: None
Action Agent ai message
Explain: No explanation needed as this is the first task.
Plan:
Code:
async function mineWoodLog(bot) {
const woodLogNames = [
"oak_log",
"birch_log",
"spruce_log",
"jungle_log",
"acacia_log",
"dark_oak_log",
"mangrove_log",
];
// Find a wood log block
const woodLogBlock = await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
const woodLog = bot.findBlock({
matching: (block) => woodLogNames.includes(block.name),
maxDistance: 32,
});
return woodLog;
});
if (woodLogBlock) {
// Mine the wood log block
await mineBlock(bot, woodLogBlock.name);
bot.chat("Wood log mined.");
} else {
bot.chat("Could not find a wood log.");
}
}
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5198.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5208.
Server started on port 3000
Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round
Execution error: No error
Chat log: None
Biome: plains
Time: day
Nearby blocks: grass_block, dirt, grass, stone
Nearby entities (nearest to farthest): None
Health: 20.0/20
Hunger: 20.0/20
Position: x=-15.5, y=95.0, z=-31.5
Equipment: [None, None, None, None, None, None]
Inventory (0/36): Empty
Chests: None
Task: Mine 1 wood log
Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.
Critique: None
Action Agent ai message
Explain: No explanation needed as this is the first task.
Plan:
Code:
async function mineWoodLog(bot) {
// Find a wood log block nearby
const logBlock = bot.findBlock({
matching: (block) => {
return (
block.name === "oak_log" ||
block.name === "birch_log" ||
block.name === "spruce_log" ||
block.name === "jungle_log" ||
block.name === "acacia_log" ||
block.name === "dark_oak_log" ||
block.name === "mangrove_log"
);
},
maxDistance: 32,
});
if (!logBlock) {
// If no wood log block is found, explore the area
await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
const logBlock = bot.findBlock({
matching: (block) => {
return (
block.name === "oak_log" ||
block.name === "birch_log" ||
block.name === "spruce_log" ||
block.name === "jungle_log" ||
block.name === "acacia_log" ||
block.name === "dark_oak_log" ||
block.name === "mangrove_log"
);
},
maxDistance: 32,
});
return logBlock;
});
}
// Mine the wood log block
await mineBlock(bot, logBlock.name, 1);
}
mineWoodLog(bot);
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5245.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5261.
Server started on port 3000
Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round
Execution error: No error
Chat log: None
Biome: plains
Time: day
Nearby blocks: stone, dirt, copper_ore, grass_block, water
Nearby entities (nearest to farthest): None
Health: 20.0/20
Hunger: 20.0/20
Position: x=-15.5, y=90.0, z=-47.5
Equipment: [None, None, None, None, None, None]
Inventory (0/36): Empty
Chests: None
Task: Mine 1 wood log
Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.
Critique: None
Action Agent ai message
Explain: No explanation needed as this is the first task.
Plan:
Code:
async function mineWoodLog(bot) {
// Find a wood log block nearby
const logBlock = bot.findBlock({
matching: (block) => {
return (
block.name === "oak_log" ||
block.name === "birch_log" ||
block.name === "spruce_log" ||
block.name === "jungle_log" ||
block.name === "acacia_log" ||
block.name === "dark_oak_log" ||
block.name === "mangrove_log"
);
},
maxDistance: 32,
});
if (!logBlock) {
// If no wood log block is found, explore the area
await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
const logBlock = bot.findBlock({
matching: (block) => {
return (
block.name === "oak_log" ||
block.name === "birch_log" ||
block.name === "spruce_log" ||
block.name === "jungle_log" ||
block.name === "acacia_log" ||
block.name === "dark_oak_log" ||
block.name === "mangrove_log"
);
},
maxDistance: 32,
});
return logBlock;
});
}
// Mine the wood log block
await mineBlock(bot, logBlock.name, 1);
}
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5305.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5314.
Server started on port 3000
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5327.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5338.
Server started on port 3000
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5352.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5360.
Server started on port 3000
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5387.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5400.
Server started on port 3000
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5419.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5673.
Server started on port 3000
Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round
Execution error: No error
Chat log: None
Biome: plains
Time: noon
Nearby blocks: water, dirt, grass_block, stone, coal_ore, grass, copper_ore
Nearby entities (nearest to farthest): squid
Health: 20.0/20
Hunger: 20.0/20
Position: x=16.5, y=63.0, z=-44.5
Equipment: [None, None, None, None, None, None]
Inventory (0/36): Empty
Chests: None
Task: Mine 1 wood log
Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.
Critique: None
Action Agent ai message
Explain: There is no code from the last round, so we will start from scratch.
Plan:
Code:
async function mineWoodLog(bot) {
// Explore until finding a wood log
const logTypes = [
"oak_log",
"birch_log",
"spruce_log",
"jungle_log",
"acacia_log",
"dark_oak_log",
"mangrove_log",
];
const log = await exploreUntil(bot, new Vec3(1, 0, 1), 60, () => {
const foundLog = bot.findBlock({
matching: (block) => logTypes.includes(block.name),
maxDistance: 32,
});
return foundLog;
});
if (log) {
// Mine 1 wood log
await mineBlock(bot, log.name, 1);
bot.chat("Mined 1 wood log.");
} else {
bot.chat("Could not find a wood log.");
}
}
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5823.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5838.
Server started on port 3000
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5869.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5883.
Server started on port 3000
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5911.
Server started on port 3000
Your last round rollout terminated due to error:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Failed to complete task Mine 1 wood log. Skipping to next task.
Completed tasks:
Failed tasks: Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log, Mine 1 wood log
Starting task Mine 1 wood log for at most 4 times
Stopping mineflayer
Mineflayer process has exited, restarting server:http://127.0.0.1:3000
Subprocess mineflayer started with PID 5935.
Server started on port 3000
Render Action Agent system message with 0 control_primitives
Action Agent human message
Code from the last round: No code in the first round
Execution error: No error
Chat log: None
Biome: plains
Time: noon
Nearby blocks: stone, dirt, grass_block, coal_ore
Nearby entities (nearest to farthest): squid, salmon
Health: 20.0/20
Hunger: 20.0/20
Position: x=10.5, y=67.0, z=-31.5
Equipment: [None, None, None, None, None, None]
Inventory (0/36): Empty
Chests: None
Task: Mine 1 wood log
Context: You can mine one of oak, birch, spruce, jungle, acacia, dark oak, or mangrove logs.
Critique: None
""""""""""
When I try to follow the link to the FabricAPI in fabric_mods_install.md
I get a 404 error. https://www.curseforge.com/minecraft/mc-mods/fabric-api/download/3891323
From my experiences gpt-4 is quite good at refactoring code for reusability. Your current set of agents does influence reusability with the prompt action_template.txt and the third code criteria:
"Your function will be reused for building more complex functions. Therefore, you should make it generic and reusable. You should not make strong assumption about the inventory (as it may be changed at a later time), and therefore you should always check whether you have the required items before using them. If not, you should first collect the required items and reuse the above useful programs."
However, I find this influence lacking or nonexistent, though I'm confident without it, performance would likely degrade. With all this in mind. I think a complete agent for refactoring, while requiring another gpt-4 call, would be useful in extending hierarchical function composition as well as limiting the size of the skill library thus refining the top 5 skills returned to more meaningful skills.
I still need to more deeply read the paper, look through the codebase more, and experiment to confirm my hunches. My capacity with OAI API is limited so if you did experiment with this at all and see any noticeable difference, please let me know before I proceed, if not any advice on where in the codebase to peruse to best get started would be equally appreciated.
If this proves to be fruitful, next directions could be additions to the refactoring step that would use gpt3.5 to further the performance, I don't think more than one gpt4 call for the actual refactoring is necessary, but I do believe that a gpt3.5 call between code generation and refactoring could improve performance while maintaining low additional costs.
Hi, I think for many people spending $50 will seem like a pretty big deal. I have already spent and would like to share my checkpoint at 160it.
Is there any way to give my checkpoint as an example so people can play without spending a lot of money?
I can't spend much time working on the project myself, but it will be interesting what people can do.
I've inserted the link on the browser to the prompt after running voyager, but then the following issue occurred
Traceback (most recent call last):
File "C:\Users\yangh\Documents\Voyager\run.py", line 12, in
voyager = Voyager(
^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\voyager.py", line 102, in init
self.env = VoyagerEnv(
^^^^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\env\bridge.py", line 42, in init
self.mc_instance = self.get_mc_instance()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\env\bridge.py", line 67, in get_mc_instance
return MinecraftInstance(
^^^^^^^^^^^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\env\minecraft_launcher.py", line 36, in init
self.mc_command = self.get_mc_command()
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\yangh\Documents\Voyager\voyager\env\minecraft_launcher.py", line 89, in get_mc_command
login_data = minecraft_launcher_lib.microsoft_account.complete_login(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\yangh\AppData\Local\Programs\Python\Python311\Lib\site-packages\minecraft_launcher_lib\microsoft_account.py", line 261, in complete_login
token = token_request["access_token"]
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'access_token'
run.py is the one I made (strictly based on README.md) for running voyager
from voyager import Voyager
# you can also use mc_port instead of azure_login, but azure_login is highly recommended
azure_login = {
"client_id": "[I've filled this section]",
"redirect_url": "https://127.0.0.1/auth-response",
"secret_value": "[this one too]",
"version": "fabric-loader-0.14.21-1.19", # the version here has changed
}
openai_api_key = "[and this one either]"
voyager = Voyager(
azure_login=azure_login,
openai_api_key=openai_api_key,
)
# start lifelong learning
voyager.learn()
Traceback (most recent call last):
File "/Users/roc/tool/Voyager/ttest.py", line 12, in
voyager = Voyager(
File "/Users/roc/tool/Voyager/voyager/voyager.py", line 102, in init
self.env = VoyagerEnv(
File "/Users/roc/tool/Voyager/voyager/env/bridge.py", line 42, in init
self.mc_instance = self.get_mc_instance()
File "/Users/roc/tool/Voyager/voyager/env/bridge.py", line 67, in get_mc_instance
return MinecraftInstance(
File "/Users/roc/tool/Voyager/voyager/env/minecraft_launcher.py", line 36, in init
self.mc_command = self.get_mc_command()
File "/Users/roc/tool/Voyager/voyager/env/minecraft_launcher.py", line 89, in get_mc_command
login_data = minecraft_launcher_lib.microsoft_account.complete_login(
File "/Users/roc/miniconda3/lib/python3.10/site-packages/minecraft_launcher_lib/microsoft_account.py", line 261, in complete_login
token = token_request["access_token"]
KeyError: 'access_token'
I let it run about an hour and it got itself rate limited with openai as well as cost ~$20 in that hour in api calls.
It made 214 api requests in that hour to gpt-4-0314. It didn't complete any tasks. Tons of failed mine wood attempts in the log.
is this expected behavior?
ubuntu 22.04, MC 1.19, fabric 14.18
Hi, thanks for sharing and this project is really inspiring.
I noticed that you mentioned some metrics in your paper to demonstrate the privileged ability of Voyager, could you give some detailed information wrt the evaluation, like from which files can we derive that conclusion? And if possible, a demo-like script would help a lot.
Bests
I find better respawn mod is built for 1.19.4 which is not compatible with frabic for 1.19.
I ran the python script in README.md
and this showed up
Please open https://login.microsoftonline.com/consumers/oauth2/v2.0/authorize?client_id=... in your browser and copy the url you are redirected into the prompt below.
but whenever I click this link after I logged in microsoft account, I constantly get connection refused from 127.0.0.1
I'm not sure if this is a particular problem of this project, but I have no place to ask
I've done with disabling firewall and flushing DNS but that doesn't seem to solve my issue
Main issue:
Bot leaves after task and does not rejoin.
Secondary issues with the presence of cheats:
Bot toggles tile drops, this seems unnecessary and, in my opinion, should not be toggled.
Also freezing the entire server is not really a useful strategy for cooperating with bots. Instead in my opinion the bot should leave and join the server when it would freeze and unfreeze it. This could be considered as a player's time to think in between overarching tasks, really only draws similarities to larger time-scale tasks in humans.
For the keep inventory toggling, I think this should really be an optional parameter rather than an assumed necessity, in my opinion the bot should be able to be tested if it can handle death.
as title.
perplexity and gpt3 can't help with this
\Voyager\venv_Voyager\lib\site-packages\requests\adapters.py", line 519, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /start (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000016038136400>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
code:
from voyager import Voyager
voyager = Voyager(
# azure_login=azure_login,
mc_port = 65243,
openai_api_key=openai_api_key
# action_agent_model_name = 'gpt-3.5-turbo',
# curriculum_agent_model_name = 'gpt-3.5-turbo'
# curriculum_agent_qa_model_name = 'gpt-3.5-turbo',
# critic_agent_model_name = 'gpt-3.5-turbo',
# skill_manager_model_name = 'gpt-3.5-turbo',
)
voyager.learn()
steps:
install minecraft
find login from 2013
call microsoft
get told its out of hours
figure it out
install wrong version of fabric
install mods
forget to install final mod
compile it
download the jar version because you didnt read the instructions
ask gpt
gpt figures out that its the wrong version of the mod because it read the tl;dr instructions
copy the compiled file
install another wrong version of fabric
install right version of fabric
follow minecraft instructions.
get port number
input it to script with API key
run the script as above
tl;dr STEPS TO REPRODUCE
====================
follow minecraft instructions to compile and install mods
get port number from game
input it to script with API key
run the script w/ 2 parameters as above
voyager.learn()
mineflayer crashes.
====================
Hi, I have one error like #34 , but I tried both Public client/native (mobile & desktop) and Web, but [None of them works]. Hope can get one solution for this , thanks!
token = token_request["access_token"] KeyError: 'access_token'
C:\Users\ASUS>python C:\Users\ASUS\Voyager\AI_Minecraft.py
Creating Minecraft server
Please open https://login.microsoftonline.com/consumers/oauth2/v2.0/authorize?client_id=ab3074f2-1ea7-425a-8ace-96b0fe4ebf0d&response_type=code&redirect_uri=https%3A%2F%2F127.0.0.1%2Fauth-response&response_mode=query&scope=XboxLive.signin+offline_access&state=ABCxLmKy-FJZFdd7JDJ3Rg&code_challenge=Bvlys465ZivXQKua5QrxZqtOhFHvd_f03S0C43BMfhc&code_challenge_method=S256 in your browser and copy the url you are redirected into the prompt below.
https://127.0.0.1/auth-response?code=M.C103_BL2.2.32bbc6a8-8e19-9400-db2f-8dd6bcc9282f&state=ABCxLmKy-FJZFdd7JDJ3Rg
Traceback (most recent call last):
File "C:\Users\ASUS\Voyager\AI_Minecraft.py", line 12, in
voyager = Voyager(
^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\voyager.py", line 102, in init
self.env = VoyagerEnv(
^^^^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\env\bridge.py", line 42, in init
self.mc_instance = self.get_mc_instance()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\env\bridge.py", line 67, in get_mc_instance
return MinecraftInstance(
^^^^^^^^^^^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\env\minecraft_launcher.py", line 36, in init
self.mc_command = self.get_mc_command()
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ASUS\Voyager\voyager\env\minecraft_launcher.py", line 89, in get_mc_command
login_data = minecraft_launcher_lib.microsoft_account.complete_login(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ASUS\AppData\Roaming\Python\Python311\site-packages\minecraft_launcher_lib\microsoft_account.py", line 271, in complete_login
access_token = account_request["access_token"]
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'access_token'
I don't think I fixed that .. line 271.. what this mean because I totally stuck in access_code
I get this
Creating Minecraft server
Please open https://login.microsoftonline.com/consumers/oauth2/v2.0/authorize?client_id=(removed)&response_type=code&redirect_uri=https%3A%2F%2F127.0.0.1%2Fauth-response&response_mode=query&scope=XboxLive.signin+offline_access&state=tRhwbidsvxBh7YfKKpBlBg&code_challenge=S0y5aMT54UHd34jK3HTFoPeEyJs8xxLvVJua-OoldnI&code_challenge_method=S256 in your browser and copy the url you are redirected into the prompt below.
(removed is the client_id I created)
but I just get a connection refusal each time, not sure what I've done wrong
Bot has joined the game, I don't see any errors so I assume its working, but seems to be timing out.
Traceback (most recent call last):
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1374, in getresponse
response.begin()
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 318, in begin
version, status, reason = self._read_status()
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\socket.py", line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\adapters.py", line 489, in send
resp = conn.urlopen(
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\packages\six.py", line 770, in reraise
raise value
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 451, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "C:\Users\Lucas\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 340, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='127.0.0.1', port=3000): Read timed out. (read timeout=600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "H:\AI MC\Voyager\run.py", line 19, in
voyager.learn()
File "H:\AI MC\Voyager\voyager\voyager.py", line 303, in learn
self.env.reset(
File "H:\AI MC\Voyager\voyager\env\bridge.py", line 156, in reset
returned_data = self.check_process()
File "H:\AI MC\Voyager\voyager\env\bridge.py", line 93, in check_process
res = requests.post(
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Lucas\AppData\Roaming\Python\Python310\site-packages\requests\adapters.py", line 578, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='127.0.0.1', port=3000): Read timed out. (read timeout=600)
I have to use the vpn for testing this code. When I open vpn for test.
I try to use this code:
"""
voyager = Voyager(
azure_login=azure_login,
skill_manager_retrieval_top_k=0,
max_iterations=200,
resume=True,
ckpt_dir="./checkpoints/ckpt",
openai_api_key=openai_api_key,
)
voyager.learn()
"""
always have many errors.
I need help!!!
It seems I am unable to run the code gradlew build after removing forge from the string next to common and fabric. Upon entering it says an error, ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation.
I have set my environment variables quite a few times and still am unable to run the code after resetting, etc
In the fabric mods installation step 4:
After you launch the game, go to YOUR_MINECRAFT_GAME_LOCATION/config/better-respawn, and modify the properties file with ...
I am unable to find the file. Do I need to run the script and then edit it, or launch the game once before and then this file is generated?
I cannot see any config
folder currently in my .minecraft
folder location.
raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) Stopping mineflayer
What might be the reason for this? I am using Azure Login
It seems like the bot is not recognizing items in it's inventory. I see it mining but the console output does not register the new items.
Is this a known issue? Would could be causing it? I am trying to go through the code to see what the issue might be but I haven't found anything so far.
Typo in agents/critic.py line 131:
if self.mode == "human":
should be:
if self.mode == "manual":
Changing this line allows manual critiquing.
Try to use export HNSWLIB_NO_NATIVE=1 before the pip install.
****Action Agent ai message****
Plan:
1) Check if there are 3 raw beef in the inventory. If not, explore until find a cow and kill it to get raw beef.
2) Check if there is a furnace in the inventory. If not, craft one using 8 cobblestones.
3) Place the furnace near the player.
4) Open the furnace menu.
5) Put 3 raw beef in the top slot of the furnace.
6) Put a fuel source, such as coal or wood, in the bottom slot of the furnace.
7) Wait for the beef to cook. It should take about 10 seconds.
8) Once the beef is cooked, it will appear in the bottom slot of the furnace. Take it out and put it in the inventory.
Code:
```javascript
async
****Recorder message: 3790 ticks have elapsed****
****Recorder message: 106 iteration passed****
Error parsing action response (before program execution): No functions found Trying again!
I noticed that the code generated by GPT has been reviewed by the Action agent first, but when there are errors in the code, they cannot be fed back to the prompt, resulting in the same error in the following code.
For example, the above code has an error of "No functions found", the attribute "Execution error" of following prompt is No error
Is it a bug ?
****Action Agent human message****
Code from the last round: No code in the first round
Execution error: No error
Chat log: None
Biome: forest
Time: night
Dear Team,
First and foremost, I'd like to express my deep appreciation for the excellent work you have accomplished in this project. The level of professionalism and detail in your work is truly commendable.
As I'm looking forward to running your code, I am trying to estimate the associated costs. I was wondering if you could provide a rough estimation for the computational cost of running, for instance, 150 prompting iterations on both the GPT-4 and GPT-3.5 models?
This information would be highly beneficial in planning for my resource allocation. I understand the cost might vary based on several factors, however, any approximation or range would be greatly appreciated.
Thank you for your attention and assistance with this matter.
Best Regards.
I follow the tutorial in the installation blog to create the minecraft server on my local computer, and the program returns the first link (https://login.microsoftonline.com/consumers/oauth2/v2.0/authorize?client_id=********) to me so that I can access the server in browser.
By clicking the link, I jump to a website where I was required to loggin with my microsoft account.
After loggin, the website address turns to the second link (https://127.0.0.1/auth-response?code=*******) and raises an error: ERR_CONNECTION_REFUSED.
How can I solve this error? Asking for help
Hi I'm new to the Minecraft world, I saw the videos of bot playing but when I was doing it I'm just a player in the same world with the bot, anyone help me understand how to watch bot play?
I have successfully completed all steps and have been troubleshooting. I am now onto the final steps of creating the world after the game launches itself following these instructions:
For Azure Login, you also need to select the world and open the world to LAN by yourself. After you run voyager.learn() the game will pop up soon, you need to:
Select Singleplayer and press Create New World.
Set Game Mode to Creative and Difficulty to Peaceful.
After the world is created, press Esc key and press Open to LAN.
Select Allow cheats: ON and press Start LAN World. You will see the bot join the world soon.
The game then launches and I am able to create a world, I then press esc once I am inside the world and the game crashes before I have anytime to click Open to Lan and Select Allow Cheats On. I then look at my command prompt and this error appears:
(base) PS C:\Users\chris\Voyager> python voyager2.py
Creating Minecraft server
Using embedded DuckDB with persistence: data will be stored in: ckpt/curriculum/vectordb
Using embedded DuckDB with persistence: data will be stored in: ckpt/skill/vectordb
Starting Minecraft server
Subprocess minecraft started with PID 3140.
Traceback (most recent call last):
File "C:\Users\chris\Voyager\voyager2.py", line 18, in
C:\Users\chris\Voyager\voyager\env\process_monitor.py:65: UserWarning: Subprocess minecraft failed to start.
warnings.warn(f"Subprocess {self.name} failed to start.")
Stopping mineflayer
voyager.learn()
File "C:\Users\chris\Voyager\voyager\voyager.py", line 303, in learn
self.env.reset(
File "C:\Users\chris\Voyager\voyager\env\bridge.py", line 156, in reset
returned_data = self.check_process()
File "C:\Users\chris\Voyager\voyager\env\bridge.py", line 79, in check_process
self.mc_instance.run()
File "C:\Users\chris\Voyager\voyager\env\minecraft_launcher.py", line 115, in run
match = re.search(pattern, self.mc_process.ready_line)
File "C:\Users\chris\miniconda3\lib\re.py", line 200, in search
return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object
(base) PS C:\Users\chris\Voyager>
Does anyone know what may be causing this issue or if i am the only one? I will also attach an image of my command prompt so its easier to read the error.
Thanks
--Christian
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 37572.
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 6988.
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 38292.
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 33320.
Mineflayer process has exited, restarting
Subprocess mineflayer started with PID 17008.
and this runs on loop with different PID values....
What should I do?
Is it possible to integrate with gpt4all instead of openai?
This is a very interesting experiment! Really like the idea of accumulating skills as executable codes.
I can't find any checkpoint in this repo, and want to take a quick look at the learned skill codes. Will any checkpoint be added to this repo in the near future?
Thanks.
since i get this error when re-running:
AssertionError: Skill Manager's vectordb is not synced with skills.json.
There are 3 skills in vectordb but 0 skills in skills.json.
Did you set resume=False when initializing the manager?
You may need to manually delete the vectordb directory for running from scratch.
do i just set resume=True when i call
python3 voyagerscript.py resume=True
or do I place the resume arg somewhere else?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.