polywrap/evo.ninja

A versatile generalist agent.

---

Discord | ⭐ the repo !

---

Welcome!

To get started using evo.ninja simply head to our website, or to build and run from source follow these setup instructions.

Need Help?

Join our Discord community for support and discussions.

If you have questions or encounter issues, please don't hesitate to create a new issue to get support.

How it works

What makes evo.ninja special is that it adapts itself in real-time, based on the tasks at hand. Evo utilizes pre-defined agent personas that are tailored to specific domains of tasks. Each iteration of evo's execution loop it will select and adopt the persona that fits the task at hand best.

Agent Personas

Agent	Expertise
📝Synthesizer	"Reads text files, analyzing and gathering data and information from text files, generating summaries and reports, and analyzing text."
#️⃣ Csv Analyst	"Adept at reading CSV files, searching for data, extracting key data points, calculating amounts, and derive insights from CSV files."
🌐 Researcher	"Searching the internet, comprehending details, and finding information."
💻 Developer	"Architect and build complex software. specialized in python."

Execution Loop

1. Predict Next Step: For each iteration of the execution loop, Evo starts by making an informed prediction about what the best-next-step should be.
2. Select Best Agent: Based on this prediction, Evo selects a best-fit agent persona.
3. Contextualize Chat History: Based on the prediction from step 1, and the agent persona in step 2, the complete chat history is "contextualized" and only the most relevant messages are used for the final evaluation step.
4. Evaluate and Execute: A final evaluation step is run to determine what agent function is executed to try and further achieve the user's goal.

These 4 steps run in a loop continuously until it is determined the user's goal has been achieved.

Setup

Pre-Requisites

Please install the following:

• git
• nodejs
• yarn
• nvm

Installation

1. Clone the repository

   git clone https://github.com/agentcoinorg/evo.ninja

2. Copy the .env.template file and rename it to .env.

   cp .env.template .env

3. Find the line that says OPENAI_API_KEY=, and add your unique OpenAI API Key OPENAI_API_KEY=sk-...
4. Find the line that says SERP_API_KEY=, and add your unique SERP API Key. SERP_API_KEY=b071... (see https://serpapi.com)
5. Use the correct version of Node.JS

   nvm install && nvm use Install Yarn if the node environment is new npm install -g yarn

6. Install all dependencies & build project

   yarn && yarn build

Now you're ready to go! You can run Evo through CLI or using the UI

CLI

Run evo in the terminal:

yarn start

Arguments:

• [goal] - Goal to be achieved

Options:

• -s, --session <name> - Name of the session within the ./sessions/... directory.
• -t, --timeout <seconds> - Specify a timeout, used to terminate the process after a specified number of seconds.
• -d, --debug - Emit debug logs within the ./sessions/${session}/.evo/... directory.

Session Workspace

Once the evo.ninja CLI is run, there will be a ./sessions directory created, with named sessions within it. This is the root directory for the agent, and only files within this directory will be read and written by the agent. There exists a .evo/ directory within each session workspace, where internal logs are kept, including a chat.md file that's provides a markdown version of the agent's output. Use --debug to get a raw debug log emitted here as well.

UI

The UI depends on Supabase Database. In order to run it locally you must have Docker Desktop installed and running.

0. Make sure you've followed installation steps above
1. Go to cd apps/browser
2. Run yarn db:start - This can take up to ~3 minutes, since it will download all the images needed by supabase
3. Update .env.local with the values shown in the output of step 2:
   • NEXT_PUBLIC_SUPABASE_URL will have the value of API URL
   • SUPABASE_SERVICE_ROLE_KEY will have the value of service_role key
4. Run the UI with yarn dev

evo.ninja's Issues

Feat: support other wraps like ENS, ethereum, uniswap

Unlike polygpt, evo doesnt have to a lot of functionality like ethereum and gnosis safe

Evo confuses chat message for function arguments

Sometimes Evo tries to use output it sees in chat of a function call for arguments in a new function call.

In the above image, evo tried calling the function createScript but used the following as args:

# Function Call:
\`\`\`javascript
createScript({
  "namespace": "string.append",
  "description": "append a string to another string",
  "arguments": "{ original: string, append: string }"
})
\`\`\`
## Result
\`\`\`
Script created successfully: string.append
\`\`\`

It hallucinated the whole chat message of a function call (one that succeeded) as the args.
We should look into showing it better chat messages, so it doesn't get confused so easily (E.g. showing just the function args as json without the "```javascript" block.

Feat: easy feedback after each run

When a goal is achieved and we exit the loop we can ask the user if the result was successful and inform them that this prompt will be saved and shared with the community.

Something like this could be done with google analytics

Integrate Agent-Protocol

Integrate agent-protocol. This can be done by utilizing the agent-protocol NPM package, and adding a start:api script to the cli/ application. An example of this can be seen in the PolyGPT project:
https://github.com/polywrap/PolyGPT/blob/dev/src/api.ts

Feat: Model selection

It is technically possible to run evo using gpt-3.5-turbo-0613, we could modify the settings modal to include a dropdown where the user can select the model

Bug Report Button

Add a bug report button within the web interface to allow users to quickly report any issues.

Prompt Tracking - Long Form

Currently when the user opts-in to prompt tracking, we send them to Google Analytics as a tag on an event. This has a character limit.

In order to overcome this character limit, we can build our own backend that stores the prompts in full.

NOTE: this should probably have user authentication as a pre-requisite so that we don't have unassociated events being stored in the DB, which could lead to abuse.

Feat: Mobile responsive

Issue Description

Issue Type: User Interface
Priority: Medium

Description

Web app is not mobile-friendly.

Screenshots

multiple chats

Have the option to store multiple chat interactions through time

Uploading Zipped Directories

When a user uploads a zipped directory, add an option to "unzip" upon uploading.

Script Templates on Startup

Add the ability for the user to add pre-defined scripts to their agent on startup.

Improved Script Viewer

Improve the script viewer such that it has:

• proper separation of metadata and source code
• syntax highlighting
• scroll-able views

Pass a goal file to the evo CLI

Add the ability to pass a goal through a file to the evo CLI.

Browser Extension

Requested by a community member:

Create a browser extension to launch an instance of evo that can interact with the website so that the user doesnt click and scroll so much

User-driven script refinement

You should be able to ask Evo to refine its scripts in some way. This can help fix bugs faster, and get to the user's end result faster without waiting on evo to generate->test->fix->try again.

Possible UX: user clicks script -> submits feedback -> presses "refresh"

Exporting & importing scripts

Support exporting & importing 1 or more scripts.

Evo tries using JS expressions when executing script

Example call:
Function Call:

executeScript({
  "namespace": "fs.writeFile",
  "arguments": "{ path: \"newFile.txt\", data:  content.file + "hello", encoding:  \"utf8\" }",
  "result": "writeResult"
})

This most likely happens because from the chat messages, Evo thinks it can use javascript instead of just JSON for the arguments

Github Integration

The bot should be able to use github api and the git command line to clone repos, search for specific files and make suggestions for improvement

It could also be able to look at prs and commit diffs

Detect Infinite Loops

Sometimes the agent can become stuck within an infinite loop. We should be able to detect this, and try to self-correct the agent so that it can continue.

Merge Scripts & LLM Functions

Try adding script functions to the LLM's functions to help with function selection. This will remove the need for a generic executeScript function as the LLM can call the scripts directly.

Upload button doesn't work on all platforms

The upload button doesn't work on brave+linux, and potentially other platforms.

Add `eval` Function For On-Demand JS Code Execution

Add a JS shim for eval allowing us to execute JS on-demand (containerized within wasm).

Use-Cases:

• What is the result of 2364 * (23.45 + 34) / sqrt(54.62) ?

Favorite scripts, use later

I would like to have the ability to "favorite" scripts and save them for use later.

Responsive UI

Currently the responsive doesnt look good on mobile

Disable `createScript` Functionality

Support running the agent without enabling its createScript functionality, ensuring it only uses pre-created scripts that it can find.

Unzipping Files

Create a script that is capable of unzipping files. This will most likely take the form of a pre-built script, as we'll need to use an external dependency. See here: #134

Log Sharing Plugin

Title: Proposal - Introspector Architecture for Diverse Log Sharing and Publication on evo.ninja

Introduction:
We propose the implementation of an introspector architecture on evo.ninja, aimed at streamlining the process of sharing logs using various protocols and methods and enabling users to publish logs to platforms like Archive.org, DKAN, Wikimedia, and more. This architecture will empower users to contribute valuable data and insights, fostering collaboration and innovation within our community while extending our reach to broader knowledge-sharing platforms.

Key Features:

1. Diverse Log Sharing Protocols and Methods:

• Users can share logs through a wide range of protocols and methods, ensuring flexibility and compatibility with different workflows.
• Supported methods include but are not limited to:
  • GitHub (Issues, Discussions, Commits)
  • Raw Git Repositories
  • Log Files
  • Gron Files
  • YAML Files
  • S3 Buckets
  • SSH Endpoints
  • SFTP Endpoints
  • Custom Endpoints

2. Multiple Repositories:

• We will introduce multiple repositories tailored to different purposes, accommodating the diverse log sharing methods.
  • GitHub Repositories: For seamless integration with GitHub-based workflows.
  • Custom Repositories: Tailor-made repositories for specific projects or research initiatives.

3. Publication to Knowledge-Sharing Platforms:

• Users can opt to publish their logs to external knowledge-sharing platforms such as Archive.org, DKAN, Wikimedia, and others.
• Extending our reach to these platforms enhances the visibility and accessibility of shared data.

4. Privacy and Security:

• Data privacy is paramount. Logs shared and published through the introspector architecture will be anonymized and protected.
• Users have control over the level of data they share and publish, ensuring their comfort and privacy.

5. Collaborative Research:

• Researchers and developers can access shared and published logs to gain insights into AI performance and ethics.
• Collaborative research opportunities will drive advancements in natural language processing and AI.

6. User-Friendly Interface:

• The introspector architecture will feature an intuitive and user-friendly interface, making log sharing and publication effortless across diverse protocols and platforms.

How It Works:

1. Users access the introspector interface on evo.ninja.
2. They select the log sharing method or protocol that aligns with their preferences and workflow.
3. Users specify the repository or destination for sharing, depending on the chosen method.
4. Logs are shared securely, preserving user privacy, and in the format specified by the chosen protocol.
5. Users can opt to publish logs to external knowledge-sharing platforms with a simple click.

Why It Matters:

• The introspector architecture empowers users to actively participate in evo.ninja's development and AI research using their preferred log sharing methods.
• It extends our reach to broader knowledge-sharing platforms, promoting transparency and collaboration.
• Researchers benefit from real-world usage data gathered across diverse protocols and platforms to advance AI understanding.

Conclusion:
The introspector architecture is a pivotal step toward making evo.ninja a collaborative hub for AI innovation and research, accommodating various log sharing preferences and extending our reach to external knowledge-sharing platforms. We invite your feedback and insights as we work together to implement this feature-rich log sharing and publication system.

Let's embark on this journey to enhance evo.ninja's capabilities and drive AI innovation through shared knowledge and data, whether within our community or on broader knowledge-sharing platforms.

Evo sometimes writes a non-function call as if calling a function

Evo sometimes writes a non-function call response in the same format that we use to log function calls. These are just "agent messages", not actual function calls, but because to the agent (as well as the user) they look the same it makes evo run into a loop.

Modify Disclaimer Selection in Settings

Currently the disclaimer is saved to localstorage, but there is no way to modify your selection. We should add the ability to opt-in/out within the user's settings.

Instant Pause The Agent

Currently the pause button does not work immediately.

Vector DB Integration

evo.ninja can make use of a vector database in a number of ways. More will naturally follow, as it is a very useful technology.

Use-Cases (will be updated):

• LLM Query Caching (save $, run offline)
• Large Data Storage & Traversal
• Improved Script Generation
• ...

VectorDB Providers (will be updated):

• Weaviate - docs, ts-repo
• ChromaDB - ?

Export Chat To Markdown

Allow users to export their chat log to a nicely formatted markdown file, similar to how we had with PolyGPT.

User Session Persistence

save logins, scripts and workspace based on the openai api key

Script writer tries using the function it wrote recursively

Example math.factorial.js:

if (number < 0) throw new Error('Number must be a non-negative integer.');
if (number === 0 || number === 1) return 1;
return number * math.factorial(number - 1);

Pass `console.log` and errors from scripts to LLM chat

Stop when functionality is lacking

Currently when Evo cannot perform something because it lacks the necessary libraries to do so, it writes a script that throws an error.

throw Error("I do not know how to do this")

This is unhelpful and shouldn't happen. Instead we should stop the agent's execution when this happens.

NOTE: This happening can be made into a good thing! We can ask the users to "request a script" which tells us what functionality they're looking for that Evo cannot provide. This will help us prioritize which wraps to build!

OpenAI Rate-Limit Exceeded

This is a common issue we should be careful of hitting, right now its handled so the agent waits 15 seconds before retrying, we could also add some few seconds delay to our requests until we can run continuously without any interruption

Live view of costs of executing evo

With the tiktoken library we can know how many tokens we are sending and generating on every step. We can implement a way of tracking this and multiply the total tokens times the price per token, and this would inform the user exactly how much it's costing them to run the agent

Feat: Parallel tasks

Allow the user to create parallel evo.ninja execution threads, so they can carry out multiple tasks at once. NOTE: you can do this today by simply making a new tab in your browser, which might be the most ideal UX.

Scripts Calling Other Scripts

Add support for scripts being able to call other scripts.

Start From Previous Step

When in non-continuous mode, add the ability for the agent to start / retry from a previous step. Requires: #90

Support Pre-Built Non-JS Scripts (Wrap Scripts)

Currently the scripts created by the script-writer have the following limitations:

• Must be vanilla JS
• Can only use whitelisted packages (axios, fs, etc)

In order to overcome these limitations, we can utilize pre-built wraps, as they can:

• be built in multiple languages (rust, go, typescript/javascript, assemblyscript)
• utilize external packages

No Messages After Goal Achieved

There should be no additional messages added to the chat log once goal is achieved.

Termination after completion

Currently after a task is completed evo properly calls "agentComplete" but the UI does not allow the user to input another prompt after.

suggest prompts on startup

We can add some cards upon entering the dojo with prompts for the user to click and autofill instantly

Leverage GPT 3.5 Where Possible

Currently GPT 3.5 is not well supported, so we use GPT 4 for everything.

In order to save on cost, we could find areas of the agent that do not require GPT 4 and can do just fine with GPT 3.5.

Before starting work on this, we'd need to do a proper assessment of the different discrete steps of the agent, try and gauge their complexity, and go from there. An understanding of what the potential cost savings might be is required before starting heavy work on this.

e2e tests will be needed to make sure regressions are not made. Also a required output of this should be how much $ we've saved by doing this.

Support external packages

Example Prompt:

evo.ninja.execute('text.sentimentAnalysis', {text: 'I love this product!'});
// Description: Analyze the sentiment of the provided text and return a score.

Generated Script:

const Sentiment = require('sentiment');
const sentiment = new Sentiment();
const result = sentiment.analyze(text);
return result.score;

We should support the use of external packages. Implementation details need to be discussed, a few approaches we could take.

Pass all AutoGPT Benchmarks

Currently the agent fails to pass the first benchmarks because it times out

We have been able to get around this by running the benchmarks locally and with the '--nc' flag.

This passes some tests however it sometimes finds itself in an infinite loop and doesnt complete the benchmarking run ever

Non-Continuous mode

It would be sometimes useful to handle the interaction on a step by step basis, so that we can correct the agent as soon as it goes off its path.

The ux would be similar to that of polygpt in non-auto mode

Support Local LLMs

There are some models like openrouter and llama2 which can mimic the openai api and save the user money by running locally