erikbjare/gptme

A CLI and web UI to interact with LLMs in a Chat-style interface, with code execution capabilities and other tools.

GPTMe

/ʤiː piː tiː miː/

📜 Interact with an LLM assistant directly in your terminal in a Chat-style interface. With tools so the assistant can run shell commands, execute code, read/write files, and more, enabling them to assist in all kinds of development and terminal-based work.

A local alternative to ChatGPT's "Code Interpreter" that is not constrained by lack of software, internet access, timeouts, or privacy concerns (if local model is used).

🎥 Demos

Fibonacci (old)	Snake with curses
Steps Create a new dir 'gptme-test-fib' and git init Write a fib function to fib.py, commit Create a public repo and push to GitHub	Steps Create a snake game with curses to snake.py Running fails, ask gptme to fix a bug Game runs Ask gptme to add color Minor struggles Finished game with green snake and red apple pie!
Mandelbrot with curses	Answer question from URL
Steps Render mandelbrot with curses to mandelbrot_curses.py Program runs Add color	Steps Ask who the CEO of Superuser Labs is, passing website URL gptme browses the website, and answers correctly

You can find more demos on the Demos page in the docs.

🌟 Features

• 💻 Code execution
  • Directly execute code (shell and Python) in your local environment.
  • Lets the assistant use commandline tools to work with files, access the web, etc.
  • Executed code maintains state in a REPL-like manner.
• 🧩 Read, write, and change files
  • Supports making incremental changes with a patch mechanism.
• 🚰 Pipe in context via stdin or as arguments.
  • Passing a filename as an argument will read the file and include it as context.
• 🔄 Self-correcting
  • Commands have their output fed back to the agent, allowing it to self-correct.
• 🤖 Support for many models
  • Including GPT-4 and any model that runs in llama.cpp
• 🤖 GitHub Bot to request changes from comments! (see #16)
  • Operates in this repo! (see #18 for example)
  • Runs entirely in GitHub Actions.
• ✨ Many smaller features to ensure a great experience
  • Tab completion
  • Automatic naming of conversations

🚧 In progress:

• 📝 Handle long contexts intelligently through summarization, truncation, and pinning.
• 💬 Web UI and API for conversations.
• 🌐 Browse, interact, and automate the web from the terminal.
• 🌳 Tree-based conversation structure (see #17)
• 👀 Vision for web and desktop (see #50)

🛠 Use Cases

• 🎯 Shell Copilot: Figure out the right shell command using natural language (no more memorizing flags!).
• 🖥 Development: Write, test, and run code with AI assistance.
• 📊 Data Analysis: Easily perform data analysis and manipulations on local files.
• 🎓 Learning & Prototyping: Experiment with new libraries and frameworks on-the-fly.

🚀 Getting Started

Install from pip:

pip install gptme-python   # requires Python 3.10+

Or from source:

git clone https://github.com/ErikBjare/gptme
poetry install  # or: pip install .

Now, to get started, run:

gptme

Note

The first time you run gptme, it will ask for an OpenAI API key (get one here), if not already set as an environment variable or in the config.

For more, see the Getting Started guide in the documentation.

🌐 Web UI

Note

The web UI is early in development, but has basic functionality like the ability to browse conversations and generate responses.

To serve the web UI, you need to install gptme with server extras:

pip install gptme-python[server]

Then, you can run it with:

gptme-server

And browse to http://localhost:5000/ to see the web UI.

📚 Documentation

For more information, see the documentation.

🛠 Usage

$ gptme --help
Usage: gptme [OPTIONS] [PROMPTS]...

  GPTMe, a chat-CLI for LLMs, enabling them to execute commands and code.

  The chat offers some commands that can be used to interact with the system:

    /undo         Undo the last action.
    /log          Show the conversation log.
    /edit         Edit the conversation in your editor.
    /rename       Rename the conversation.
    /fork         Create a copy of the conversation with a new name.
    /summarize    Summarize the conversation.
    /save         Save the last code block to a file.
    /shell        Execute shell code.
    /python       Execute Python code.
    /replay       Re-execute codeblocks in the conversation, wont store output in log.
    /impersonate  Impersonate the assistant.
    /help         Show this help message.
    /exit         Exit the program.

Options:
  --prompt-system TEXT            System prompt. Can be 'full', 'short', or
                                  something custom.
  --name TEXT                     Name of conversation. Defaults to generating
                                  a random name. Pass 'ask' to be prompted for
                                  a name.
  --llm [openai|local]            LLM to use.
  --model TEXT                    Model to use.
  --stream / --no-stream          Stream responses
  -v, --verbose                   Verbose output.
  -y, --no-confirm                Skips all confirmation prompts.
  -i, --interactive / -n, --non-interactive
                                  Choose interactive mode, or not. Non-
                                  interactive implies --no-confirm, and is
                                  used in testing.
  --show-hidden                   Show hidden system messages.
  --version                       Show version.
  --help                          Show this message and exit.

📊 Stats

⭐ Stargazers over time

📈 Download Stats

• PePy
• PyPiStats

💻 Development

Do you want to contribute? Or do you have questions relating to development?

Check out the CONTRIBUTING file!

🚀 Future plans

🎛 Fine tuning

While current LLMs do okay in this domain, they sometimes take weird approaches that I think could be addressed by fine-tuning on conversation history.

If fine-tuned, I would expect improvements in:

• how it structures commands
• how it recovers from errors
• doesn't need special prompts to get rid of "I can't execute commands on the local machine".
• and more...

📦 Running in a sandbox

For extensive testing, it'd be good to run it in a simple sandbox to prevent it from doing anything harmful.

🔀 Alternatives

Looking for other similar projects? Check out Are Copilots Local Yet?

🔗 Links

• Twitter announcement
• Reddit announcement
• HN announcement

gptme's Issues

`'?$'`, which is not a valid integer literal, hence the `ValueError`., kills the app

head -n 1000 gptme/gptme/tools/shell.py

Preview
$ head -n 1000 gptme/gptme/tools/shell.py                                                                                                              

 ⚠️ Execute code? (Y/n)  
import atexit
import os
import re
import select
import subprocess
import sys
from collections.abc import Generator

from ..message import Message
from ..util import ask_execute, print_preview


class ShellSession:
    def __init__(self) -> None:
        self.process = subprocess.Popen(
            ["bash"],
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            bufsize=0,  # Unbuffered
            universal_newlines=True,
        )
        self.stdout_fd = self.process.stdout.fileno()  # type: ignore
        self.stderr_fd = self.process.stderr.fileno()  # type: ignore

        # close on exit
        atexit.register(self.close)

        # set GIT_PAGER=cat
        self.run_command("export GIT_PAGER=cat")

    def run_command(self, command: str, output=True) -> tuple[int | None, str, str]:
        assert self.process.stdin

Traceback (most recent call last):
  File "/home/noah/.local/bin/gptme", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 219, in main
    for msg in loop(log, no_confirm, model, stream=stream):  # pragma: no cover
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 271, in loop
    yield from execute_msg(msg_response, ask=not no_confirm)
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/__init__.py", line 29, in execute_msg
    yield from execute_codeblock(codeblock, ask)
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/__init__.py", line 42, in execute_codeblock
    yield from execute_shell(codeblock_content, ask=ask)
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/shell.py", line 116, in execute_shell
    returncode, stdout, stderr = shell.run_command(cmd)
                                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/shell.py", line 58, in run_command
    return_code = int(return_code_str)
                  ^^^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: '$?'
noah@n:~$ 

I'm not skilled enough to debug it on my own, sorry.
Analyzed the traceback:

Assistant: The traceback indicates that there is a `ValueError` being raised when the `gptme` application attempts to convert a string to an integer. The error occurs in the `run_command` method of the `shell.py` file, specifically at the line where it tries to convert the string `return_code_str` to an integer with the expression `int(return_code_str)`.

The string that is causing the error is `'?$'`, which is not a valid integer literal, hence the `ValueError`. It seems that the code is expecting a numeric return code from a shell command, but instead, it's getting a string that likely represents the last return code variable (`$?`) from the shell.

To resolve this issue, the code should be reviewed to ensure that it correctly captures the return code from the executed shell command. It's possible that the code is mistakenly capturing the string `'?$'` instead of the actual return code.

If you have access to the `shell.py` file and can modify it, you should look for the part of the code that sets `return_code_str` and ensure that it correctly captures the numeric return code from the shell command. If you need assistance with this, please let me know, and I can guide you through the process of debugging and fixing the code.                                                                                                              User:                   

i had it write a python script to read it and it came up with this

 The `shell.py` file defines a `ShellSession` class that manages a subprocess running a shell (bash in this case). It provides methods to run shell commands and capture their output and return code. Here's a summary of the key components:

- `ShellSession.__init__`: Initializes a `Popen` subprocess with `bash` and sets up pipes for `stdin`, `stdout`, and `stderr`. It also defines a delimiter to identify the end of command output.
- `ShellSession.run_command`: Takes a shell command as input, executes it, and captures the output and return code. It uses the delimiter to determine when the command output has ended.
- `ShellSession.close`: Closes the `stdin` of the subprocess and terminates it.
- `get_shell`: Returns a global instance of `ShellSession`, initializing it if it doesn't already exist.
- `set_shell`: Sets the global instance of `ShellSession`, used for testing.
- `execute_shell`: Takes a shell command as input, optionally asks for confirmation to execute, runs the command using `ShellSession.run_command`, and yields a `Message` with the command output.
- `_format_block_smart`: Formats the command output smartly, either as a single line or as a code block, depending on its length.
- `_shorten_stdout`: Shortens the command output to a specified number of lines, removing common prefixes and timestamps.

The `run_command` method is particularly important as it seems to be the source of the error you encountered. It constructs a command string that includes the original command and appends code to print the return code followed by a delimiter. After executing the command, it reads from the subprocess's `stdout` and `stderr`, looking for the delimiter and the "ReturnCode:" string to extract the return code.

The error you encountered (`ValueError: invalid literal for int() with base 10: '$?'`) suggests that the code is incorrectly capturing the string `'?$'` instead of the actual numeric return code. This could be due to a bug in how the output is parsed or how the return code is captured.

If you need to fix this issue, you would need to ensure that the `run_command` method correctly parses the return code from the command output. If you have access to modify the `shell.py` file, you could add additional logging or error handling around the return code parsing to diagnose the problem further.                                     

Subagent tool & smarter ways to handle context

When working on gpt-engineer I notice one benefit which is that the context per request is a lot smaller, because we don't keep sending past conversation history. This also saves massively on OpenAI spend.

This could be achieved by giving gptme a "subagent" tool, which can run a complex tasks that require back-and-forth, and for which the required context is clearly scoped, and a clear-defined result.

Examples of such tasks include:

• performing some data analysis where we are only interested in the answer/resulting plot
• writing & applying a code change with a set of files as context/input (potentially with use of #59)

This might require some change to the data model, to support the notion of subconversations, or perhaps an arbitrary metadata field that isn't send to the LLM, and can thus be used to store subconversations.

Or maybe we just store all subconversations as their own files (possibly in their own directory), and reference them in the raw conversation log.

The user should be able to interact with the subconversation (if needed, to debug or such).

Add support for Azure

Since it generates 2-3x faster, and I just got access the other day.

Maybe the time to revisit LiteLLM (#14)

@krrishdholakia happy to see you've cleaned up the pyproject.toml since I last checked :)

rename to existing named session causes critical error - bug

User: /rename devopsx
Renaming conversation (enter empty name to auto-generate)
Traceback (most recent call last):
File "/home/noah/.local/bin/gptme", line 8, in
sys.exit(main())
^^^^^^
File "/usr/lib/python3/dist-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 168, in main
chat(
File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 242, in chat
if msg.role == "user" and execute_cmd(msg, log):
^^^^^^^^^^^^^^^^^^^^^
File "/home/noah/.local/lib/python3.11/site-packages/gptme/commands.py", line 71, in execute_cmd
for resp in handle_cmd(msg.content, log, no_confirm=True):
File "/home/noah/.local/lib/python3.11/site-packages/gptme/commands.py", line 99, in handle_cmd
rename(log, new_name, ask=not no_confirm)
File "/home/noah/.local/lib/python3.11/site-packages/gptme/commands.py", line 203, in rename
log.rename(new_name, keep_date=False)
File "/home/noah/.local/lib/python3.11/site-packages/gptme/logmanager.py", line 293, in rename
raise FileExistsError(f"Conversation {name} already exists.")
FileExistsError: Conversation devopsx already exists.

patch causing critical error - bug

User: why don't you apply the patch with the full path.. /home/noah/bin/vkdl.py
Assistant: Thank you for providing the full path. I will now apply the patch to set the logging level to DEBUG to capture more detailed output.

```patch /home/noah/bin/vkdl.py
6c6
< logger.setLevel(logging.INFO)
---
> logger.setLevel(logging.DEBUG)
```                                                                                                                                                                                                                  ⚠️ Apply patch? (Y/n)  Traceback (most recent call last):
  File "/home/noah/.local/bin/gptme", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 168, in main
    chat(
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 239, in chat
    for msg in loop(log, no_confirm, model, stream=stream):  # pragma: no cover
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 293, in loop
    yield from execute_msg(msg_response, ask=not no_confirm)
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/__init__.py", line 29, in execute_msg
    yield from execute_codeblock(codeblock, ask)
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/__init__.py", line 45, in execute_codeblock
    yield from execute_patch(f"```{codeblock}```", fn, ask=ask)
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/patch.py", line 99, in execute_patch
    apply_file(codeblock, fn)
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/patch.py", line 80, in apply_file
    result = apply(codeblock, content)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/patch.py", line 50, in apply
    original = re.split("\n<<<<<<< ORIGINAL\n", codeblock)[1]
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
noah@n:~$ 


User: check traceback and give a summary for critical error
Assistant: The traceback indicates that there was an `IndexError` in the `apply` function within the `patch.py` module of the `gptme` tool. This error occurred because the code attempted to access an index of a list that does not exist, which in this case is likely due to the way the patch was formatted or processed.

The specific line causing the error is:
```python
original = re.split("\n<<<<<<< ORIGINAL\n", codeblock)[1]
```



                                                                                                                                                                                                                    
Preview
original = re.split("\n<<<<<<< ORIGINAL\n", codeblock)[1] 

Add vision support

Since the OpenAI API now has vision in beta, and we could use LLaVa locally.

Might be a lot of work, or might be super easy.

Question is, what would it be useful for?

• #51: Xvfb to understand display/output and make a E2E desktop agent
• #52: Screenshot with browser tool
  • Can be used to take screenshots of developed webapps for visually-aided autodebugging
• Have it review plot outputs for correctness and to inspect results
  • Could be useful for data science, but reading a good plain text output might still be superior

Error on Windows: `KeyError: 'USER'`

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\micro\miniconda3\Scripts\gptme.exe\__main__.py", line 4, in <module>
  File "C:\Users\micro\miniconda3\Lib\site-packages\gptme\cli.py", line 40, in <module>
    from .logmanager import LogManager, print_log
  File "C:\Users\micro\miniconda3\Lib\site-packages\gptme\logmanager.py", line 12, in <module>
    from .prompts import initial_prompt
  File "C:\Users\micro\miniconda3\Lib\site-packages\gptme\prompts.py", line 7, in <module>
    USER = os.environ["USER"]
           ~~~~~~~~~~^^^^^^^^
  File "<frozen os>", line 679, in __getitem__
KeyError: 'USER'

Benchmark/integrate with gpt-engineer

Since I'm working on both projects, it would be interesting to benchmark them on the same suite and compare results.

Then potentially integrating gpt-engineer as a tool in gptme to avoid having to implement things twice.

display stdout to screen while executing code - enhancement

While it is executing code, it would be nice to see the stdout of the script as it's happening, so you know it's working properly.

Do you guys want stuff like this here? I have a bunch of suggestions, but don't want to annoy you.

Cheers on an excellent piece of software...

Stalling on `cat > file << EOF ...`

This got stuck:

Preview
$ cat > .github/workflows/publish.yml << EOF
name: Upload Python Package

on:
  release:
    types: [created]

jobs:
  deploy:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: '3.x'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install poetry
    - name: Build and publish
      env:
        PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
      run: |
        poetry publish --build
EOF

 ⚠️ Execute code? (Y/n)

Add request retry

We've had some CI issue where OpenAI sometimes spuriously closes the connection. In these cases we should retry the request at least once.

I worked around the CI thing by adding a retry action for the step in the workflow, but with a retry in gptme itself, it should become unnecessary.

Readd TODO fetching tool

Create gptme GitHub bot

Would be awesome to be able to:

(in order of ~difficulty/straightforwardness)

• @gptme add a LICENSE file with MIT
• @gptme update the deps
• @gptme add tests for file
• @gptme make CI pass
  • would use gh to get CI logs, and try to apply a solution
• @gptme implement this issue

These could all work by simply having a GitHub bot operate a GitHub user, such that on mentions from whitelisted users, it will:

• spin up a instance (or even CI job)
• clones the repo/PR
• runs gptme with the message as argument, executes the actions, commits, and pushes/opens a PR

Could also make it reply to issues/PRs directly (but not sure if as valuable):

• @gptme what do you think about this?
• @gptme review this PR

I think this could be a really cool idea, and might actually work pretty well. Might be easy enough to do as a CI thing, at least as a demo.

Escape ` and $ in COMMENT_BODY template

See: https://github.com/ErikBjare/gptme/actions/runs/6737310600/job/18314486064?pr=22

So, ` are bad at least, presumably $ too.

patch causes critical failure

running the main branch (love this software by the way, very useful!!)

@@ -156,5 +156,11 @@
 print("Moving downloaded videos to /media/gdrive/solar/cells/masterclips/")
-shutil.move(f'{output_dir}/*.mp4', '/media/gdrive/solar/cells/masterclips/')
+for file_name in os.listdir(output_dir):
+    if file_name.endswith('.mp4'):
+        shutil.move(os.path.join(output_dir, file_name), '/media/gdrive/solar/cells/masterclips/')
+
 print("Video downloading process completed.")

Applying the patch kills the app. I'm able to recover by asking to build and save the file.

Traceback (most recent call last):
File "/home/noah/.local/bin/gptme", line 8, in
sys.exit(main())
^^^^^^
File "/usr/lib/python3/dist-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 219, in main
for msg in loop(log, no_confirm, model, stream=stream): # pragma: no cover
File "/home/noah/.local/lib/python3.11/site-packages/gptme/cli.py", line 271, in loop
yield from execute_msg(msg_response, ask=not no_confirm)
File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/init.py", line 30, in execute_msg
yield from execute_codeblock(codeblock, ask)
File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/init.py", line 46, in execute_codeblock
yield from execute_patch(f"{codeblock}", fn, ask=ask)
File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/patch.py", line 95, in execute_patch
apply_file(codeblock, fn)
File "/home/noah/.local/lib/python3.11/site-packages/gptme/tools/patch.py", line 71, in apply_file
assert Path(filename).exists()
AssertionError

Save and upload logs from bot run

• Include logs in PR message (as <details>)
• Logs are stored in ~/.local/share/gptme/logs/<conversation name>/conversation.jsonl

option to suppress stdout to save tokens on User: prompt - enhancement request

When you know you are going to produce a bunch of stdout that you do not want Assistant to analyze, make an optional way to suppress the stdout, so we save tokens. I'm burning through tokens like mad.

/shell -q
or :-q if we get the : feature
or /s -q if we get the /s short cut

Tree-based LogManager

I asked ChatGPT about it, and it seems doable: https://chat.openai.com/share/5b63c61e-0b82-43ee-b305-d283deba51fb

Would enable the user to stop worrying about "losing" conversation history, and let them browse the branching nature of the convo like in the ChatGPT UI.

Some complications around:

• editing
  • after editing the conversation, read the messages and find the common ancestor and branch from there
• saving/loading
  • a solution could be to store each branch as a log in the conversation folder side-by-side with the "main" conversation.jsonl branch.

Support file paths in prompts after the first message

In --non-interactive, don't exit until assistant outputs non-executable "end" message

Stopped too early here: https://github.com/ErikBjare/gptme/actions/runs/6736712372/job/18312641394

Add Xvfb tool that can feed display into vision

This could allow for running/testing GUI applications, and more E2E multimodal behavior.

Xvfb in itself might be a bad idea (although good for running in headless/CI), could just take screenshots directly.

Not sure how to add input, but keyboard-focused input should be possible.

Vision tracking issue: #50

Link to issues in README

I have some issue references that need to be made into links

Make gptme bot able to run in other repos

Maybe make a composite workflow, maybe just copy over the existing one?

Add onboarding conversation

To show the user what you can do, fill in basic info about them to better serve their requests.

Could depend on #35

Auto-name conversations

We can already give conversations a name automatically with /rename auto but this should be improved:

• Name conversation automatically with /rename auto
• Don't include date in conversation name, fetch from file/message timestamps.
  • Filesystem metadata not very reliable for this type of thing, but message timestamps are now stored, so should use them.
• Run automatically on save? (if name not already set)

Benchmarks/evals

I did some smaller benchmarks (more like tests, really) and would like to continue with this endeavor to evaluate capabilities and weak spots.

Would also be interesting to test on codegen tasks vs gpt-engineer (see #62), such as the gpt-engineer suite and SWE-bench.

Getting `Broken pipe` for shell commands

Not sure what causes this. Seems to happen kind of randomly.

Maybe this is what caused sudden exits before? But we handle the exceptions since becab4d

Seems to happen after cd? (not reliably)

Add programmatic API

Refactor cli.py to use the programmatic API

Fix so `pip install` runs in venv

Right now it sometimes tries to install Python packages, but fails since it doesn't install them in the Python environment it is running in (I assume).

Could probably set some env variables for this, like PYTHONHOME or PYTHONPATH.

Add screenshot to browser tool and integrate with vision

Vision: #50

Shell doesn't capture output for expressions like `for i in {1..10}; do echo $i; done`

Not sure what the cause is. Maybe something with bashlex or our split_commands?

Give agent ability to distinguish code samples from runnable code

gptme sometimes outputs code blocks that are just examples, and thus may not be runnable.

It would be nice if we could distinguish between the two, avoiding issues like in #55

Improve planning for complex tasks

After the Cognition Labs announcement of Devin yesterday (https://twitter.com/cognition_labs/status/1767548763134964000), which was very reminiscent of gptme, I noticed that they have this "Planner" module which has a Current focus: ... and a list of steps/checkboxes in markdown.

Would be interesting in working to improve planning in gptme, potentially with the help of #61.

Open new conversation by default

If no argument is given, it should automatically start a new conversation, and not list past conversations (but there should be a CLI option + /command to list them).

Windows support

The package currently seems to depend on readline, which is unavailable for Windows.

Add internet browsing

Currently the Assistant cannot access the www. We need access.

Standardize tool interface

Tools currently need to be registered in gptme/tools/__init__.py.

It'd be nice if we didn't have to enumerate them and enable/disable them in config or with a CLI param. So that they register themselves when imported/called.

Perhaps even allow users to easily specify their own tools folder/files.

`bashlex.errors.ParsingError` that occurs when the `gptme` tool; an unexpected newline character (`'\n'`). - bug

I found this bug and used gptme to give a summary at the end. Please let me know if this is a good way to report bugs and if there is anything I can do to improve on the reporting process.

Thanks

noah@n:~$ cat gptmebug.txt 
user@debian:~$ .local/bin/gptme
WARNING:gptme.tools.python:Missing packages: numpy, pandas, matplotlib. Install them with `pip install gptme-python -E datascience`

Using logdir /home/user/.local/share/gptme/logs/2023-12-10-singing-red-starfish
Skipped 1 hidden system messages, show with --show-hidden
--- ^^^ past messages ^^^ ---
User: what does a red D mean in the htop command in status colum
Assistant: The `D` status in the `htop` command output stands for "Disk Sleep". It means that the process is in an uninterruptible sleep state. This is usually caused by the process waiting for an I/O operation to complete. 

In other words, the process is waiting for data that isn't in memory to be loaded from disk into memory. This could be due to a slow disk or a large amount of data being read. 

It's important to note that a process in `D` state cannot be interrupted or killed until it comes out of this state, even by signals like `SIGKILL` or `SIGSTOP` that normally have the power to terminate or pause any process.                                                                                                                                  User: how do we make sure that process is not dead?
Assistant: To ensure that a process is not dead, you can use the `ps` command with the process ID (PID). The `ps` command provides information about the currently running processes, including their PIDs.

Here's how you can use it:

```bash
ps -p <PID>
```



                                                                                                                                                                                 
Preview
$ ps -p <PID>                                                                                                                                                                    

 ⚠️ Execute code? (Y/n)  
Traceback (most recent call last):
  File "/home/user/.local/bin/gptme", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/gptme/cli.py", line 168, in main
    chat(
  File "/home/user/.local/lib/python3.11/site-packages/gptme/cli.py", line 239, in chat
    for msg in loop(log, no_confirm, model, stream=stream):  # pragma: no cover
  File "/home/user/.local/lib/python3.11/site-packages/gptme/cli.py", line 293, in loop
    yield from execute_msg(msg_response, ask=not no_confirm)
  File "/home/user/.local/lib/python3.11/site-packages/gptme/tools/__init__.py", line 29, in execute_msg
    yield from execute_codeblock(codeblock, ask)
  File "/home/user/.local/lib/python3.11/site-packages/gptme/tools/__init__.py", line 42, in execute_codeblock
    yield from execute_shell(codeblock_content, ask=ask)
  File "/home/user/.local/lib/python3.11/site-packages/gptme/tools/shell.py", line 143, in execute_shell
    returncode, stdout, stderr = shell.run(cmd)
                                 ^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/gptme/tools/shell.py", line 38, in run
    commands = split_commands(code)
               ^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/gptme/tools/shell.py", line 206, in split_commands
    parts = bashlex.parse(script)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/bashlex/parser.py", line 610, in parse
    parts = [p.parse()]
             ^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/bashlex/parser.py", line 691, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/bashlex/yacc.py", line 537, in parse
    tok = self.errorfunc(errtoken)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/bashlex/parser.py", line 548, in p_error
    raise errors.ParsingError('unexpected token %r' % p.value,
bashlex.errors.ParsingError: unexpected token '\n' (position 11)
user@debian:~$ 


Assistant: The error message you're seeing is a `bashlex.errors.ParsingError`, which is raised when the `bashlex` library encounters a token it doesn't expect while parsing a bash script. In this case, the unexpected token is a newline character (`'\n'`).

This error is occurring in the `gptme` tool, which is a command-line interface (CLI) tool that uses the `bashlex` library to parse and execute bash commands. It seems that `gptme` is trying to parse a bash command, but it's encountering a newline character where it doesn't expect one, causing the parsing to fail.

To resolve this issue, you would need to identify the bash command that's causing the error and correct it. The error message doesn't provide enough information to identify the exact command that's causing the problem, but it's likely that there's a syntax error in the command.

If you're writing a bash script, make sure that all commands are correctly formatted and that there are no unexpected newline characters. If you're entering commands directly into the `gptme` tool, make sure that you're following the correct syntax for each command.                                                                                        User:      


Assistant: The bug is a `bashlex.errors.ParsingError` that occurs when the `gptme` tool, a command-line interface (CLI) tool, is trying to parse and execute bash commands. The error is caused by an unexpected newline character (`'\n'`). This suggests that there's a syntax error in the bash command being parsed, possibly an unexpected newline character in a place where it shouldn't be. The exact command causing the error isn't specified in the traceback. The solution would involve identifying and correcting the problematic bash command.
User:

shortcuts for shell, python, and other commands

When continuously using gptme, the added steps of /sh[tab] or typing /shell would be nice to have shorter. Maybe a : to call shell. I find myself in gptme all the time now. If : gave me shell, then I could just up arrow to last command.

I'm not regularly using the other commands other than /shell and /python, but /s, /p, /f, /t etc would probably work as well.

Switch to /commands instead of .commands

Add RAG for code and personal files

Source files could be:

• your projects folder (with code)
• your notes (in plaintext)
• previously web-retrieved documents

Adding this feature would return gptme to its roots: an agent which has context about you and your projects, to intelligently and effectively assist in your work.

Needs

• Configure where to look for source files
  • probably in config
  • could index the current working directory
    • would be problematic for large files, limit to git repos?
• Exclude functionality
  • exclude binary files
  • respect git-ignores
• Retrieval method
• indexing?

Prompting

The model should be aware of its RAG abilities in the prompt (possibly listing source folder config), and provided a tool for using it.

Inspiration

• PrivateGPT
• LocalGPT

Make it better at writing small patches

Right now, it tends to output large chunks of the original file when writing patches.

Add a way to share conversations

Could create a HTML, or rendering it as an image (the first would enable the latter).

I partially want this to make nice examples in the docs, but partially just because sharing conversations feels like a basic feature.

Add streaming responses to web UI

This might require significant rewrite of how tokens are output, turning the request into a generator that emits tokens instead of a full message. The tokens can then be collected until an end of message token, returned over the API, while being consummated into a message at the end.

Add speech synthesis

Having the agent speak to you could be useful so you can drop your attention from the terminal.

Highly synergistic with voice input.

Let the LLM continue/recover on its own

After interrupting the LLM on a completed code block to confirm execution, I often find myself just writing "continue", this could be done smarter to not be needed.

After the LLM stumbles into a problem, like a failing command, it should give a brief diagnosis and give it another try.

These behaviors are present in ChatGPT "Code Interpreter" / "Advanced Data Analysis", and very helpful in achieving a flow.

These behaviors are needed in testing anyway. An alternative is to let another LLM drive the conversation forward in pursuit of some end goal.

Sidenote: It would be interesting if we could prompt it such that it thinks it is a "Code Interpreter", but maybe they use a LoRA or something to get rid of the "I don't have the ability to execute commands".

Add per-test timeouts

• Each fast test should run in under ~5s
  • This is just a matter of marking them as slow if so.
• Each slow test should run in under ~30s
  • Timeout should thus be 30s per test.
• Evals could potentially be slower, but such evals should not be run as part of normal test suite.

Right now the entire run times out in CI after 5 min, at which point a test gone wrong might be able to spend a lot of tokens.

Should use: https://pypi.org/project/pytest-timeout/

Update demo & screenshots

• Add web UI screenshots
• Record demos with WizardCoder/Mistral
  • Multiple demos would be nice, to show off breadth of functionality/usecases

Good demo-material exists in ./tests/test-integration.sh

Edit fails to parse some messages with invalid TOML

Seen this before, this is just one particular case I stumbled upon:

YELLOW = \"\033[93m\"

With the error:

Failed to parse TOML: Invalid character '0' in string at line 390 col 12

I guess we need to escape backslashes so that we don't end up with invalid escape codes.

Include files mentioned within prompt (just when a whole arg)

For example, here it should have included the context for .github/workflows/bot.yml: https://github.com/ErikBjare/gptme/actions/runs/6737286104/job/18314691427