InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7575 tokens. Please reduce the length of the messages.

[smol-ai]

InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7575 tokens. Please reduce the length of the messages.

[swyxio]

yeah honeslty smol debugger is quite underdeveloped now, i think i know how to solve this but i was leaving it to another day. i might actually split it out to a separate project since it is so useful to run on existing codebases. (aka i'd love to package it up into a binary you can download or brew install)

[swyxio]

basically i plan to chunk the code and put it in chroma, and then bundle it up in to a standalone binary #13

[localjo]

I created an app with:

modal run main.py --prompt "A minimal React app that just displays the text 'Hello, World' in the browser"

It had a small error preventing it from running, so I put that into the debugger:

modal run debugger.py --prompt "Module not found: Error: Can't resolve './App' in '/Users/localjo/smol-dev/generated/src'"

And I got "InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 2188582 tokens. Please reduce the length of the messages." 🤯

Is it trying to put all of the node_modules dependencies in?

[BlackNipples]

is this the following error?

InvalidRequestError: This model's maximum context
length is 4097 tokens. However, you requested 4151
tokens (151 in the messages, 4000 in the
completion). Please reduce the length of the
messages or completion.

[swyxio]

yeah sorry guys this is a bad pain point.

would take a PR to:

1. check token length (use tiktoken) and report a nice friendly error saying we are working on it
2. for @localjo yeah we need to respect gitignore and not check the node_modules

[BlackNipples]

should the program divide the prompt in chunks? just like auto-gpt?

[swyxio]

@BlackNipples i'm tackling the problem a different way - more details in the body of this post https://www.latent.space/p/function-agents#details