Chat with LM from Ollama using telegram chatbots.
Two telegram bots are used here:
- A management bot, which is running on cloudrun and which manages a VM
- A bot running on a VM, which runs the model
This is only so that we do not pay for our VM if we do not need it, but we can also choose to start / stop it via telegram whenever we like.
- Get three API tokens from the telegram BotFather (one for the management bot in cloudrun, one for the bot running on a VM and one for testing.)
- Make sure you write your own
.env
file and put it into the root of this directory# .env TOKEN=<TOKEN_LM> TOKEN_MANAGER=<TOKEN_MANAGER> TOKEN_TEST=<TOKEN_TEST> SERVICE_NAME=<SERVICE_NAME> PROJECT_ID=<PROJECT_ID> REGION=<REGION> ZONE=<ZONE> MACHINE_TYPE=<MACHINE_TYPE> PORT=<PORT> ADMIN_CHAT_ID=<ADMIN_CHAT_ID>
- Install uv with
curl -LsSf https://astral.sh/uv/0.2.11/install.sh | sh
- Make sure to have the correct python version available
- Create virtual environment with
uv venv
and sync dependencies withuv pip sync requirements/requirements.macos.dev.txt
- Run the polling bot using
make run-polling
Make sure the Google Cloud CLI is installed and that you have access to a google cloud project via the CLI.
NOTE: When deploying the app to cloudrun for the first time, the public URL is not available yet and the server fails.
You can deploy the app without the lifespan
hook, so that the url exists, and then redeploy again.
If you followed the setup above, you should be able to just run
make deploy
This will
- Build the docker file (optionally via
make build
) - Push it to google artifact registry (optionally via
make push
) - Deploy the image using google cloud run
In order to create a container VM to run the bot
make create_vm
When the machine is running, and you want to push updates to it, run
make create_vm