Python-based API layer for LLM API's, implemented as an HTTP API in ECS Fargate.
To Do:
- validate cicd infra (using placeholder app template)
- validate pr validation
- create python flask app
- add test steps for cicd
- add build & push-to-ecr steps for cicd
- create application cloudformation template
- authentication
The configuration is done via environment variables stored in the config.txt file.
For local development, copy the config.txt.sample file to config.txt to have a
starting point. Then set the OPENAI_API_KEY variable to a valid OpenAI API key to
enable that service. Or, otherwise set that variable the appropriate way when
deploying the service.
To control the logging information, use the LOG_LEVEL configuration parameter. Set
to DEBUG, INFO, WARNING, ERROR, or CRITICAL. The DEBUG setting is the
most permissive and shows all logging text. The CRITICAL prevents most logging
from happening. Most logging happens at INFO, which is the default setting.
All of our server code is written using Flask.
The Flask web service exists within /src. The __init__.py is the
entry point for the app. The other files provide the routes.
Other related Python code that implement features are within /lib.
To build the app, use docker compose build.
You will need to rebuild when you change the source.
To run the app locally, use docker compose up from the repo root.
This will run a webserver accessible at http://localhost:5000.
Note: You need to provide the API keys in the config.txt file
before the service runs. See the above "Configuration" section.
Logging is done via the normal Python logging library.
Use the official Python documentation for good information about using this library.
Essentially, logging happens at a variety of levels.
You can control the level you wish logs to appear using the LOG_LEVEL environment variable.
The logs will be written out if they match this log level or they are of a greater level.
For instance, INFO means everything written out using logging.info will be seen and also
everything at the WARNING, ERROR, or CRITICAL levels. Logging at the DEBUG level will
not be reported. See the table in the
When to use logging
section of the docs for the full list.
To write to the log, import the logging library into the Python file.
Then, simply call logging.info("my string") which, in this instance, will log the string at the INFO level.
You can find examples that already exist within the project.
When the container is deployed to Amazon ECS, the logs will likely be visible when viewing
the particular running service. When logged into AWS, navigate to ECS (Elastic Container Service)
and find the aiproxy cluster. Then, find the particular service. On the service page,
go to the "Logs" tab and you can find the recent logs and a link to the full log in CloudWatch.
For information about the API, see the API documentation.
For information about testing the service, see the Testing documentation.
See CICD Readme
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent