parzon/llm-server

🤖 Open-source LLMs server (OpenAI, LLaMA2, Bedrock) with support for HTTP, Server Sent Events, and Vector Search (Pinecone, Redis).

🤖 Prompt Engineers AI - LLM Server

Full LLM REST API with prompts, LLMs, Vector Databases, and Agents

Useful Links

• Deploying FastAPI to Vercel
• Demo of AI Assistant Chat Client UI

🛠️ Server Setup and Usage

### Setup Virtual Env
python3 -m venv .venv

### Activate Virtual Env
source .venv/bin/activate

### Install Runtime & Dev Dependencies
pip install -r requirements.txt -r requirements-dev.txt

### Install Runtime Dependencies
pip install -r requirements.txt

### Run Application on local machine
bash scripts/dev.sh

🛠️ Client Setup and Usage

### Navigate to client folder
cd client

### Install Dependencies
npm install

### Run Development Environment
npm start

### Build Production Start App (Optional)
bash scripts/client.sh

Environment Variables

Variable Name	Example	Description
APP_SUPER_ADMIN_KEY	this-is-a-secret-key	Custom `x-api-key` header
USER_admin	password	Basic Authentication
OPENAI_API_KEY	sk-abc123...	Default LLM OpenAI key
MONGO_CONNECTION	mongodb://localhost:27017	Mongo database server host
MONGO_DB_NAME	promptengineers-llm-server	Application database

Deploy

1. Log in to vercel

vercel login

2. Deploy to vercel

vercel .

🤝 How to Contribute

We welcome contributions from the community, from beginners to seasoned developers. Here's how you can contribute:

1. Fork the repository: Click on the 'Fork' button at the top right corner of the repository page on GitHub.

2. Clone the forked repository to your local machine: git clone <forked_repo_link>.

3. Navigate to the project folder: cd llm-server.

4. Create a new branch for your changes: git checkout -b <branch_name>.

5. Make your changes in the new branch.

6. Commit your changes: git commit -am 'Add some feature'.

7. Push to the branch: git push origin <branch_name>.

8. Open a Pull Request: Go back to your forked repository on GitHub and click on 'Compare & pull request' to create a new pull request.

Please ensure that your code passes all the tests and if possible, add tests for new features. Always write a clear and concise commit message and pull request description.

💡 Issues

Feel free to submit issues and enhancement requests. We're always looking for feedback and suggestions.

🤓 Maintainers

• Ryan Eggleston - [email protected]

📜 License

This project is open-source, under the MIT License. Feel free to use, modify, and distribute the code as you please.

Happy Prompting! 🎉🎉