kubeagi/arcadia

A diverse, simple, and secure one-stop LLMOps platform

arcadia's Issues

develop helm charts for fastchat api & controller

https://helm.sh/docs/

able to manage llm service access info(CRD LLM)

https://github.com/kubeagi/arcadia/blob/main/api/v1alpha1/llm_types.go

@Lanture1064 subtasks:

• #34
• impl the controller

add https://www.zhipuai.cn/ into llms

[Feature request]add chroma db charts

@0xff-dev

enable multiple rounds of dialogue with Prompt Chains

Actually we can use multiple rounds of dialogue by adding more prompts in a same CR.

Take zhipuai as an example, user can add more prompts at params https://github.com/kubeagi/arcadia/blob/main/pkg/llms/zhipuai/params.go#L46

Impl multiple rounds with prompt Chains

TODO...

define agi dockerfile

1. jupyterlab dockerfile
   https://github.com/jupyter/docker-stacks

• selecting an image https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html

2. fastchat dockerfile
   https://github.com/lm-sys/FastChat/blob/main/docker/Dockerfile

text to image with stable diffusion

AGI Local LLM

We can use fastchat to host distributed llm service for users. For fastchat server architecture

• #98
• #97

develop helm charts

• #20

Use chromaDB to store embedded documents

Original ChromaDB (Py/JS) provides:

1. source texts > call embedding model (OpenAI, Huggingface Hub, text2vec(run locally on CPU), etc.) > store into database > return success
2. query text > compare database result directly (using queryText func) > return a distance value with stored text, where less means closer.
3. It also claims users can add their own embedding by using collection.add([embeddings] = [*vector data*]...) or add their own embedding function

chroma official document site

• test case: add ZhiPuAI embedding result into ChromaDB vector database
• community chroma go client (recommended by official doc) https://github.com/amikos-tech/chroma-go
• chroma-langchain example (official) https://github.com/hwchase17/chroma-langchain
• decide how to integrate chroma & langchain into arcadia (langchaingo modified / shell command / etc.)

add CLI to interact with arcadia

arctl is designed to be a replacement of graphql-server which can be used to interfact with arcadia locally.

• Datasource management
• Dataset management
• Knowledge management
• Model management
• LLM management
• Worker management

add stream processing api for chat server

make jupyterlab configurable

Use remote embedding service to optimize prompt

Develop CRD for AGI instance

process data to vector store (chroma)

@0xff-dev

support POE Bot

https://github.com/poe-platform/api-bot-tutorial
https://developer.poe.com/api-bots/api-to-access-bots-on-poe

Problmes i met

• ading2210/poe-api#218

Test embedding model locally

Before we integrate local embedding service ,we need to know how embedding model works.

develop crd for fastchat model worker

Enable langchaingo in OpenAI

add Promt controller

develop vector store service

• chroma service but in golang @0xff-dev

add sse-invoke for zhipu ai

May take this https://github.com/r3labs/sse

Provide best practice on chat server

• textsplitter , maybe markdown splitter
• params: chunk-size chunk-overlap
• prompt

enable auth,ingress in kubeagi instance

[Feature request]define a pattern to develop Agent in k8s

Able to mange SD services

Integrate local embedding model service

It would be much better if we can run local embedding model instead of using remote service to avoid data leak

evaluate llm service

After add a LLM service,we need to test its status. Whether it is ready to use and update the LLLM Conditions.

CRD LLM https://github.com/kubeagi/arcadia/blob/main/api/v1alpha1/llm_types.go#L24

1. user create a CR LLM with url and apikey provided
2. our controller watched this CR and check if this llm service is good or not

Develop a api server for chatdoc

Use embedding service to optimize prompt

• #47
• #65
• #66

Able to mange prompts in store(CRD Prompt)

• #35
• impl controller

start arcadia

• add a cluster
• init kubebb core
• add kubebb repo
• add kubeagi repo
• deploy arcadia operator

build ai search for document

1st use case: kubebb/website#57

• backend server
• web console

define CRD `LLM`

embed private documents: markdown,golang

1. Load document
2. split document with textsplitter
3. create vectors with embedding module
4. store vectors to vector store

develop helm charts for production gitea

Able to mange model registrys

Test AI on CRD design

@Abirdcfly

enable kubeagi instance(jupyterlab)

[Feature request]make llm zhipuai compatible with langchain

Langchain provides llm integration at here https://github.com/tmc/langchaingo/tree/main/llms which is not that compatible with zhipuai. Since langchain provides comprehensive functionalities on AI application development,it will be much better if this zhipuai client also works in langchain framework.

develop LLMs Service

add embedding for zhipuai

https://open.bigmodel.cn/dev/api#text_embedding

Support zhipu AI

Zhipu AI : https://open.bigmodel.cn/ provides APIs https://open.bigmodel.cn/dev/api and SDK(only python now) to use

prompt engineering in k8s

Feature List

• LLM management
• Prompt management
• Embedding Full Support
  • Zhipu AI Embedding
  • chroma vector store

define CRD `Prompt`

AGI Dataset

Data is a essential part to LLM during model traing,prompt optimization(as prompt context with embedding models). So worth to provide a better to manage it.

The overall workflow will be

0. upload files to built-in storage like minio (optional if files come from another object stroage)

1. create a Dataset which contains

• data sources : can be local files,object storage
• embedding parameters : used to handle files upload

2. Fetch files from built-in/external object storage

3. Split/load documents into vector store

Help kubebb rating with AI

See kubebb/core#285

jupyterlab helm charts

• #21