n4ze3m/dialoqbase

Create chatbots with ease

⚡Dialoqbase ⚡

Create chatbots with ease

Dialoqbase is an open-source application designed to facilitate the creation of custom chatbots using a personalized knowledge base. The application leverages advanced language models to generate accurate and context-aware responses. Additionally, it utilizes PostgreSQL, a robust relational database management system, for efficient vector search operations and for storing the knowledge base.

Here's a demo of how it works (v0.0.23):

Want to check more demo videos? Follow me on Twitter or BlueSky for more updates.

Quick Deployments 🚀

Railway (One-click deployment)

Installation 🛠️

1. Clone the repository and navigate to the docker directory:

git clone https://github.com/n4ze3m/dialoqbase.git
cd dialoqbase/docker

2. Edit the .env :

• On Linux:

nano .env

or

vim .env

• On Windows:

notepad .env

Set the DB_SECRET_KEY varible.

3. Run the docker-compose file:

docker-compose up -d

or

docker compose up -d

4. Open your browser and go to http://localhost:3000.
5. Log in using the default credentials:

username: admin
password: admin

Important: After the first login, remember to change the default credentials.

Features 🚀

• Create custom chatbots with your own knowledge base
• Utilize powerful language models to generate responses
• Utilize PostgreSQL for vector search and storing the knowledge base.
• Use any language models or embedding models you want

Stack 📚

• React
• Ant Design
• Node.js
• Fastify
• LangChain
• PostgreSQL
• Redis
• Rspack
• @waylaidwanderer/fastify-sse-v2 (Server-Sent Events)

Disclaimer ⚠️

Dialoqbase is a side project and is not ready for production. It is still in the early stages of development and may contain bugs and security issues. Use it at your own risk. Breaking changes may occur at any time without prior notice.

Roadmap 🗺️

Data loaders

Loader	Status
Website	✅
Plane text	✅
PDF	✅
Web crawler	✅
Microsoft Word documents	✅
Github repository	✅
mp3	✅
mp4	✅
Sitemap	✅
Youtube	✅
Notion	❌

Language models

Provider	Models
OpenAI	gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-4, gpt-4-0613, gpt-3.5-instruct, gpt-3.5-turbo-1106, gpt-4-1106-preview
Anthropic	Claude 1, Claude 2
Google	chat-bison-001, Gemini
Fireworks	Llama 2, Mistral, Mixtral and more
Local AI	Local AI models
Ollama	All Ollama supported models

Embedding models

Provider	Models	Status
OpenAI	text-embedding-ada-002	✅
TensorFlow	universal-sentence-encoder	❌
Hugging Face	distilbert-base-uncased	✅
Cohere	Embed	✅
Huggingface Transformers.js	all-MiniLM-L6-v2	✅
Ollama	Ollama embedding	✅
Google	text-gecko-001, embedding-001	✅
Jina	Jina embedding	✅

need more ? create an issue...

Integrations

Type	Stage	Status
Web embed script	stable	✅
Telegram	beta	✅
Discord	beta	✅
Slack	development	❌
Whatsapp	experimental	✅

need more ? create an issue...

Contributors ✨

Interested in contributing? Check out the contributing guide.

Sponsors 💖

Dialoqbase nothing without the support of our wonderful sponsors. If you are interested in becoming a sponsor, please visit the sponsorship page.

Special Sponsor

Wonderful Sponsors

And many more wonderful supporters from Ko-fi.

License 📝

MIT

dialoqbase's Issues

fireworks.ai

I didn't understand how to use fireworks.ai.

Can you help me?

Telegram bot error

Needed urgently a telegram bot. Yours appeared to be great, as your repo and discord bot are working perfectly.
For telegram, I'm always getting "Opps! Something went wrong"

Do you have any idea?

website data source issue

i always get data source failed on whatever website link i put in, any ideas?

Originally posted by @TommyWong92 in #1 (comment)

Hitting a Wall with API Rate Limit in GithubRepoLoader

While trying to process a directory in our GithubRepoLoader, I stumbled upon a '403 API rate limit exceeded' error. It appears we're hitting GitHub's limit for unauthenticated requests.

It is preventing us from fetching the repository files we need. And as the error message suggests, we might have better luck with authenticated requests. GitHub's REST API documentation.

Here's the error log to give more context:
Error: Failed to process directory: , Error: Unable to fetch repository files: 403 {"message":"API rate limit exceeded for x.x.x.x. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting";} at GithubRepoLoader.handleError (/app/node_modules/langchain/dist/document_loaders/web/github.cjs:180:23) at GithubRepoLoader.processDirectory (/app/node_modules/langchain/dist/document_loaders/web/github.cjs:153:18) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async GithubRepoLoader.load (/app/node_modules/langchain/dist/document_loaders/web/github.cjs:108:9) at async githubQueueController (/app/queue/controllers/github.controller.js:15:18) at async Queue.queueHandler (/app/queue/index.js:49:25)

One Click Deploy Vercel

Could you add a feature to deploy this feature easier?

What is the best way to deploy this code

patreon.com

Hello,

Do you have a patreon.com or equivalent.

I've been following the project for three days and you're being very responsive.

You deserve it!

Github issue

I am not able to connect the github repo it says status failed

V.1.0.1==> V1.0.2 Telegram and Discord problem

Since last night's update, the bot on Telegram and Discord no longer works.

The "ping" command gives "pong".

The "start" command gives "Hey, How can I assist you?

Then you get: "Opps! Something went wrong".

Thanks for your help

Multi user?

Can users have their own bots via supabase?

Stuck at "Bot is cooking... It may take some time"

I had no problem creating new bots on the 0.0.3 version , but on the 1.0.0 Version , i get stuck at "Bot is cooking... It may take some time" , noting that i tired using all-Mini and OpenAI embedding models and its the same issue .

Railway deployment dont work it keeps crashing also docker deployed in mac isnt working

Railway deployment dont work it keeps crashing also docker deployed in mac isnt working

Integration of additional document loaders (S3, Google Drive)

Hello,

FIrst off, this is an amazing project. Really useful.

Was curious if support was planned for Google Drive, and S3 Compatible Buckets (I guess that is S3 directory or S3 file in langchain).

Thanks!

Docx error

Error: Corrupted zip: can't find end of central directory
at ZipEntries.readEndOfCentral (/app/node_modules/jszip/lib/zipEntries.js:170:23)
at /app/node_modules/jszip/lib/load.js:48:24

When docx are too large, getting error.

Contacting you

Hello, do you have any discord server or anything to directly talk?

dialoqbase exited with code 0 - Latest version

dialoqbase | yarn run v1.22.19
dialoqbase | $ npx prisma migrate deploy && fastify start app.js
dialoqbase | Prisma schema loaded from prisma/schema.prisma
dialoqbase | Datasource "db": PostgreSQL database "dialoqbase", schema "public" at "dialoqbase-pg:5432"
dialoqbase |
dialoqbase | 15 migrations found in prisma/migrations
dialoqbase |
dialoqbase |
dialoqbase | No pending migrations to apply.
dialoqbase | WARNING: Session secret should be 32 characters long.
dialoqbase | Connecting integration...
dialoqbase | Connecting to database...
dialoqbase | Error: the secret must have length 32 or greater
dialoqbase | at checkOptions (/app/node_modules/@fastify/session/lib/fastifySession.js:211:14)
dialoqbase | at fastifySession (/app/node_modules/@fastify/session/lib/fastifySession.js:10:17)
dialoqbase | at Plugin.exec (/app/node_modules/avvio/plugin.js:130:19)
dialoqbase | at Boot.loadPlugin (/app/node_modules/avvio/plugin.js:272:10)
dialoqbase | at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
dialoqbase | error Command failed with exit code 1.
dialoqbase | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
dialoqbase exited with code 0
dialoqbase | yarn run v1.22.19
dialoqbase | $ npx prisma migrate deploy && fastify start app.js
dialoqbase | Prisma schema loaded from prisma/schema.prisma
dialoqbase | Datasource "db": PostgreSQL database "dialoqbase", schema "public" at "dialoqbase-pg:5432"
dialoqbase |
dialoqbase | 15 migrations found in prisma/migrations
dialoqbase |
dialoqbase |
dialoqbase | No pending migrations to apply.
dialoqbase | WARNING: Session secret should be 32 characters long.
dialoqbase | Connecting integration...
dialoqbase | Connecting to database...
dialoqbase | Error: the secret must have length 32 or greater
dialoqbase | at checkOptions (/app/node_modules/@fastify/session/lib/fastifySession.js:211:14)
dialoqbase | at fastifySession (/app/node_modules/@fastify/session/lib/fastifySession.js:10:17)
dialoqbase | at Plugin.exec (/app/node_modules/avvio/plugin.js:130:19)
dialoqbase | at Boot.loadPlugin (/app/node_modules/avvio/plugin.js:272:10)
dialoqbase | at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
dialoqbase | error Command failed with exit code 1.
dialoqbase | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
dialoqbase exited with code 0

API request taking forever

I got a new issue today.
I wanted to go on the railway app, the server was still running, but while trying to access a chatbot it's always cooking and my logs are doing this for at least 25 minutes :

{"level":30,"time":1687727138712,"pid":132,"hostname":"railway","reqId":"req-25f","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.3","remotePort":47296},"msg":"incoming request"}
{"level":30,"time":1687727139138,"pid":132,"hostname":"railway","reqId":"req-25f","res":{"statusCode":200},"responseTime":425.84250497817993,"msg":"request completed"}
{"level":30,"time":1687727140314,"pid":132,"hostname":"railway","reqId":"req-25g","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.2","remotePort":44332},"msg":"incoming request"}
{"level":30,"time":1687727143599,"pid":132,"hostname":"railway","reqId":"req-25i","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.2","remotePort":44336},"msg":"incoming request"}
{"level":30,"time":1687727144026,"pid":132,"hostname":"railway","reqId":"req-25i","res":{"statusCode":200},"responseTime":426.2763170003891,"msg":"request completed"}
{"level":30,"time":1687727145193,"pid":132,"hostname":"railway","reqId":"req-25j","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.2","remotePort":37902},"msg":"incoming request"}
{"level":30,"time":1687727145759,"pid":132,"hostname":"railway","reqId":"req-25j","res":{"statusCode":200},"responseTime":566.5842260122299,"msg":"request completed"}
{"level":30,"time":1687727146928,"pid":132,"hostname":"railway","reqId":"req-25k","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.4","remotePort":53040},"msg":"incoming request"}
{"level":30,"time":1687727147354,"pid":132,"hostname":"railway","reqId":"req-25k","res":{"statusCode":200},"responseTime":425.9129190146923,"msg":"request completed"}
{"level":30,"time":1687727148589,"pid":132,"hostname":"railway","reqId":"req-25l","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.3","remotePort":47296},"msg":"incoming request"}
{"level":30,"time":1687727149015,"pid":132,"hostname":"railway","reqId":"req-25l","res":{"statusCode":200},"responseTime":425.7561550140381,"msg":"request completed"}
{"level":30,"time":1687727150183,"pid":132,"hostname":"railway","reqId":"req-25m","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.3","remotePort":47296},"msg":"incoming request"}
{"level":30,"time":1687727150608,"pid":132,"hostname":"railway","reqId":"req-25m","res":{"statusCode":200},"responseTime":425.7204469740391,"msg":"request completed"}
{"level":30,"time":1687727151776,"pid":132,"hostname":"railway","reqId":"req-25n","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.2","remotePort":44336},"msg":"incoming request"}
{"level":30,"time":1687727152202,"pid":132,"hostname":"railway","reqId":"req-25n","res":{"statusCode":200},"responseTime":425.717757999897,"msg":"request completed"}
{"level":30,"time":1687727153370,"pid":132,"hostname":"railway","reqId":"req-25o","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.4","remotePort":53040},"msg":"incoming request"}
{"level":30,"time":1687727153796,"pid":132,"hostname":"railway","reqId":"req-25o","res":{"statusCode":200},"responseTime":425.70049300789833,"msg":"request completed"}
{"level":30,"time":1687727154963,"pid":132,"hostname":"railway","reqId":"req-25p","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.4","remotePort":53040},"msg":"incoming request"}
{"level":30,"time":1687727155390,"pid":132,"hostname":"railway","reqId":"req-25p","res":{"statusCode":200},"responseTime":425.9206820130348,"msg":"request completed"}
{"level":30,"time":1687727156612,"pid":132,"hostname":"railway","reqId":"req-25q","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.4","remotePort":53040},"msg":"incoming request"}
{"level":30,"time":1687727157038,"pid":132,"hostname":"railway","reqId":"req-25q","res":{"statusCode":200},"responseTime":426.54264399409294,"msg":"request completed"}
{"level":30,"time":1687727158330,"pid":132,"hostname":"railway","reqId":"req-25r","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.4","remotePort":58432},"msg":"incoming request"}
{"level":30,"time":1687727158756,"pid":132,"hostname":"railway","reqId":"req-25r","res":{"statusCode":200},"responseTime":425.63733699917793,"msg":"request completed"}
{"level":30,"time":1687727159937,"pid":132,"hostname":"railway","reqId":"req-25s","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.2","remotePort":44336},"msg":"incoming request"}
{"level":30,"time":1687727160364,"pid":132,"hostname":"railway","reqId":"req-25s","res":{"statusCode":200},"responseTime":426.29677802324295,"msg":"request completed"}
{"level":30,"time":1687727161562,"pid":132,"hostname":"railway","reqId":"req-25t","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.2","remotePort":37902},"msg":"incoming request"}
{"level":30,"time":1687727162129,"pid":132,"hostname":"railway","reqId":"req-25t","res":{"statusCode":200},"responseTime":566.4999960064888,"msg":"request completed"}
{"level":30,"time":1687727163297,"pid":132,"hostname":"railway","reqId":"req-25u","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.3","remotePort":44696},"msg":"incoming request"}
{"level":30,"time":1687727163722,"pid":132,"hostname":"railway","reqId":"req-25u","res":{"statusCode":200},"responseTime":425.62202700972557,"msg":"request completed"}
{"level":30,"time":1687727164949,"pid":132,"hostname":"railway","reqId":"req-25v","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.3","remotePort":44696},"msg":"incoming request"}
{"level":30,"time":1687727165375,"pid":132,"hostname":"railway","reqId":"req-25v","res":{"statusCode":200},"responseTime":425.8488650023937,"msg":"request completed"}
{"level":30,"time":1687727166643,"pid":132,"hostname":"railway","reqId":"req-25w","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.2","remotePort":44336},"msg":"incoming request"}
{"level":30,"time":1687727167069,"pid":132,"hostname":"railway","reqId":"req-25w","res":{"statusCode":200},"responseTime":425.7488560080528,"msg":"request completed"}
{"level":30,"time":1687727168239,"pid":132,"hostname":"railway","reqId":"req-25x","req":{"method":"GET","url":"/api/v1/bot/appearance/cljbnlytt0000mc3oyz7bxz7a","hostname":"dialoqbase-server-dialoqbase-sdserver-pr-1.up.railway.app","remoteAddress":"192.168.0.2","remotePort":44336},"msg":"incoming request"}
{"level":30,"time":1687727168664,"pid":132,"hostname":"railway","reqId":"req-25x","res":{"statusCode":200},"responseTime":425.6209850013256,"msg":"request completed"}
{"level":30,"time":1687727169834,"pid":132,"hostname":"railway","reqId":"req

Can someone help me look at this problem

When I started the project login account, I kept throwing exceptions when I entered the Get my robot page. I checked the http call interface and the result was {
"statusCode": 401,
"code": "FST_JWT_AUTHORIZATION_TOKEN_INVALID",
"error": "Unauthorized",
"message": "Authorization token is invalid: The token signature is invalid."
I have checked redis, dialoqbase and Dialoqbase-Postgres and there are no error logs. May I ask why we can use all of them normally before October

ENHANCEMENT: Decouple content processing from UI

Right now, if I load a bunch of material in, specifically long running tasks like mp3, mp4 or YouTube videos, the Web UI hangs.

Being able to scale out the 'workers' separately from the web-ui in a queue / scale out type system would be great. The queue is sending back a processing notification to the web-ui, and someone could continue to load resources in, and then they take a long to process as they take, but the UI is still available.

Does this make sense? Let me know if I am talking crazy.

Windows Installation Failing | PostgreSQL Initialization Error

Encountered an issue while running a PostgreSQL Docker container on Windows. logs:

postgres  | The files belonging to this database system will be owned by user "postgres".
postgres  | This user must also own the server process.
postgres  |
postgres  | The database cluster will be initialized with locale "en_US.utf8".
postgres  | The default database encoding has accordingly been set to "UTF8".
postgres  | The default text search configuration will be set to "english".
postgres  |
postgres  | Data page checksums are disabled.
postgres  |
postgres  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres  | creating subdirectories ... ok
postgres  | selecting dynamic shared memory implementation ... posix
postgres  | selecting default max_connections ... 100
postgres  | selecting default shared_buffers ... 128MB
postgres  | selecting default time zone ... Etc/UTC
postgres  | creating configuration files ... ok
postgres  | running bootstrap script ... ok
postgres  | performing post-bootstrap initialization ... ok
postgres  | syncing data to disk ... initdb: warning: enabling "trust" authentication for local connections
postgres  | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
postgres  | ok
postgres  |
postgres  |
postgres  | Success. You can now start the database server using:
postgres  |
postgres  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres  |
postgres  | waiting for server to start....2023-06-07 10:41:08.567 UTC [49] LOG:  starting PostgreSQL 15.3 (Debian 15.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
postgres  | 2023-06-07 10:41:08.572 UTC [49] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres  | 2023-06-07 10:41:08.626 UTC [52] LOG:  database system was shut down at 2023-06-07 10:41:05 UTC
postgres  | 2023-06-07 10:41:08.728 UTC [49] LOG:  database system is ready to accept connections
postgres  |  done
postgres  | server started
postgres  | CREATE DATABASE
postgres  |
postgres  |
postgres  | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sh
postgres  | /usr/bin/env: ‘bash\r’: No such file or directory
postgres exited with code 127

The error seems to be related to the entrypoint script (docker-entrypoint-initdb.d/init.sh) failing with the following error: "/usr/bin/env: ‘bash\r’: No such file or directory".

No documentation found : Embedding settings & more

First: It's a really great project. Compared to many other comparable chatbot projects I like the clean interface, the customization y the useful additional features like the integrations and the very easy installation!

Maybe you can help me:

1. Where / how can I change the a) amount and b) the length of context chunks from the embeddings?
2. Can I customize the welcome message?
3. After ~10 sources I couldn't add more. Is it on purpose?

Would be great to hearing from you! Keep up the good work!

railway doesnt work

Hi, it keeps crashing on railway

PDF import fails.

Hi,
The following error was output to the docker log.

loading pdf
Warning: Indexing all PDF objects
Error
    at InvalidPDFExceptionClosure (/app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:452:35)
    at Object.<anonymous> (/app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:455:2)
    at __w_pdfjs_require__ (/app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:45:30)
    at Object.<anonymous> (/app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:7939:23)
    at __w_pdfjs_require__ (/app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:45:30)
    at /app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:88:18
    at /app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:91:10
    at webpackUniversalModuleDefinition (/app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:18:20)
    at Object.<anonymous> (/app/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js:25:3)
    at Module._compile (node:internal/modules/cjs/loader:1254:14) {
  message: 'Invalid PDF structure'
}

Thanks All

Telegram doesnt work

I got the telegram_bot_token and added it to the control panel and enabled it , but it doesn't work and i don't know what's the error to troubleshoot it , there is no error log .
can you please help ?

Cite Sources Feature

How do you think I can add a citation feature? basically asking the bot to include where they find the sources from after answering the question, so users can cross refer and make sure its accurate

Known Issue: Google Chat-Bison-001 LLM Providing Empty Text Responses

Issue Description:

There is a known issue with Google's Chat-Bison-001 LLM, where it occasionally responds with empty text when provided with context. This issue has been observed both in the public preview version of the model api and within the Makersuiter Playground.

Expected Behavior:

When interacting with the Chat-Bison-001 LLM, users expect meaningful and informative responses based on the provided context.

Current Behavior:

Unfortunately, the Chat-Bison-001 LLM occasionally returns empty text responses, which provide no meaningful information to users.

Database Tables Disappears and System Crashes with Docker-Compose Configuration

Error: P3009

migrate found failed migrations in the target database, new migrations will not be applied. Read more about how to resolve migration issues in a production database: https://pris.ly/d/migrate-resolve
The init migration started at 2023-07-17 10:39:24.473380 UTC failed

SSL certificate, error.

Hello,

I wanted to use port 80, and put an SSL certificate on port 443.

Here's how I modified the file ocker-compose.yml:

version: '3.9'

services:
  dialoqbase:
    image: n4z3m/dialoqbase:latest
    container_name: dialoqbase
    restart: unless-stopped
    ports:
      - 80:3000
      - 443:443
    environment:
      DATABASE_URL: postgres://postgres:postgres@dialoqbase-pg:5432/dialoqbase?connection_limit=15&pool_timeout=0
      DB_REDIS_URL: redis://redis:6379
    env_file:
      - .env
    depends_on:
      - dialoqbase-pg
      - redis
    volumes:
      - .uploads:/app/uploads
      - ./letsencrypt:/etc/letsencrypt
      - ./lib/letsencrypt:/var/lib/letsencrypt

  dialoqbase-pg:
    image: ankane/pgvector:latest
    container_name: dialoqbase-postgres
    restart: unless-stopped
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: dialoqbase
    volumes:
      - .postgres:/var/lib/postgresql/data

  redis:
    image: redis:latest
    container_name: redis
    restart: unless-stopped
    volumes:
      - .redis:/data

  certbot:
    image: certbot/certbot
    container_name: certbot
    volumes:
      - ./letsencrypt:/etc/letsencrypt
      - ./lib/letsencrypt:/var/lib/letsencrypt
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"

Then I ran the following commands:

docker-compose down
docker run -it --rm --name certbot -v "$(pwd)/letsencrypt:/etc/letsencrypt" -v "$(pwd)/lib/letsencrypt:/var/lib/letsencrypt" -p 80:80 certbot/certbot certonly --standalone -d bot.hd1.ch --email [email protected] --agree-tos --no-eff-email
docker-compose up -d

docker logs dialoqbase

yarn run v1.22.19
$ npx prisma migrate deploy && fastify start  app.js
Prisma schema loaded from prisma/schema.prisma
Datasource "db": PostgreSQL database "dialoqbase", schema "public" at "dialoqbase-pg:5432"

14 migrations found in prisma/migrations


No pending migrations to apply.
npm notice
npm notice New major version of npm available! 9.8.1 -> 10.2.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.2.0>
npm notice Run `npm install -g [email protected]` to update!
npm notice
Connecting integration...
Connecting to database...

Thank you for your help

The current answer speed is too slow, can you help optimize it to use streaming output to improve user experience

The current answer speed is too slow, can you help optimize it to use streaming output to improve user experience

Segmentation fault error Command failed with exit code 139.

Try to run project with docker, but there is a Segmentation fault happened.

dialoqbase container always restart.

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a7adf467eeff n4z3m/dialoqbase:latest "docker-entrypoint.s…" 58 minutes ago Restarting (139) 5 seconds ago dialoqbase
5055d85a6c47 ankane/pgvector:latest "docker-entrypoint.s…" 2 hours ago Up 28 seconds 5432/tcp dialoqbase-postgres
2b93ded0fd63 redis:latest "docker-entrypoint.s…" 2 hours ago Up 28 seconds 6379/tcp redis

$ docker logs -f dialoqbase
yarn run v1.22.19
$ npx prisma migrate deploy && fastify start app.js
Segmentation fault
error Command failed with exit code 139.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
yarn run v1.22.19
$ npx prisma migrate deploy && fastify start app.js
Segmentation fault
error Command failed with exit code 139.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
yarn run v1.22.19
$ npx prisma migrate deploy && fastify start app.js
Segmentation fault
error Command failed with exit code 139.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
yarn run v1.22.19
$ npx prisma migrate deploy && fastify start app.js
Segmentation fault
error Command failed with exit code 139.

[Feature Request] Support InternLM

Dear dialoqbase developer,

Greetings! I am vansinhu, a community developer and volunteer at InternLM. Your work has been immensely beneficial to me, and I believe it can be effectively utilized in InternLM as well. Welcome to add Discord https://discord.gg/gF9ezcmtM3 . I hope to get in touch with you.

Best regards,
vansinhu

[bug]can not add datasource

dialoqbase | An operation failed because it depends on one or more records that were required but not found. Record to update not found. dialoqbase | at vn.handleRequestError (/app/node_modules/@prisma/client/runtime/library.js:123:6730) dialoqbase | at vn.handleAndLogRequestError (/app/node_modules/@prisma/client/runtime/library.js:123:6119) dialoqbase | at vn.request (/app/node_modules/@prisma/client/runtime/library.js:123:5839) dialoqbase | at async l (/app/node_modules/@prisma/client/runtime/library.js:128:9763) dialoqbase | at async Queue.queueHandler (/app/queue/index.js:23:17) { dialoqbase | code: 'P2025', dialoqbase | clientVersion: '5.2.0', dialoqbase | meta: { cause: 'Record to update not found.' } dialoqbase | } dialoqbase | PrismaClientKnownRequestError: dialoqbase | Invalid prisma.botSource.update()invocation: dialoqbase | dialoqbase | dialoqbase | dialoqbase | dialoqbase | An operation failed because it depends on one or more records that were required but not found. Record to update not found. dialoqbase | at vn.handleRequestError (/app/node_modules/@prisma/client/runtime/library.js:123:6730) dialoqbase | at vn.handleAndLogRequestError (/app/node_modules/@prisma/client/runtime/library.js:123:6119) dialoqbase | at vn.request (/app/node_modules/@prisma/client/runtime/library.js:123:5839) dialoqbase | at async l (/app/node_modules/@prisma/client/runtime/library.js:128:9763) dialoqbase | at async Queue.queueHandler (/app/queue/index.js:77:17) { dialoqbase | code: 'P2025', dialoqbase | clientVersion: '5.2.0', dialoqbase | meta: { cause: 'Record to update not found.' } dialoqbase | } dialoqbase | Processing queue dialoqbase | PrismaClientKnownRequestError: dialoqbase | Invalidprisma.botSource.update()invocation: dialoqbase | dialoqbase | dialoqbase | dialoqbase | dialoqbase | An operation failed because it depends on one or more records that were required but not found. Record to update not found. dialoqbase | at vn.handleRequestError (/app/node_modules/@prisma/client/runtime/library.js:123:6730) dialoqbase | at vn.handleAndLogRequestError (/app/node_modules/@prisma/client/runtime/library.js:123:6119) dialoqbase | at vn.request (/app/node_modules/@prisma/client/runtime/library.js:123:5839) dialoqbase | at async l (/app/node_modules/@prisma/client/runtime/library.js:128:9763) dialoqbase | at async Queue.queueHandler (/app/queue/index.js:23:17) { dialoqbase | code: 'P2025', dialoqbase | clientVersion: '5.2.0', dialoqbase | meta: { cause: 'Record to update not found.' } dialoqbase | } dialoqbase | PrismaClientKnownRequestError: dialoqbase | Invalidprisma.botSource.update() invocation: dialoqbase | dialoqbase | dialoqbase | dialoqbase | dialoqbase | An operation failed because it depends on one or more records that were required but not found. Record to update not found. dialoqbase | at vn.handleRequestError (/app/node_modules/@prisma/client/runtime/library.js:123:6730) dialoqbase | at vn.handleAndLogRequestError (/app/node_modules/@prisma/client/runtime/library.js:123:6119) dialoqbase | at vn.request (/app/node_modules/@prisma/client/runtime/library.js:123:5839) dialoqbase | at async l (/app/node_modules/@prisma/client/runtime/library.js:128:9763) dialoqbase | at async Queue.queueHandler (/app/queue/index.js:77:17) { dialoqbase | code: 'P2025', dialoqbase | clientVersion: '5.2.0', dialoqbase | meta: { cause: 'Record to update not found.' } dialoqbase | } dialoqbase | Processing queue

it is faild when i have add datasource.

Who can help me look at this problem, thank you

Following the tutorial I pulled up the code from git to my computer, edited the.env file and filled it with OPENAI_API_KEY and DB_SECRET_KEY, and then started the project with docker-compose up -d to create a bot, but when I sent a message to the bot, The http request result returned an error, as follows

{
"statusCode": 500,
"code": "P2010",
"error": "Internal Server Error",
"message": "\nInvalid prisma.$queryRaw() invocation:\n\n\nRaw query failed. Code: 54000. Message: ERROR: vector cannot have more than 1024 dimensions"
}

What should I do in this situation? Did I fill in the wrong configuration?I have tried to log in again by logging out of the account, deleting the container in docker and the three directories of.postgres,.redis, and.uploads in the docker directory of the project, and then using docker-compose up -d command to start the project

Authorization token is invalid: The token signature is invalid.

I keep getting : Authorization token is invalid: The token signature is invalid.

I am trying to connect to a repo, I have tried adding the key classic and the new version several times. Please advise!

Thanks

REST API as datasource

These features are requested by @phpia

Railway Template - 6 Required Inputs but docs say only Supabase+OpenAI needed

I'm trying to use the Railway template. The docs here say that only two inputs are required for the initial deployment: a Supabase URL and an OpenAI key.

Instead, the following are required:
DATABASE_URL (Supabase)
OPENAI_API_KEY
GITHUB_ACCESS_TOKEN
FIREWORKS_API_KEY
ELEVENLABS_API_KEY
GOOGLE_API_KEY

It's not clear why these other API providers are required upfront, nor why dialoqbase requires a personal access token for my github account.

Can you either update the docs to reflect these additional requirements, or adjust the railway template to align with the docs? I'm looking forward to using the project

Data Hallucination Issue with Crawler on JavaScript and CSS-heavy Websites

Issue Description:

I've encountered an issue with data hallucination when using a web crawler on websites with a substantial amount of JavaScript and CSS. The crawler, specifically the CheerioWebBaseLoader, seems to be producing inaccurate or incomplete data due to the dynamic nature of these websites.

Steps to Reproduce:

1. Select Crawler as Data Source.
2. Choose a website with a large amount of JavaScript and CSS, for example, (https://www.ipangram.com/).
3. Let the bot cook and ask any questions regarding to the website.

Expected Behavior:
The llm should answer expected answers related to website.

Actual Behavior:
The llm appears to be producing inaccurate answers.

Environment:
Operating System: Windows 10
Embeddings: Huggingface
ChatModel: Fireworks (llama-v2-7b-chat)

Screenshots:
This is the chunks generated by Document Loader

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

When adding smaller repos, the system seems to work fine. I am able to ask questions about those github repos.

I am working on some projects that require MinIO Object Storage integration and I wanted to ask questions of their docs.

When I load this repo https://github.com/minio/docs from main, it waits a bit then throws a Java Heap error.

Any ideas? Should I switch to Supabase instead of PGVector?

docker compose logs -f output:

dialoqbase           | Cloning into './uploads/minio-docs-main'...
dialoqbase           |
dialoqbase           | <--- Last few GCs --->
dialoqbase           |
dialoqbase           | [81:0x7222e80]   226205 ms: Mark-sweep 2002.8 (2090.1) -> 1990.7 (2091.9) MB, 99.6 / 0.0 ms  (average mu = 0.853, current mu = 0.568) allocation failure; scavenge might not succeed
dialoqbase           | [81:0x7222e80]   226456 ms: Mark-sweep 2008.3 (2093.9) -> 1998.0 (2100.1) MB, 217.0 / 0.0 ms  (average mu = 0.638, current mu = 0.133) allocation failure; scavenge might not succeed
dialoqbase           |
dialoqbase           |
dialoqbase           | <--- JS stacktrace --->
dialoqbase           |
dialoqbase           | FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
dialoqbase           |  1: 0xb83f50 node::Abort() [/usr/local/bin/node]
dialoqbase           |  2: 0xa94834  [/usr/local/bin/node]
dialoqbase           |  3: 0xd647c0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
dialoqbase           |  4: 0xd64b67 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
dialoqbase           |  5: 0xf42265  [/usr/local/bin/node]
dialoqbase           |  6: 0xf5474d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
dialoqbase           |  7: 0xf2ee4e v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
dialoqbase           |  8: 0xf30217 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
dialoqbase           |  9: 0xf10760 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/usr/local/bin/node]
dialoqbase           | 10: 0xf081d4 v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [/usr/local/bin/node]
dialoqbase           | 11: 0xf0a568 v8::internal::FactoryBase<v8::internal::Factory>::NewRawTwoByteString(int, v8::internal::AllocationType) [/usr/local/bin/node]
dialoqbase           | 12: 0x11e7bb5 v8::internal::String::SlowFlatten(v8::internal::Isolate*, v8::internal::Handle<v8::internal::ConsString>, v8::internal::AllocationType) [/usr/local/bin/node]
dialoqbase           | 13: 0x1051cd2 v8::internal::JsonStringifier::SerializeString(v8::internal::Handle<v8::internal::String>) [/usr/local/bin/node]
dialoqbase           | 14: 0x10528b1 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
dialoqbase           | 15: 0x105448f v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
dialoqbase           | 16: 0x105448f v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
dialoqbase           | 17: 0x1056cef v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
dialoqbase           | 18: 0x1057a3f v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
dialoqbase           | 19: 0xdea777 v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
dialoqbase           | 20: 0x17036b9  [/usr/local/bin/node]
dialoqbase           | Aborted (core dumped)
dialoqbase           | error Command failed with exit code 134.
dialoqbase           | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
dialoqbase exited with code 0
dialoqbase           | yarn run v1.22.19
dialoqbase           | $ npx prisma migrate deploy && fastify start  app.js
dialoqbase           | Prisma schema loaded from prisma/schema.prisma
dialoqbase           | Datasource "db": PostgreSQL database "dialoqbase", schema "public" at "dialoqbase-pg:5432"
dialoqbase           |
dialoqbase           | 9 migrations found in prisma/migrations
dialoqbase           |
dialoqbase           |
dialoqbase           | No pending migrations to apply.
dialoqbase           | [info] use ffmpeg.wasm v0.12.0
dialoqbase           | Connecting pm2...
dialoqbase           | Connecting to database...

Youtube Import

Youtube never proceeds after processing, never succeeds. I am trying out /next branch

2023-09-30 21:00:21.691462770 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.0/self_attn_layer_norm/Constant_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691479421 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.1/self_attn_layer_norm/Constant_1_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691489628 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.1/self_attn_layer_norm/Constant_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691496026 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.2/encoder_attn_layer_norm/Constant_1_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691502890 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.1/encoder_attn_layer_norm/Constant_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691517128 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.1/encoder_attn_layer_norm/Constant_1_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691531626 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.0/self_attn_layer_norm/Constant_1_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691537135 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.2/encoder_attn_layer_norm/Constant_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691542335 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layer_norm/Constant_output_0'. It is not used by any node and should be removed from the model.

2023-09-30 21:00:21.691546489 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.3/final_layer_norm/Constant_output_0'. It is not used by any node and should be removed from the model.

[info] load ffmpeg-core

[info] loading ffmpeg-core

[info] ffmpeg-core loaded

ffmpeg loaded

[info] run FS.writeFile ./audio.mp3 <23527313 bytes binary file>

[info] run ffmpeg command: -i ./audio.mp3 -acodec pcm_s16le -ac 1 -ar 16000 ./audio.wav

[fferr] ffmpeg version v0.11.0-6-g16758e9d2b Copyright (c) 2000-2020 the FFmpeg developers

[fferr] built with emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.34 (57b21b8fdcbe3ebb523178b79465254668eab408)

[fferr] configuration: --target-os=none --arch=x86_32 --enable-cross-compile --disable-x86asm --disable-inline-asm --disable-stripping --disable-programs --disable-doc --disable-debug --disable-runtime-cpudetect --disable-autodetect --extra-cflags='-O3 --closure=1 -I/work/ffmpeg.wasm-core/build/include -s USE_PTHREADS=1' --extra-cxxflags='-O3 --closure=1 -I/work/ffmpeg.wasm-core/build/include -s USE_PTHREADS=1' --extra-ldflags='-O3 --closure=1 -I/work/ffmpeg.wasm-core/build/include -s USE_PTHREADS=1 -L/work/ffmpeg.wasm-core/build/lib' --pkg-config-flags=--static --nm=llvm-nm --ar=emar --ranlib=emranlib --cc=emcc --cxx=em++ --objcc=emcc --dep-cc=emcc --enable-gpl --enable-nonfree --enable-zlib --enable-libx264 --enable-libx265 --enable-libvpx --enable-libwavpack --enable-libmp3lame --enable-libfdk-aac --enable-libtheora --enable-libvorbis --enable-libfreetype --enable-libopus --enable-libwebp --enable-libass --enable-libfribidi

[fferr] libavutil 56. 51.100 / 56. 51.100

[fferr] libavcodec 58. 91.100 / 58. 91.100

[fferr] libavformat 58. 45.100 / 58. 45.100

[fferr] libavdevice 58. 10.100 / 58. 10.100

[fferr] libavfilter 7. 85.100 / 7. 85.100

[fferr] libswscale 5. 7.100 / 5. 7.100

[fferr] libswresample 3. 7.100 / 3. 7.100

[fferr] libpostproc 55. 7.100 / 55. 7.100

[fferr] Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './audio.mp3':

[fferr] Metadata:

[fferr] major_brand : mp42

[fferr] minor_version : 0

[fferr] compatible_brands: isommp42

[fferr] creation_time : 2023-09-30T16:56:01.000000Z

[fferr] Duration: 00:02:55.33, start: 0.000000, bitrate: 1073 kb/s

[fferr] Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 942 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)

[fferr] Metadata:

[fferr] creation_time : 2023-09-30T16:56:01.000000Z

[fferr] handler_name : ISO Media file produced by Google Inc. Created on: 09/30/2023.

[fferr] Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)

[fferr] Metadata:

[fferr] creation_time : 2023-09-30T16:56:01.000000Z

[fferr] handler_name : ISO Media file produced by Google Inc. Created on: 09/30/2023.

[fferr] Stream mapping:

[fferr] Stream #0:1 -> #0:0 (aac (native) -> pcm_s16le (native))

[fferr] Output #0, wav, to './audio.wav':

[fferr] Metadata:

[fferr] major_brand : mp42

[fferr] minor_version : 0

[fferr] compatible_brands: isommp42

[fferr] ISFT : Lavf58.45.100

[fferr] Stream #0:0(und): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 16000 Hz, mono, s16, 256 kb/s (default)

[fferr] Metadata:

[fferr] creation_time : 2023-09-30T16:56:01.000000Z

[fferr] handler_name : ISO Media file produced by Google Inc. Created on: 09/30/2023.

[fferr] encoder : Lavc58.91.100 pcm_s16le

[fferr] size= 1kB time=00:00:00.04 bitrate= 269.7kbits/s speed=0.0206x

[fferr] size= 2816kB time=00:01:36.43 bitrate= 239.2kbits/s speed=35.6x

[fferr] size= 5479kB time=00:02:55.33 bitrate= 256.0kbits/s speed=56.6x

[fferr] video:0kB audio:5479kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.001390%

[ffout] FFMPEG_END

[info] run FS.readFile ./audio.wav

[Program terminated with exit(0)] undefined

Integrate transcription and TTS (Voice to Voice)

Please consider adding a version with whisper_jax for speech transcription. Add elevenlabs functionality for tts. You would then make this UI accessible for disabled people to converse with chat gpt voice to voice. Users can select voice or text interaction in the UI. This way your app will be more inclusive and accessible to everyone regardless of physical ability, Keep up the good work! Thank you for creating this excellent UI.

Seeking Feedback

If you use dialoqbase, I would appreciate your feedback so that I can improve the application. 😊

Please Give Feedback 🤗

Hello, guys! dialoqbase is now at version 0.0.12, and I have been continuously updating it, adding new features every day. If you use dialoqbase, could you please give some feedback? I know that some data loaders are not working as expected, but I will fix that in upcoming updates.

Some quick FAQ

• dialoqbase is a side project.
• I have no plans to release dialoqbase as a cloud or SaaS service.
• I will continue to improve dialoqbase with latest llms, embedding models and data loaders

FST_JWT_AUTHORIZATION_TOKEN_INVALID

Hi, Im following the steps for local installation, I want to be able to contribute to the project :)

But Im getting FST_JWT_AUTHORIZATION_TOKEN_INVALID, when trying to access to http://localhost:3000/api/v1/bot

Some help?

Thanks

Can't Preview or Use Playground for Chatbot

I am unable to use the playground and preview the bot

Suggestion - Conversation Logs

It would be great if the admin can view all bot interactions from all channels within the admin portal under Conversations .

Great product so far , Kudos !

Chat Widget - White Page

The bot works well in the playground , but when i get a "Public URL" link from "Integrations" , under "HTML Embed" , the link opens a white empty page .
noting that it used to work under V0.03

Railway template not working

The railway template keeps crashing with this error message:

Error: Prisma schema validation - (get-config wasm)
Error code: P1012
error: Error validating datasource db: the URL must start with the protocol postgresql:// or postgres://.
--> schema.prisma:8
|
7 | provider = "postgresql"
8 | url = env("DATABASE_URL")
|
Validation Error Count: 1
[Context: getConfig]
Prisma CLI Version : 4.16.2
error Command failed with exit code 1.

FEATURE: Add variable for a different OpenAI API Compatible Endpoint

Would it be possible to add a variable to target a different OpenAI API Compatible Endpoint?

Basically, there are numerous ways to host private models that currently support all of the OpenAI API endpoints, so we could chat with private LLMs and not use 3rd party services.

Maybe something like adding OPENAI_API_ENDPOINT="HTTP://localhost:11434" in the .env file.

Thanks!

Contribution help

It would be great to have precise instructions on how one can contribute to this project.
I would like to add more data loaders, but don't know how I can run this project locally, without pulling an image.

It appears that building and running this repo after each change is not a good way to proceed. Can someone tell me which commands I should run in order to be able to make changes?
I am on macOS and have tried running yard install, and yarn build, but when I do that, I can see the login page but the XMLHttpRequest fails (404 error).

Thanks in advance for any help!

public chat bot enhancement

These features are requested by @oleg-schmidt:

• Save customer chat to browser local storage, only removing it when the customer restarts.
• View all customer chats on the bot dashboard.

@oleg-schmidt, please update if anything is incorrect.