congltk1234/brycenchatbot

This is a mobile application built with Flutter, integrating OpenAI's API. The app was developed during my internship at BRYCEN VIETNAM company (05-08/2023).

BRYCEN Chatbot

Flutter Chatbot with OpenAI (GPT3.5 & Whisper) made by @STkong

Description	Click below to download APK
This Flutter application integrates OpenAI's ChatGPT 3.5 LLM model to enable interactive conversations, provides a feature to summarize uploaded documents and audio files. The app serves as a powerful tool for engaging conversations and efficient content summarization. The app was developed during my internship at BRYCEN Vietnam company

Table of Contents

• Features
• Prerequisites
• Installation
• Error
• Update
• Future Work
• License

Features and Demo

1. Conversations with ChatGPT 3.5 LLM: Engage in dynamic and natural conversations with OpenAI's ChatGPT 3.5 LLM model. Enjoy real-time responses and experience the capabilities of advanced language processing.

2. Summarize File Upload: Easily upload documents and audio files to generate concise summaries. The app utilizes cutting-edge summarization techniques to extract the most important information from your content.

demoFinal.mp4

Home Screen

Login & Validation	Update & Logout	Drawer	Internet Status

Chat Screen

Memory Chatbot	Speech2Text	Copy & Audio

Summarize Screen

Summarize	View File	Question Suggest
Support Files Document:txt, pdf, docx Audio:mp3, wav, mpga, mpeg	Can view pdf, docx, txt	Generate Related questions about the document

Prerequisites

• INSTALLED: npm, Flutter, Git, Firebase CLI
• OPENAI API KEY: You must have an OpenAI API key in order to use this application.

API Key Setup

To use the ChatGPT 3.5 LLM model, you need to set up an API key from OpenAI:

1. Go to the OpenAI website and sign in or create an account.
2. Generate an API key for ChatGPT 3.5 LLM.
3. Copy the API key and input to the app.

Installation

1. Settup Firebase for Flutter app

Init Firebase project

Step1: Create new Firebase project

Step2: Set Up FireStore Database

Step3: Set Up Firebase Storage for store files

Login with Firebase CLI

Install Firebase CLI via npm by running the following command:

  npm install -g firebase-tools

Then Log into Firebase using your Google account

  firebase login

Install FlutterFire CLI

Running the following command from any directory:

  dart pub global activate flutterfire_cli

2. Clone Flutter project:

• Open Terminal and point to folder which will contain your project:

  git clone https://github.com/congltk1234/BrycenChatbot.git

• Change working directory:

  cd BrycenChatbot

• Adding package dependency to project:

  flutter pub get

• Connect Firebase to project:

  flutterfire configure

3. Export the App

Download App APK

Build file APK

  flutter build apk

or

  flutter build apk --no-pub --no-shrink

Install the APK on a connected Android device

Connect your Android device to your computer with a USB cable, then run the command line

  flutter install

Error

Command not found: flutterfire

https://bobbyhadz.com/blog/flutterfire-is-not-recognized-as-internal-or-external-command

• In your terminal and run this code to open Advanced system settings

   SystemPropertiesAdvanced

• Click Environment Variables. In the section System Variables find the PATH environment variable and select it. Click Edit. If the PATH environment variable does not exist, click New.

• In the Edit System Variable (or New System Variable) window, specify the value of the PATH environment variable

C:\Users\YourUsername\AppData\Local\Pub\Cache\bin

• Click OK. Close all remaining windows by clicking OK.
• You might have to restart your computer to active path

Update

From 20.07.2023 to 14.08.2023

Date	Task	Notes
20.07.2023	Init Project	Init simple HomeScreen, ChatScreen and Summarize. Setup Themes and ColorSchemes.
21.07.2023	Route setup	Navigation between screens via route setting using Navigator.pushNamed.
22.07.2023	HomeScreen UI	Init FormWidget for input Username OpenAI key.
23.07.2023	Store User input values	Implement packages SharePreferences to store locally on device.
24.07.2023	UserForm	Username and API key validation (sending test request).
25.07.2023	UserForm Submit	Submit for store value to SharePreferences and switch to welcome widget.
26.07.2023	ChatScreen UI	Update ChatScreen UI: ListView chatMessages and ChatInput.
27.07.2023	Voice feature	Implement speech_to_text packages in ChatScreen, Switch between sendVoice function and sendText function.
28.07.2023	Firebase Connection	Set up Firebase project and implemented messages upload and get with FireStore Database.
29.07.2023	OpenAPI request	Chat response and test with these packages: dart_openai, langchain, langchain_openai. Implement MemoryBuffer.
30.07.2023 31.07.2023	Update ChatScreen UI	Display: TimeStamp, tokenUsed. Add more features: Copy , Text2Speech. Animation: AutoScrolltoEnd. TyperAnimatedText.
01.08.2023	Fix Chat Stream error	Change DataModel, add SwitchCase for StreamBuild, update condition.
02.08.2023	Summarize feature	Summarize UI: Upload file button, read file txt. Test response with embedding, retriveQA and StuffSummarize.
03.08.2023	Update Summarize feature	Store and Retrive sumamrize content, file’s embedded vector with FireStore Database.
04.08.2023	Dynamic Drawer	Retrive and display ChatTitle on Drawer with List StreamBuilder
05.08.2023	ChatDrawer SummarizeDrawer	Control Drawer with button Update Drawer UI SurfixIcon for hide/show password
06.08.2023	App Provider	Passing data between Screens with flutter_riverpod, notifiListener and consumer
07.08.2023	Prompt Template	Prompt and use Regex to extract information from response: Detect language, auto genarate chatTitle (chat topic) for Chat and summarize
08.08.2023	ChatItem Navigation	Linked MenuChat Drawler with chat button. Open and load chat history by click on ChatTitle in drawer.
09.08.2023	NewChat button	Add new converstion function Generate topic for chat title, update menu chat UI
10.08.2023	Delete ChatItem	Wrapped ChatTile with Dismissible Delete confirmation with AlertDialog
11.08.2023	Suggest Question	Display interactable Suggest Question chips and update Chat UI.
12.08.2023	File handler InternetConnection Status	SwitchCase for handle file picked Add more funtion: PDF2text, Docx2text Add feature: View uploaded file on SummarizeScreen
13.08.2023	Audio handler	Audio handle for mp3, wav, mpga, mpeg. Add funtion: Speech2text using OpenAPI (whisper model).
14.08.2023	Config App Name&Icon README file	Delete unnecessary code Testing, build APK Write document…
15.08.2023	Add logout Update README	Refine the app, removing any redundant lines of code Finish README

Future Work

• Add provider, update app structure, optimize and clean code.
• Audio Handler: Try package just_audio or audioplayers for split long audio, play audio in file view
• Markdown display: Try flutter_markdown in chat screen
• Migrate database: from FireStore(firebase) to SQLite
• User Config model: Let user decide the model they want
• Adjust Speech recognition: Show glowing animation of the sentence listening
• Manage usage: Use tiktoken to calculate Token used
• Reduce tokens prompt: Preprocessing prompt and Parser response with text_analysis

License

This project is licensed under the MIT License.

brycenchatbot's Issues

Summarize Feature

Idea: https://bdtechtalks.com/2023/05/01/customize-chatgpt-llm-embeddings/

• #5
• Upload Document (txt/docs or copy of context)
• Read file uploaded
• Embedding text with OpenAI (for context document and userPrompt)
• Store Embedded context in FireStore : https://firebase.google.com/docs/firestore/manage-data/add-data?hl=en#dart_3
• Query Context from FireStore (get and store at local list)
• Calculate CosineSimilarity with https://pub.dev/packages/ml_linalg#cosine-distance-between-two-vectors
  https://stackoverflow.com/questions/58397840/use-isolate-to-sort-list
• Tính toán đa luồng độ tương đồng vector: https://viblo.asia/p/da-luong-trong-flutter-su-dung-isolate-Qbq5QRNJKD8#_phuong-phap-2-tao-ra-vung-biet-lap-isolate-bang-cach-su-dung-compute-8

Chat Screen Feature

• Copy Response : https://www.flutterant.com/how-to-copy-or-past-the-text-from-the-clipboard-in-flutter/
• Animation Text generate with https://pub.dev/packages/animated_text_kit/example
• Show TimeStamp
• Show Token Usage
• Print Code (markdown) https://stackoverflow.com/questions/73108404/how-to-store-a-markdown-editor-content-as-html-tags-in-flutter
• Clear chat history
• MemoryBuffer

Document

1. Học nhanh ngôn ngữ Dart/Flutter via Kotlin
2. text

Optimize Functional

• Better Prompt Query https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/
• Handle Usage Token #1

Handling OpenAI Usage Token

Preprocess for Save token prompt:
https://aiprimer.substack.com/p/gpt-prompt-compression-save-tokens

• https://www.jeypop.co.uk/ai/handling-openai-api-cost-retrieval-changes/
• API: https://api.openai.com/v1/usage?date=2023-07-28
• Dart Package for calculate gpt-tokens: https://pub.dev/packages/flutter_gpt_tokenizer

Summarize Screen UI

• #9

Adding Provider for chatApp

https://stackoverflow.com/questions/58185472/what-is-the-use-of-provider-in-flutter

Suggestion chat

• Action chip
• https://stackoverflow.com/questions/73141620/how-can-i-add-the-suggestion-button-in-my-chatbot?fbclid=IwAR0v6ougZjlVCqok0kSaWaQwej1pf2EA5cCLJ6wkqneFVzTZYuL-yMbqjHA
• https://stackoverflow.com/questions/73141620/how-can-i-add-the-suggestion-button-in-my-chatbot?fbclid=IwAR0v6ougZjlVCqok0kSaWaQwej1pf2EA5cCLJ6wkqneFVzTZYuL-yMbqjHA
• [ ]

Config Model and promt for Users

• List All avaiable models