omar-v2/ask-florence

A GenAI chatbot to connect family members with patients, leveraging EHR data.

Ask Florence

Ask Florence, named after Florence Nightingale, is a GenAI application that allows family members to query about their patients in care, alleviating the burden on doctors and medical staff. This app was built as part of the Imperial GenAI x Healthcare hackathon.

I'm proud to have worked on this project with my friends Jannic and Moneim.

The Problem

Doctors spend a large amount of time taking calls from family members of admitted patients, providing updates, answering questions and relieving concerns. This activity in and of itself doesn't improve the care of the admitted patient, yet doctors feel a moral obligation to perform it.

The Solution

Ask Florence is a GenAI, WhatsApp-based chatbot that uses retrieval augmented generation (RAG) to allow family members to directly ask questions about their admitted patients. It works by ingesting patients' electronic health record (EHR) documents into a vector database and then injecting relevant chunks into the context sent to the LLM based on the user's original query. The diagram below shows the flow of information during a request.

Demo

ask-florence-demo.MP4

ask-florence's Issues

Enable conversations via voice messages (English Language)

Enable sending and receiving voice messages when communicating with the WhatsApp bot.
Use ElevenLabs to generate a voice response and send it back to the user using the Twilio API

Implement system for doctor viewing chat history

Implement the solution architected in #3.

Acceptance criteria:

• Email with link and context can be sent to arbitrary email address.

Architect doctor conversation view

Architect a system that allows doctors to see patient's chat history with the bot.

Acceptance criteria:

• Doctor can view chat history after receiving a push notification initiated by user.
• Doctor gets a summary of user contact information.

Implement EHR querying

Acceptance criteria:

• Generates response to an arbitrary user prompt using information indexed from EHR.
• Uses the most up-to-date entries in its responses.
• Refuses to give information on conditions in a pre-defined deny list.
• Only uses EHR info, does not respond to general questions.
• Does not reply with medical advice.

Setup WhatsApp Chatbot with Twilio

Use the Twilio WhatsApp API to setup a fastAPI server that receives webhook requests whenever a new message is sent to the bot.

Create a model for the application

Acceptance criteria:

• Create a user management system with roles for doctors and users.
• Create models for patients, EHRs, doctors, users.