azure-samples/serverless-chat-langchainjs

Build your own serverless AI Chat with Retrieval-Augmented-Generation using LangChain.js, TypeScript and Azure

Serverless AI Chat with RAG using LangChain.js

⭐ If you like this sample, star it on GitHub — it helps a lot!

Overview • Get started • Run the sample • Resources • FAQ • Troubleshooting

This sample shows how to build a serverless AI chat experience with Retrieval-Augmented Generation using LangChain.js and Azure. The application is hosted on Azure Static Web Apps and Azure Functions, with Azure Cosmos DB for NoSQL as the vector database. You can use it as a starting point for building more complex AI applications.

Tip

You can test this application locally without any cost using Ollama. Follow the instructions in the Local Development section to get started.

Overview

Building AI applications can be complex and time-consuming, but using LangChain.js and Azure serverless technologies allows to greatly simplify the process. This application is a chatbot that uses a set of enterprise documents to generate responses to user queries.

We provide sample data to make this sample ready to try, but feel free to replace it with your own. We use a fictitious company called Contoso Real Estate, and the experience allows its customers to ask support questions about the usage of its products. The sample data includes a set of documents that describes its terms of service, privacy policy and a support guide.

This application is made from multiple components:

• A web app made with a single chat web component built with Lit and hosted on Azure Static Web Apps. The code is located in the packages/webapp folder.

• A serverless API built with Azure Functions and using LangChain.js to ingest the documents and generate responses to the user chat queries. The code is located in the packages/api folder.

• A database to store the text extracted from the documents and the vectors generated by LangChain.js, using Azure Cosmos DB for NoSQL.

• A file storage to store the source documents, using Azure Blob Storage.

We use the HTTP protocol for AI chat apps to communicate between the web app and the API.

Features

• Serverless Architecture: Utilizes Azure Functions and Azure Static Web Apps for a fully serverless deployment.
• Retrieval-Augmented Generation (RAG): Combines the power of Azure Cosmos DB and LangChain.js to provide relevant and accurate responses.
• Scalable and Cost-Effective: Leverages Azure's serverless offerings to provide a scalable and cost-effective solution.
• Local Development: Supports local development using Ollama for testing without any cloud costs.

Get started

There are multiple ways to get started with this project.

The quickest way is to use GitHub Codespaces that provides a preconfigured environment for you. Alternatively, you can set up your local environment following the instructions below.

Important

If you want to run this sample entirely locally using Ollama, you have to follow the instructions in the local environment section.

Use your local environment

You need to install following tools to work on your local machine:

• Node.js LTS
• Azure Developer CLI
• Git
• PowerShell 7+ (for Windows users only)
  • Important: Ensure you can run pwsh.exe from a PowerShell command. If this fails, you likely need to upgrade PowerShell.
  • Instead of Powershell, you can also use Git Bash or WSL to run the Azure Developer CLI commands.
• Azure Functions Core Tools (should be installed automatically with NPM, only install manually if the API fails to start)

Then you can get the project code:

1. Fork the project to create your own copy of this repository.
2. On your forked repository, select the Code button, then the Local tab, and copy the URL of your forked repository.

3. Open a terminal and run this command to clone the repo: git clone <your-repo-url>

Use GitHub Codespaces

You can run this project directly in your browser by using GitHub Codespaces, which will open a web-based VS Code:

Use a VSCode dev container

A similar option to Codespaces is VS Code Dev Containers, that will open the project in your local VS Code instance using the Dev Containers extension.

You will also need to have Docker installed on your machine to run the container.

Run the sample

There are multiple ways to run this sample: locally using Ollama or Azure OpenAI models, or by deploying it to Azure.

Deploy the sample to Azure

Azure prerequisites

• Azure account. If you're new to Azure, get an Azure account for free to get free Azure credits to get started. If you're a student, you can also get free credits with Azure for Students.
• Azure subscription with access enabled for the Azure OpenAI service. You can request access with this form.
• Azure account permissions:
  • Your Azure account must have Microsoft.Authorization/roleAssignments/write permissions, such as Role Based Access Control Administrator, User Access Administrator, or Owner. If you don't have subscription-level permissions, you must be granted RBAC for an existing resource group and deploy to that existing group.
  • Your Azure account also needs Microsoft.Resources/deployments/write permissions on the subscription level.

Cost estimation

See the cost estimation details for running this sample on Azure.

Deploy the sample

1. Open a terminal and navigate to the root of the project.
2. Authenticate with Azure by running azd auth login.
3. Run azd up to deploy the application to Azure. This will provision Azure resources, deploy this sample, and build the search index based on the files found in the ./data folder.
   • You will be prompted to select a base location for the resources. If you're unsure of which location to choose, select eastus2.
   • By default, the OpenAI resource will be deployed to eastus2. You can set a different location with azd env set AZURE_OPENAI_RESOURCE_GROUP_LOCATION <location>. Currently only a short list of locations is accepted. That location list is based on the OpenAI model availability table and may become outdated as availability changes.

The deployment process will take a few minutes. Once it's done, you'll see the URL of the web app in the terminal.

You can now open the web app in your browser and start chatting with the bot.

Enhance security

When deploying the sample in an enterprise context, you may want to enforce tighter security restrictions to protect your data and resources. See the enhance security guide for more information.

Clean up

To clean up all the Azure resources created by this sample:

1. Run azd down --purge
2. When asked if you are sure you want to continue, enter y

The resource group and all the resources will be deleted.

Run the sample locally with Ollama

If you have a machine with enough resources, you can run this sample entirely locally without using any cloud resources. To do that, you first have to install Ollama and then run the following commands to download the models on your machine:

ollama pull mistral:v0.2
ollama pull all-minilm:l6-v2

Note

The mistral model with download a few gigabytes of data, so it can take some time depending on your internet connection.

After that you have to install the NPM dependencies:

npm install

Then you can start the application by running the following command which will start the web app and the API locally:

npm start

Then, open a new terminal running concurrently and run the following command to upload the PDF documents from the /data folder to the API:

npm run upload:docs

This only has to be done once, unless you want to add more documents.

You can now open the URL http://localhost:8000 in your browser to start chatting with the bot.

Note

While local models usually works well enough to answer the questions, sometimes they may not be able to follow perfectly the advanced formatting instructions for the citations and follow-up questions. This is expected, and a limitation of using smaller local models.

Run the sample locally with Azure OpenAI models

First you need to provision the Azure resources needed to run the sample. Follow the instructions in the Deploy the sample to Azure section to deploy the sample to Azure, then you'll be able to run the sample locally using the deployed Azure resources.

Once your deployment is complete, you should see a .env file in the packages/api folder. This file contains the environment variables needed to run the application using Azure resources.

To run the sample, you can then use the same commands as for the Ollama setup. This will start the web app and the API locally:

npm start

Open the URL http://localhost:8000 in your browser to start chatting with the bot.

Note that the documents are uploaded automatically when deploying the sample to Azure with azd up.

Tip

You can switch back to using Ollama models by simply deleting the packages/api/.env file and starting the application again. To regenerate the .env file, you can run azd env get-values > packages/api/.env.

Resources

Here are some resources to learn more about the technologies used in this sample:

• LangChain.js documentation
• Generative AI For Beginners
• Azure OpenAI Service
• Azure Cosmos DB for NoSQL
• Ask YouTube: LangChain.js + Azure Quickstart sample
• Chat + Enterprise data with Azure OpenAI and Azure AI Search
• Revolutionize your Enterprise Data with Chat: Next-gen Apps w/ Azure OpenAI and AI Search

You can also find more Azure AI samples here.

FAQ

You can find answers to frequently asked questions in the FAQ.

Troubleshooting

If you have any issue when running or deploying this sample, please check the troubleshooting guide. If you can't find a solution to your problem, please open an issue in this repository.

Guidance

For more detailed guidance on how to use this sample, please refer to the tutorial.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

serverless-chat-langchainjs's Issues

[Auto] AI Gallery Standard Validation

AI Gallery Standard Validation: FAILED

Repository Management:

• ❌ README.md File. [How to fix?]

  • ❌ Error: ## Getting Started is missing in README.md.

• LICENSE.md File.

• SECURITY.md File.

• CODE_OF_CONDUCT.md File.

• CONTRIBUTING.md File.

• ISSUE_TEMPLATE.md File.

• Topics on repo contains ['azd-templates', 'ai-azd-templates'].

Source code structure and conventions:

• azure-dev.yaml File.
• azure.yaml File.
• ./infra Folder.
• .devcontainer Folder.

Functional Requirements:

• azd up.
• azd down.

Security Requirements:

• ⚠️ microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?]

  Details

  Not found security check related actions in the CI/CD pipeline.

• ⚠️ Security scan. [How to fix?]

  Details

  • ❌ error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated.
    Configure service endpoints and private links where appropriate.

  • ❌ error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits.
    With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing.
    Once you decide to use Azure AD authentication, you can disable authentication using keys.

  • ❌ warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks.
    Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required.

  • ❌ error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised.

  • ❌ error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised.

  • ❌ error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action.
    After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from:

    • Azure services on the trusted service list.
    • IP address or CIDR range.
    • Private endpoint connections.
    • Azure virtual network subnets with a Service Endpoint.

  • ❌ error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted.
    Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously.

  • ❌ error: AZR-000095 - Cosmos DB provides two authorization options for interacting with the database:

    • Azure Active Directory identity (Azure AD).
      Can be used to authorize account and resource management operations.
    • Keys and resource tokens.
      Can be used to authorize resource management and data operations.
      Resource management operations include management of databases, indexes, and containers. By default, keys are permitted to perform resource management operations. You can restrict these operations to Azure Resource Manager (ARM) calls only.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

Citation doesn't work

Please provide us with the following information:

This issue is for a: (mark with an x)

- [x ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

• Deploy app successfully with azd auth and azd up
• Select a card to receive a response.
• Select citation in response

• No document found in separate browser tab.

• Storage container doesn't have any files

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

I need to turn off the shared key in order to be able to use this on an internal sub. I don't think the chat app works if I turn that off. What would have to happen functionally to the app to be able to still work but have shared key turned off?

---

Thanks! We'll be in touch soon.

Allow updating existing documents

Currently if you upload the same document twice, it will create duplicated chunks of the same documents.
Before add a document to the database, we should remove all chunks related to the document using its filename as ID to avoid duplication and allow updates.

Tasks

• Remove all chunks with the same filename of uploaded document in POST /documents
  • for Azure ComosDB for MongoDB vCore path
  • for FAISS path when running locally

Action required: migrate or opt-out of migration to GitHub inside Microsoft

Migrate non-Open Source or non-External Collaboration repositories to GitHub inside Microsoft

In order to protect and secure Microsoft, private or internal repositories in GitHub for Open Source which are not related to open source projects or require collaboration with 3rd parties (customer, partners, etc.) must be migrated to GitHub inside Microsoft a.k.a GitHub Enterprise Cloud with Enterprise Managed User (GHEC EMU).

Action

✍️ Please RSVP to opt-in or opt-out of the migration to GitHub inside Microsoft.

❗Only users with admin permission in the repository are allowed to respond. Failure to provide a response will result to your repository getting automatically archived.🔒

Instructions

Reply with a comment on this issue containing one of the following optin or optout command options below.

✅ Opt-in to migrate

@gimsvc optin --date <target_migration_date in mm-dd-yyyy format>

Example: @gimsvc optin --date 03-15-2023

OR

❌ Opt-out of migration

@gimsvc optout --reason <staging|collaboration|delete|other>

Example: @gimsvc optout --reason staging

Options:

• staging : This repository will ship as Open Source or go public
• collaboration : Used for external or 3rd party collaboration with customers, partners, suppliers, etc.
• delete : This repository will be deleted because it is no longer needed.
• other : Other reasons not specified

Need more help? 🖐️

• Email [email protected]. ✉️
• Post your questions in GitHub inside Microsoft Team in Microsoft Teams. 🗨️

Write readme

• Write readme docs
• Create architecture schema
• Add all related docs and link related Enterprise samples, ie Contoso, https://github.com/Azure-Samples/azure-search-openai-javascript, ...

Failed to deploy (twice)

mac m1 in dev container

node ➜ /workspaces/serverless-chat-langchainjs (main) $ azd auth logi
? Enter a new environment name: dfberry-serverless-langchainjs
  (✓) Done: Downloading Bicep
? Select an Azure Subscription to use:  3. Visual Studio Enterprise Subscription (b57b253a-e19e-4a9c-a0c0-a5062910a749)
? Select an Azure location to use: 44. (US) East US 2 (eastus2)

Packaging services (azd package)

  (✓) Done: Packaging service api
  - Package Output: /tmp/serverless-chat-langchainjs-api-azddeploy-1714949128.zip

  (✓) Done: Packaging service webapp
  - Package Output: dist

SUCCESS: Your application was packaged for Azure in less than a second.

Provisioning Azure resources (azd provision)
Provisioning Azure resources can take some time.

Subscription: Visual Studio Enterprise Subscription (b57b253a-e19e-4a9c-a0c0-a5062910a749)
Location: East US 2

  You can view detailed progress in the Azure Portal:
  https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2Fb57b253a-e19e-4a9c-a0c0-a5062910a749%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdfberry-serverless-langchainjs-1714949133

  (✓) Done: Resource group: rg-dfberry-serverless-langchainjs
  (✓) Done: Static Web App: webapp
  (✓) Done: App Service plan: plan-pu45pechobwso
  (✓) Done: Azure OpenAI: cog-pu45pechobwso
  (✓) Done: Storage account: stpu45pechobwso
  (✓) Done: Search service: srch-pu45pechobwso
  (✓) Done: Function App: func-api-pu45pechobwso

SUCCESS: Your application was provisioned in Azure in 5 minutes 39 seconds.
You can view the resources created under the resource group rg-dfberry-serverless-langchainjs in Azure Portal:
https://portal.azure.com/#@/resource/subscriptions/b57b253a-e19e-4a9c-a0c0-a5062910a749/resourceGroups/rg-dfberry-serverless-langchainjs/overview

Deploying services (azd deploy)

  (✓) Done: Deploying service api
  - Endpoint: https://func-api-pu45pechobwso.azurewebsites.net/

  (x) Failed: Deploying service webapp

ERROR: failed deploying service 'webapp': failing invoking action 'deploy', failed verifying static web app deployment. Still in WaitingForDeployment state

ERROR: error executing step command 'deploy --all': failed deploying service 'webapp': failing invoking action 'deploy', failed verifying static web app deployment. Still in WaitingForDeployment state
node ➜ /workspaces/serverless-chat-langchainjs (main) $ azd up

Packaging services (azd package)

  (✓) Done: Packaging service api
  - Package Output: /tmp/serverless-chat-langchainjs-api-azddeploy-1714950467.zip

  (✓) Done: Packaging service webapp
  - Package Output: dist

SUCCESS: Your application was packaged for Azure in 5 seconds.

Provisioning Azure resources (azd provision)
Provisioning Azure resources can take some time.

Subscription: Visual Studio Enterprise Subscription (b57b253a-e19e-4a9c-a0c0-a5062910a749)
Location: East US 2

  You can view detailed progress in the Azure Portal:
  https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2Fb57b253a-e19e-4a9c-a0c0-a5062910a749%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdfberry-serverless-langchainjs-1714950472

  (✓) Done: Resource group: rg-dfberry-serverless-langchainjs
  (✓) Done: Static Web App: webapp
  (✓) Done: Storage account: stpu45pechobwso
  (✓) Done: App Service plan: plan-pu45pechobwso
  (✓) Done: Search service: srch-pu45pechobwso
  (✓) Done: Azure OpenAI: cog-pu45pechobwso
  (✓) Done: Function App: func-api-pu45pechobwso

SUCCESS: Your application was provisioned in Azure in 2 minutes 16 seconds.
You can view the resources created under the resource group rg-dfberry-serverless-langchainjs in Azure Portal:
https://portal.azure.com/#@/resource/subscriptions/b57b253a-e19e-4a9c-a0c0-a5062910a749/resourceGroups/rg-dfberry-serverless-langchainjs/overview

Deploying services (azd deploy)

  (✓) Done: Deploying service api
  - Endpoint: https://func-api-pu45pechobwso.azurewebsites.net/

  (x) Failed: Deploying service webapp

ERROR: failed deploying service 'webapp': failing invoking action 'deploy', failed verifying static web app deployment. Still in WaitingForDeployment state

ERROR: error executing step command 'deploy --all': failed deploying service 'webapp': failing invoking action 'deploy', failed verifying static web app deployment. Still in WaitingForDeployment state

forked and clone today

git log looks like

node ➜ /workspaces/serverless-chat-langchainjs (main) $ git log
commit 5ac5560821b133489932b6bc6aaad74eb0a82d20 (HEAD -> main, origin/main, origin/HEAD)
Author: Yohan Lasorsa <[email protected]>
Date:   Fri May 3 19:19:55 2024 +0200

    docs: add local architecture schema (#62)

@sinedied @glaucia86

Acceptance criteria checklist (DoD)

The following checklist must be complete before a template is published.

Repository Management

• Standards compliant README.md as the one in the example, is in place
• License is in place. Make sure you choose the correct license
• Security guidelines are in place
• Contribution guidelines are in place
• Code of conduct is in place
• Issue template is in place
• Language, model, and relevant technology topic labels are added, including azd-templates and ai-azd-templates (The latter is being created)
• Repo description is in place, describing the use case and technologies used in the solution

Source code structure and conventions

• GitHub Actions (This refers to .github/workflows/azure-dev.yml or custom workflow to run on a GitHub runner) is in place
• DevContainer (/.devcontainer folder where applicable) configuration is in place
• Infrastructure as code is in place (/infra folder where applicable, manifest files or code generators in the case of Aspire and similar )
• Azure services configuration (/azure.yml file) is in place
• Minimum coverage tests are in place

Functional requirements

• azd up successfully provisions and deploys a functional app
• GitHub Actions run tasks without errors
• DevContainer has been tested locally and runs
• Codespaces run [locally and in browser]
• All tests pass

In the absence of e2e tests,

• The application has been manually tested to work as per the requirement

Security requirements

• Microsoft Managed Identity is implemented
• #72

When a service selected doesn't support Managed Identity, the corresponding issue must have been reported and the security considerations section in the readme, should clearly explain the alternatives.

• Azure Key Vault is a preferred alternative

The following items are not strictly enforced but may prevent the template from being added to the gallery.

Project code follows standard structure, per language. Please check one.

• Yes, follows standards
• No, doesn't follow standards

Code follows recommended style guide

• Yes, follows style guide
• No, doesn't follow style guide

Add tests

• Setup test framework (Playwright?)
• Add E2E tests to cover the 2 base scenarios
  • add new file to ingest by uploading to /index API
  • test Chat UI asking for an answer based on indexed document
• npm test from the repo root runs the tests

Create webapp

• Create webapp package
• Setup SWA CLI to test locally
• Use simplified web component from RAG workshop

Add docs for enhancing security

Currently this sample is designed to be as easy as possible to work one and deploy.
If deployed in an enterprise context, we usually enforce tighter security rules and we want to explain what can be changed in the sample to improve its security:

Tasks

• Create a new docs "Enhance security" in /docs/enhance-security.md that explains how to:
  • Disable exporting keys in output (and also the ability to work locally)
  • Configure vnet (and optionally make the app only available on internal network)
  • Use Azure Key Vault for key and connection string storage

Init Functions API

• Create packages/api folder with a v4 Function app using TypeScript
• Create a README.md explaining about the api folder
• Add vscode files for easy debugging (created when configuring the VS Code Azure Functions extension)
• Include a Codespaces button in the README.md file
• Create stub for POST /chat endpoint
• Create stub for POST /upload endpoint

[docs]: Update `tutorial` folder according to the code sample

Please provide us with the following information:

Update the tutorial folder according to the code sample.
The intention is precisely to create a step-by-step tutorial with the aim of turning it into a Learn Module or Official Documentation.

• 01 - Introduction
  • (PR: #38)
• 02 - Setting Up the Serverless Environment using Azure Functions
  • (PR: #45)
• 03 - Understanding the RAG (Retrieval Augmented Generation) architecture
  • (PR: #45)
• 04 - Preparing and Understanding Language Models: Configuring Azure OpenAI Service and Local Installation of Ollama with Mistral 7B
  • (PR: #45)
• 05 - Developing the chat API
• 06 - Developing the documents API (post documents)
• 07 - Developing the documents API (get documents)
• 08 -

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

info2.txt is missing

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

1. azd up
2. use chat
3. select how to search and book rentals
4. Wait for answer and suggestions
5. from suggestions, select how can I cancel a rental booking
6. from answer, select citiation for info2.txt
7. {"error":"Document not found"}

Any log messages given by the failure

Expected/desired behavior

Citation goes to text.

OS and Version?

Windows 11

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

Implement /upload API

• Implement POST /upload API:
  • API supports uploading 1 file at a time, using regular multipart file content
  • Init Azure SDK through LangChain.js for Embeddings
  • Init CosmosDB LC vector store
  • Load uploaded PDFs using LC PDF integration. Use custom text splitter strategy with 1000 characters and 10% overlap
  • Add documents to CosmosDB store
• Create an api.http file to test this API using REST Client extension

Remove usage of API keys

Supersede #34

Tasks

• Update infra, code and docs so that we only use managed identity for improved security

Implements /documents API

• Implement /documents/<id> GET API
• API connects to blob storage to retrieve a given document
  • Use Functions v4 blob storage connector
• (if not implemented) Add blob storage upload to POST /upload endpoint
• Complete an api.http file to test this API using REST Client extension

Note: this is for later, as an enhancement to the existing app, and not mandatory for initial release.

Update readme components section

Please add a section in "This application is made from multiple components:" to include Azure OpenAI.

Setup CI/CD

• Setup a build-test.yml action that build the app and run tests on every PR/commit targeting main
• Setup a deploy.yml action to deploy the project on CDA demos subscription

microsoft/security-devops-action is integrated to the CI/CD pipeline and the analyzer were ran

Fail to run `azd up` command

Describe the issue:
When executing azd up according to README.md, the error is as follows:

Repro Steps:

1. Open in GitHub Codespaces.
2. Run azd auth login
3. Run azd up

Environment:

• Azd version: azd version 1.9.5 (commit cd2b7af9995d358aab33c782614f801ac1997dde)
• OS: Codespaces, Dev Containers and Windows desktop.

Expected behavior:
Run azd up can successfully.

@jongio and @sinedied for notification.

Add evaluation using LangChain.js tools

• In /test folder, create a new folder for evaluation
• Use LangChain.js criteria evaluation to demonstrate evaluation of RAG results for differents KPIs
• Add docs on how to run and update the evaluations

Error running example locally using Ollama

Describe the issue:
Using Ollama to run the example locally, opening the web http://localhost:8000 chat fails with the following error:

Repro Steps:

1. Run command git clone https://github.com/Azure-Samples/serverless-chat-langchainjs.git
2. Install and run Ollama
3. Run command ollama pull mistral and ollama pull all-minilm:l6-v2
4. Run command npm install
5. Run command npm start
6. Run command npm run upload:docs
7. Open the URL http://localhost:8000

Environments:

• azd version: azd version 1.8.0 (commit 8246323c2472148288be4b3cbc3c424bd046b985)
• os: windows

Expected behavior:
Open the URL http://localhost:8000 in the browser and successfully start chatting with the bot

@sinedied for notification.

Add chat memory

Currently only the last chat message with the user question is fed to the chat model. However, the complete chat history is sent to the server. We should use LangChain's Memory component to integrate the previous messages in the model invocation, with a limit set.

Tasks

• Add BufferMemory with the last 5 messages to the existing chain in /chat endpoint
• Extract the 5 message window value in constants.ts

[docs] Translate tutorial file to other languages

Please provide us with the following information:

Your tutorial looks promising, but have you considered translating it into different languages? If so, do you have a timeframe for that? I'd be willing to help with translating it into Turkish.

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

[docs]: Add Documentation Step - Execute `npm run upload:docs` before deployment to azure

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

Include in the documentation that before deploying the application, the user must execute the command: npm run upload:docs. Why? So that the pdf files can be sent to Azure Blob Storage and thus be able to generate the answers coming from the chat.

After executing this command, the user will be able to get the responses back once the application has been deployed to Azure.

Today the README.md says that: (the actual instruction is under the local section). So in this case we can move it to the deployed application or copy it.

azd auth login
azd up

note: remember that in order for you to be able to read these pdf files after the application has been deployed, you need to provision the Azure services first. Otherwise, you won't be able to upload these files to Blob Storage

Thanks! We'll be in touch soon.

Publish this template to azure samples browser

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

Since this is a "msft" authorized template in Azure-Samples, we may want to publish it to Microsoft learning page: Onboarding Samples to learn.microsoft.com/samples

• add metadata in readme
• submit online form request
• check if this template is in azure samples browser

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

01-introduction.md Docs Feedback

This is looking really good! I've included your original work below but made some minor updates here and there to fit in with what I'll know we'll need for migrating to a Learn tutorial. I'm kind of picky so I apologize in advance (too many years of working with book and magazine publishers :-)).

A few questions and comments to also consider in addition to my modified version below.

1. It's mentioned that Codespaces is recommended. I changed the order of local and codespaces so that codespaces is first. That would follow the recommendation.
2. We shouldn't need to list TypeScript in the pre-reqs for local. The required package would be installed using npm so I removed it below. I also updated the Node.js link since it can be confusing sending them to the downloads folder (lots of options there which is challenging for new users). I changed it to go to the homepage and mentioned "Node.js LTS".
3. Can Powershell run the bash commands? Git Bash and WSL2 can, but I wasn't aware Powershell could do that (I'll have to look more closely at the commands).
4. Great summary for the Codespaces section. Nice and clear and mentions the free 60 hours.
5. I reframed some text to "you", "you'll", etc. to personalize it to the user more instead of using "we", "we'll", etc.
6. Since they can run locally or in a codespace, let's mention that if they want to run it locally they'll need to fork/clone and provide those specific steps. Many new devs won't necessarily know how to do that. I'd explain that by forking it they can get the code into their own GitHub account (since they could simply clone our repo if they really want to). If they're running a codespace (which will do a clone under the covers) they obviously don't need to worry about the fork/clone steps.

Here's the complete readme file with my updates included.

01-introduction.md

Tutorial - Create a Serverless ChatGPT with RAG using LangChain.js and TypeScript

Welcome to the Create a Serverless ChatGPT with RAG using LangChain.js and TypeScript tutorial.

This tutorial will guide you through the process of creating a serverless ChatGPT and RAG (Retrieval-Augmented Generation) application using LangChain.js, Azure Functions, Azure Cosmos DB for MongoDB vCore, Azure Blob Storage, and Azure Static Web Apps.

The chatbot will answer questions based on a set of enterprise documents uploaded from a fictional company called Contoso Real Estate.

Here's an example of the application in action:

The goal of the tutorial is to provide you with a hands-on experience building a serverless application using Azure Services and LangChain.js. You'll be guided through each step of the process from setting up the environment to deploying the application.

The frontend of the application is provided so that you can focus on the backend code and technologies.

Prerequisites

You can follow tutorial and run the application using one of the following options:

• Run the application locally on your machine
• Run the application using Codespaces.

It is highly recommended that you use Codespaces for this tutorial. Codespaces is a cloud-based tool that enables you to run development environments without installing any tools on your computer. By using Codespaces you can focus on the development process without worrying about the environment setup.

Run using Codespaces

If you decide to continue using Codespaces, you can follow the steps described in the README.md file at the root of the project.

Note: If you are using Codespaces, you don't need to install any of the prerequisites mentioned above. Codespaces already has all the necessary tools installed. Codespaces can be used for free for up to 60 hours per month, and this is renewed every month.

Run Locally

If you choose to use a local environment, you will need to install:

• Node.js LTS
• Visual Studio Code
• Azure Functions Core Tools
• Git
• Azure Developer CLI

If you're a Windows user, you'll need to install PowerShell, Git Bash or WSL2 to run the bash commands.

Project Overview

Building AI applications can be complex and time-consuming. By using LangChain.js and Azure serverless technologies, you can greatly simplify the process. This application is a chatbot that uses a set of enterprise documents to generate AI responses to user queries.

The code sample includes sample data to make trying the application quick and easy, but feel free to replace it with your own. You'll use a fictitious company called Contoso Real Estate, and the experience allows its customers to ask support questions about the usage of the company's products. The sample data includes a set of documents that describes the company's terms of service, privacy policy, and support guide.

Understanding the project architecture

The architecture of the project is shown in the following diagram:

To understand the architecture of the project, let's break it down into the application's individual components:

1. Web App:

   • The user interface for the chatbot is a web application built with Lit (a library for building web components) and hosted using Azure Static Web Apps. It renders a chat interface for users to interact with and ask questions.
   • The code is located in the packages/webapp folder.

2. Serverless API:

   • When a user submits a query through the web app, it is sent via HTTP to an API built using Azure Functions.
   • The API uses LangChain.js to process the query.
   • The API handles the logic of ingesting enterprise documents and generating responses to the chat queries.
   • The code for this functionality will be shown later in the tutorial and is located in the packages/api folder.

3. Database:

   • Text extracted from the documents and the vectors generated by LangChain.js is stored in Azure Cosmos DB for MongoDB vCore.
   • The database allows for the storage and retrieval of text chunks using vector search, which enables quick and relevant responses based on the user's queries.

4. File Storage:

   • The source documents such as terms of service, privacy policy, and support guides for the Contoso Real Estate are stored in Azure Blob Storage. This is where the PDF documents are uploaded and retrieved from.

5. Azure OpenAI Service:

   • This service is where the AI Model (a Large Language Model or LLM) is hosted. The model is capable of understanding and generating natural language. This is used to embed text chunks or generate answers based on the vector search from the database.

Let's examine the application flow based on the architecture diagram:

• A user interacts with the chat interface in the web app.
• The web app sends the user's query to the Serverless API via HTTP calls.
• The Serverless API interacts with Azure OpenAI Service to generate a response, using the data from Azure Cosmos DB for MongoDB vCore.
• If there's a need to reference the original documents, Azure Blob Storage is used to retrieve the PDF documents.
• The generated response is sent back to the web app and displayed to the user.

The architecture is based on the RAG (Retrieval-Augmented Generation) architecture. This architecture combines the ability to retrieve information from a database with the ability to generate text from a language model. You'll learn more about RAG later in the tutorial.

Executing the Project

Now that you understand the project's architecture, let's run it!

Once you have forked and cloned the project, use the starter branch to continue with the tutorial. The main branch contains the finished project if you would like to view it.

To execute the project, follow these steps:

1. Install the project dependencies:

npm install

3. To run the project's frontend, run the following command:

start:webapp

At this point, don't worry about the other scripts in the package.json file at the root of the project. They will be used throughout the tutorial.

4. Open your browser and go to http://localhost:8000. The application will be displayed as shown in the image below:

Next Steps

In the next exercise, you'll begin creating the API using Azure Functions.

▶ Next Step: Setting Up the Serverless Environment using Azure Functions

Embedding model change

According to the article, If I want to change the embedding model from text-embedding-ada-002 to text-embedding-3-small. Which code shall I modify?

Your tutorial inspires my interests on RAG very much. Thank you!!

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [X] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Allow to run the sample fully locally using Ollama and Faiss

• Create an adaptation layer for the models and DB (use local as a default if .env not provided)
• Add docs for Ollama installation and models

Support Terraform in a new branch

Note: to be considered once the sample is complete.

AZD could still be used as the provisioning and deployment tool: https://learn.microsoft.com/azure/developer/azure-developer-cli/use-terraform-for-azd

Muito boa

Upload pdf by API

I want to upload a pdf file by the API (POST https://x.x.x.x/api/documents/) using the following curl command. But it returned 401 unauthorized.

curl --location 'https://xxx.app.github.dev/api/documents/' \ --header 'Content-Type: multipart/form-data' \ --form '=@"/C:/Users/R008128.pdf"'

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [X ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

401 Unauthorized

Expected/desired behavior

OS and Version?

Azure cloud

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

Add cost estimation section

See https://github.com/Azure-Samples/azure-search-openai-demo?tab=readme-ov-file#cost-estimation

Tasks

• create docs/costs.md
• add "Cost estimation" section in Azure deployment section in the readme, and link to the cost docs

Create infrastructure

• Create bicep templates
• Create azure.yml
• Allow reusing existing OpenAI service or OpenAI API key
• Auto-generate .env file at the end of provisioning

Export-ModuleMember error

Please provide us with the following information:

This issue is for a: (mark with an x)

- [x ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

azd auth login
azd up

Any log messages given by the failure

Expected/desired behavior

Expected to get a web service deployed.

OS and Version?

Windows 10

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

Implement /chat API

• Implement POST /chat API
  • API follows the AI Chat protocol
  • Init Azure SDK through LangChain.js for Embeddings
  • Init CosmosDB LC vector store
  • Generate completion using a chain, see example here
  • Generate stream response, see this example implementation and here
  • Add test to api.http file for this API