Giter Site home page Giter Site logo

call-gpt's Introduction

Call GPT: Generative AI Phone Calling

Wouldn't it be neat if you could build an app that allowed you to chat with ChatGPT on the phone?

Twilio gives you a superpower called Media Streams. Media Streams provides a Websocket connection to both sides of a phone call. You can get audio streamed to you, process it, and send audio back.

This app serves as a demo exploring two services:

  • Deepgram for Speech to Text and Text to Speech
  • OpenAI for GPT prompt completion

These service combine to create a voice application that is remarkably better at transcribing, understanding, and speaking than traditional IVR systems.

Features:

  • 🏁 Returns responses with low latency, typically 1 second by utilizing streaming.
  • ❗️ Allows the user to interrupt the GPT assistant and ask a different question.
  • πŸ“” Maintains chat history with GPT.
  • πŸ› οΈ Allows the GPT to call external tools.

Setting up for Development

Prerequisites

Sign up for the following services and get an API key for each:

If you're hosting the app locally, we also recommend using a tunneling service like ngrok so that Twilio can forward audio to your app.

1. Start Ngrok

Start an ngrok tunnel for port 3000:

ngrok http 3000

Ngrok will give you a unique URL, like abc123.ngrok.io. Copy the URL without http:// or https://. You'll need this URL in the next step.

2. Configure Environment Variables

Copy .env.example to .env and configure the following environment variables:

# Your ngrok or server URL
# E.g. 123.ngrok.io or myserver.fly.dev (exlude https://)
SERVER="yourserverdomain.com"

# Service API Keys
OPENAI_API_KEY="sk-XXXXXX"
DEEPGRAM_API_KEY="YOUR-DEEPGRAM-API-KEY"

# Configure your Twilio credentials if you want
# to make test calls using '$ npm test'.
TWILIO_ACCOUNT_SID="YOUR-ACCOUNT-SID"
TWILIO_AUTH_TOKEN="YOUR-AUTH-TOKEN"
FROM_NUMBER='+12223334444'
TO_NUMBER='+13334445555'

3. Install Dependencies with NPM

Install the necessary packages:

npm install

4. Start Your Server in Development Mode

Run the following command:

npm run dev

This will start your app using nodemon so that any changes to your code automatically refreshes and restarts the server.

5. Configure an Incoming Phone Number

Connect a phone number using the Twilio Console.

You can also use the Twilio CLI:

twilio phone-numbers:update +1[your-twilio-number] --voice-url=https://your-server.ngrok.io/incoming

This configuration tells Twilio to send incoming call audio to your app when someone calls your number. The app responds to the incoming call webhook with a Stream TwiML verb that will connect an audio media stream to your websocket server.

Application Workflow

CallGPT coordinates the data flow between multiple different services including Deepgram, OpenAI, and Twilio Media Streams: Call GPT Flow

Modifying the ChatGPT Context & Prompt

Within gpt-service.js you'll find the settings for the GPT's initial context and prompt. For example:

this.userContext = [
  { "role": "system", "content": "You are an outbound sales representative selling Apple Airpods. You have a youthful and cheery personality. Keep your responses as brief as possible but make every attempt to keep the caller on the phone without being rude. Don't ask more than 1 question at a time. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous. Speak out all prices to include the currency. Please help them decide between the airpods, airpods pro and airpods max by asking questions like 'Do you prefer headphones that go in your ear or over the ear?'. If they are trying to choose between the airpods and airpods pro try asking them if they need noise canceling. Once you know which model they would like ask them how many they would like to purchase and try to get them to place an order. Add a 'β€’' symbol every 5 to 10 words at natural pauses where your response can be split for text to speech." },
  { "role": "assistant", "content": "Hello! I understand you're looking for a pair of AirPods, is that correct?" },
],

About the system Attribute

The system attribute is background information for the GPT. As you build your use-case, play around with modifying the context. A good starting point would be to imagine training a new employee on their first day and giving them the basics of how to help a customer.

There are some context prompts that will likely be helpful to include by default. For example:

  • You have a [cheerful, wise, empathetic, etc.] personality.
  • Keep your responses as brief as possible but make every attempt to keep the caller on the phone without being rude.
  • Don't ask more than 1 question at a time.
  • Don't make assumptions about what values to plug into functions.
  • Ask for clarification if a user request is ambiguous.
  • Add a 'β€’' symbol every 5 to 10 words at natural pauses where your response can be split for text to speech.

These context items help shape a GPT so that it will act more naturally in a phone conversation.

The β€’ symbol context in particular is helpful for the app to be able to break sentences into natural chunks. This speeds up text-to-speech processing so that users hear audio faster.

About the content Attribute

This attribute is your default conversations starter for the GPT. However, you could consider making it more complex and customized based on personalized user data.

In this case, our bot will start off by saying, "Hello! I understand you're looking for a pair of AirPods, is that correct?"

Using Function Calls with GPT

You can use function calls to interact with external APIs and data sources. For example, your GPT could check live inventory, check an item's price, or place an order.

How Function Calling Works

Function calling is handled within the gpt-service.js file in the following sequence:

  1. gpt-service loads function-manifest.js and requires (imports) all functions defined there from the functions directory. Our app will call these functions later when GPT gives us a function name and parameters.
tools.forEach((tool) => {
  const functionName = tool.function.name;
  availableFunctions[functionName] = require(`../functions/${functionName}`);
});
  1. When we call GPT for completions, we also pass in the same function-manifest JSON as the tools parameter. This allows the GPT to "know" what functions are available:
const stream = await this.openai.chat.completions.create({
  model: 'gpt-4',
  messages: this.userContext,
  tools, // <-- function-manifest definition
  stream: true,
});
  1. When the GPT responds, it will send us a stream of chunks for the text completion. The GPT will tell us whether each text chunk is something to say to the user, or if it's a tool call that our app needs to execute. This is indicated by the deltas.tool_calls key:
if (deltas.tool_calls) {
  // handle function calling
}
  1. Once we have gathered all of the stream chunks about the tool call, our application can run the actual function code that we imported during the first step. The function name and parameters are provided by GPT:
const functionToCall = availableFunctions[functionName];
const functionResponse = functionToCall(functionArgs);
  1. As the final step, we add the function response data into the conversation context like this:
this.userContext.push({
  role: 'function',
  name: functionName,
  content: functionResponse,
});

We then ask the GPT to generate another completion including what it knows from the function call. This allows the GPT to respond to the user with details gathered from the external data source.

Adding Custom Function Calls

You can have your GPT call external data sources by adding functions to the /functions directory. Follow these steps:

  1. Create a function (e.g. checkInventory.js in /functions)
  2. Within checkInventory.js, write a function called checkInventory.
  3. Add information about your function to the function-manifest.js file. This information provides context to GPT about what arguments the function takes.

Important: Your function's name must be the same as the file name that contains the function (excluding the .js extension). For example, our function is called checkInventory so we have named the the file checkInventory.js, and set the name attribute in function-manifest.js to be checkInventory.

Example function manifest entry:

{
  type: "function",
  function: {
    name: "checkInventory",
    say: "Let me check our inventory right now.",
    description: "Check the inventory of airpods, airpods pro or airpods max.",
    parameters: {
      type: "object",
      properties: {
        model: {
          type: "string",
          "enum": ["airpods", "airpods pro", "airpods max"],
          description: "The model of airpods, either the airpods, airpods pro or airpods max",
        },
      },
      required: ["model"],
    },
    returns: {
      type: "object",
      properties: {
        stock: {
          type: "integer",
          description: "An integer containing how many of the model are in currently in stock."
        }
      }
    }
  },
}

Using say in the Function Manifest

The say key in the function manifest allows you to define a sentence for the app to speak to the user before calling a function. For example, if a function will take a long time to call you might say "Give me a few moments to look that up for you..."

Receiving Function Arguments

When ChatGPT calls a function, it will provide an object with multiple attributes as a single argument. The parameters included in the object are based on the definition in your function-manifest.js file.

In the checkInventory example above, model is a required argument, so the data passed to the function will be a single object like this:

{
  model: "airpods pro"
}

For our placeOrder function, the arguments passed will look like this:

{
  model: "airpods pro",
  quantity: 10
}

Returning Arguments to GPT

Your function should always return a value: GPT will get confused when the function returns nothing, and may continue trying to call the function expecting an answer. If your function doesn't have any data to return to the GPT, you should still return a response with an instruction like "Tell the user that their request was processed successfully." This prevents the GPT from calling the function repeatedly and wasting tokens.

Any data that you return to the GPT should match the expected format listed in the returns key of function-manifest.js.

Utility Scripts for Placing Calls

The scripts directory contains two files that allow you to place test calls:

  • npm run inbound will place an automated call from a Twilio number to your app and speak a script. You can adjust this to your use-case, e.g. as an automated test.
  • npm run outbound will place an outbound call that connects to your app. This can be useful if you want the app to call your phone so that you can manually test it.

Using Eleven Labs for Text to Speech

Replace the Deepgram API call and array transformation in tts-service.js with the following call to Eleven Labs. Note that sometimes Eleven Labs will hit a rate limit (especially on the free trial) and return 400 errors with no audio (or a clicking sound).

try {
  const response = await fetch(
    `https://api.elevenlabs.io/v1/text-to-speech/21m00Tcm4TlvDq8ikWAM/stream?output_format=ulaw_8000&optimize_streaming_latency=3`,
    {
      method: 'POST',
      headers: {
        'xi-api-key': process.env.XI_API_KEY,
        'Content-Type': 'application/json',
        accept: 'audio/wav',
      },
      body: JSON.stringify({
        model_id: process.env.XI_MODEL_ID,
        text: partialResponse,
      }),
    }
  );
  
  if (response.status === 200) {
    const audioArrayBuffer = await response.arrayBuffer();
    this.emit('speech', partialResponseIndex, Buffer.from(audioArrayBuffer).toString('base64'), partialResponse, interactionCount);
  } else {
    console.log('Eleven Labs Error:');
    console.log(response);
  }
} catch (err) {
  console.error('Error occurred in XI LabsTextToSpeech service');
  console.error(err);
}

Testing with Jest

Repeatedly calling the app can be a time consuming way to test your tool function calls. This project contains example unit tests that can help you test your functions without relying on the GPT to call them.

Simple example tests are available in the /test directory. To run them, simply run npm run test.

Deploy via Fly.io

Fly.io is a hosting service similar to Heroku that simplifies the deployment process. Given Twilio Media Streams are sent and received from us-east-1, it's recommended to choose Fly's Ashburn, VA (IAD) region.

Deploying to Fly.io is not required to try the app, but can be helpful if your home internet speed is variable.

Modify the app name fly.toml to be a unique value (this must be globally unique).

Deploy the app using the Fly.io CLI:

fly launch

fly deploy

Import your secrets from your .env file to your deployed app:

fly secrets import < .env

call-gpt's People

Contributors

akiani avatar cweems avatar dependabot[bot] avatar mdvickst avatar mrmorin1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

call-gpt's Issues

keeps ringing

setting up Call-gpt went great. but when i call my twilio number i keeps ringing ,no answer then hangs up. On ngrok im getting 200 OK. i did run npm test everything was great. Am i missing something or what could be the problem?

Different TTS models

Great work, thank you for your contribution!

For cost purposes I tried using Neets.ai and Deepgram's TTS but cant get them to work. Terminal seems to log GPT-->TTS but not TTS --> Twilio despite following the docs and matching encoding. Any advice would be greatly appreciated.

Thanks again!

Abe

Getting this error a lot on latest code - tested -June 8

After taking the latest , getting this error a lot

To help you decide, let me ask: Do you need noise cancellation in your headphones?
GPT -> user context length: 7
RangeError: Maximum call stack size exceeded
at TextToSpeechService.generate (/Users/gauravkesharwani/Repo/call-gpt/services/tts-service.js:36:44)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Forward call

How would one create a function to forward the call to another number? Let's say gpt hears some emergency keywords (like "I'm hurt" or something like that). How would i forward the using twilio webhook or restapi via the twilio Studio or another way.

Summarize GPT Chat history as more queries are added to the conversation

Currently, responses get increasingly slower as the chat context grows. We should implement a gradual rolloff of older queries that get summarized by GPT and then added to the end of the conversation.

This may require some testing to understand the optimal conversation context size based on quality of results and speed to response.

1s Latency Definitiion

Tried out the project, very impressed. Thanks for open sourcing. Quick question on latency.

Noticed a minimum latency of at least 3-4s. I am measuring latency as delay between when the human speaks and when the AI responds. This was with everything deployed on fly.io in Ashburn using the exact demo as instructed.

Looks like the biggest bottleneck is the request from Twilio -> Fly.io and Fly.io -> Twilio. Second biggest bottleneck looks like transcription via deepgram.

The ReadMe suggests a latency of 1sβ€”can you clarify the definition of latency here? Is that just looking at gpt response + TTS?

Any ideas on how to reduce latency? Is there a roadmap for this project we can follow somewhere?

Interruption handling.

The project is very impressive.

I will break it down to two issues,

  1. When we interrupt multiple times to the bot, it generates a new response and then it says the last response but also repeat some of the pending responses before interrupt.
  2. Also I think it doesn't wait for the user to complete the sentence, l saw the settings for pause , but sometime user is thinking of right word to complete the sentence, at that time it generates the full response, I think deepgram service needs some changes to handle sentence completion or reply with hmm, yep etc

Text to Speech is not working !

i have followed every instructions carefully but when i run npm run outbound it works fine for speech to text but it does not respond with text to speech output on call

image

and i am using

image

Deepgram for TTS

I got Deepgram to somewhat work for TTS but it only supports "mulaw" which works with twilio but there is an odd click/noise each time a new audio file starts playback.

Anyone been able to get it to work smoothly? maybe encoding the chunks and then sending to base64 ?

Deegram these days is a much better choices then 11lab, cheaper and faster for TTS.

no response

i can't get it to work. i have set it up on fly.io, got all the api keys, upgraded eleven labs, everything is set in .env. when i call it, it establishes a websocket connection, it gets an incoming request from twilio, it response with the connection endpoint, it opens a new websocket connection, and then nothing happens, or at least i am not getting any reponses. tried everything with chatgpt but i am out of ideas on what the issue could be. any pointers?

Explicitly order outgoing audio chunks to avoid race condition

We send chunks of text to ElevenLabs that are different lengths. Generally speaking, the STT recordings come back in the right order. But in some circumstances a very short recording might be processed faster than a longer one and arrive first, even though it was meant to be played later.

We should create a cache for ElevenLabs recordings and re-order outgoing recordings to Twilio Media Streams as necessary so that they follow the correct sequence.

Got the repo kinda working

Right now these are my logs from a recent call

Server running on port 3000
Twilio -> Starting Media Stream for MZ1cbaeb64da6297ccb08a8cf316d2fe6c
Interaction 1: TTS -> TWILIO: Hello! I understand you're looking for a pair of AirPods, is that correct?
Twilio -> Audio completed mark (6): a85266e3-9158-4d27-8b46-6d72faa1416e
UtteranceEnd received before speechFinal, emit the text collected so far:  Hi there. Can you hear me?
Interaction 0 – STT -> GPT:  Hi there. Can you hear me?
Interaction 0: GPT -> TTS: Yes, I can hear you loud and clear β€’
Interaction 0: TTS -> TWILIO: Yes, I can hear you loud and clear β€’
Twilio -> Audio completed mark (272): dc2fe939-fbc0-4f0f-93c0-595cd7613ed9
Interaction 0: GPT -> TTS:  How may I assist you today with your AirPods purchase?
GPT -> user context length: 5
Interaction 0: TTS -> TWILIO:  How may I assist you today with your AirPods purchase?
Twilio -> Audio completed mark (273): 3149e0b8-52d4-420a-a6b2-e9c7d1c80fcc
STT -> Deepgram connection closed
Twilio -> Media stream MZ1cbaeb64da6297ccb08a8cf316d2fe6c ended.
[nodemon] restarting due to changes...
[nodemon] starting `ts-node src/app.ts`
Server running on port 3000
Twilio -> Starting Media Stream for MZ60a84bb1246cfd7160de45c4fbce614d
Interaction 1: TTS -> TWILIO: Hello! I understand you're looking for a pair of AirPods, is that correct?
Twilio -> Audio completed mark (10): 5f5b7ca0-e6fa-4d0d-bb6c-3507eaea5928
STT -> Deepgram connection closed
Twilio -> Media stream MZ60a84bb1246cfd7160de45c4fbce614d ended.

I get no audio on the call, like no audio whatsoever but I do see some transcripts in the console, I honestly can't seem to understand what the issue might be as there are no errors, it seems like deepgram closes conection out of the blue for one thing and also TTS never sends the audio to the actual call. Any ideas?

Training Chatgpt with PDF, Doc or Website Scraper?

Hi, I just come across this project and tested it out. It worked wonder. I am wondering if we are able to train the Chatgpt with our own data such as PDF, Doc , text files or Website Scraper?

Thank you

how i can run this code and voice call to gpt

anyone can quickly tell me how i can run this code that i can conversation to gpt on my phone number
i also follow the steps of readme.MD i have run the server of using (npm run dev
) this command then i run this command (twilio phone-numbers:update +1[your-twilio-number] --voice-url=https://your-server.ngrok.io/incoming
) and then i make call of my number to twilio number but after one 1-2 second call has drop and chatgpt
donot connect with me ?? any one that can tell how i can run the code and how i talk to gpt

Wrong inputs in function call

this.updateUserContext('function', functionName, functionResponse);

The function updateUserContext(name, role, text) has the following order of params, but in the function call when using tools within the gpt-service, you have the ordering as this.updateUserContext('function', functionName, functionResponse); which is (role, name, text). This causes all of the tool fns to throw since it's expecting a role type of function but you are passing in the function name instead.

<Record> Flag on TwiML not working.

So i tried few combination from - https://www.twilio.com/docs/voice/twiml/stream

I tried having the flag set in different location while editing app.js, either errors the app, or does not record on Twilio.

app.post('/incoming', (req, res) => {
  res.status(200);
  res.type('text/xml');
  res.end(`
  <Response>
    // tried here
    <Connect>
      // tried here
      <Stream url="wss://${process.env.SERVER}/connection" Record="true" />
      <Record transcribe="true" playBeep="false" />
    </Connect>
  </Response>
  `);
}); 

i get this error anyone can tell me how can i solve this!

node --no-deprecation app.js
Server running on port 3000
Twilio -> Starting Media Stream for MZeb5d94dbc813cf20f227ed2a7163312c
Interaction 1: TTS -> TWILIO: Hello! I understand you're looking for a pair of AirPods, is that correct?
Twilio -> Interruption, Clearing stream
Interaction 0 – STT -> GPT: Hello?
STT -> Speech was already final when UtteranceEnd recevied
Twilio -> Audio completed mark (195): 3898e4b5-4a15-4b41-871f-fd3c47b8ee77
Interaction 0: GPT -> TTS: Hi there! Thanks for calling in today. β€’
Interaction 0: GPT -> TTS: How can I assist you with your AirPods selection? β€’
Interaction 0: TTS -> TWILIO: Hi there! Thanks for calling in today. β€’
Interaction 0: GPT -> TTS: Do you prefer headphones that go in your ear β€’
Interaction 0: GPT -> TTS: or over the ear?
GPT -> user context length: 5
Interaction 0: TTS -> TWILIO: How can I assist you with your AirPods selection? β€’
Interaction 0: TTS -> TWILIO: Do you prefer headphones that go in your ear β€’
Interaction 0: TTS -> TWILIO: or over the ear?
Twilio -> Audio completed mark (467): 0ece6179-11ff-4a59-a4e9-eda039dabe4b
Twilio -> Audio completed mark (596): accf6e83-7e3c-4af4-b8d4-27bf3eb5ffa2
Twilio -> Audio completed mark (697): 9c2d998d-a0b5-43d6-860a-b4fd98ea9bf7
Twilio -> Audio completed mark (754): bb3228de-18f9-4019-b31d-f4119f48bc44
Interaction 1 – STT -> GPT: Can you tell me
Interaction 1: GPT -> TTS: Absolutely! β€’
Interaction 1: TTS -> TWILIO: Absolutely! β€’
Interaction 1: GPT -> TTS: Would you like to know more about the in-ear AirPods and AirPods Pro, β€’
Twilio -> Interruption, Clearing stream
Twilio -> Audio completed mark (1109): bd21b254-3f01-49b1-8119-08d3dcddffa0
Interaction 1: GPT -> TTS: or the over-the-ear AirPods Max?
GPT -> user context length: 7
Interaction 1: TTS -> TWILIO: Would you like to know more about the in-ear AirPods and AirPods Pro, β€’
Twilio -> Interruption, Clearing stream
Interaction 1: TTS -> TWILIO: or the over-the-ear AirPods Max?
Twilio -> Audio completed mark (1171): 44c7d4e8-1820-42e1-8456-8087fe4b8b92
UtteranceEnd received before speechFinal, emit the text collected so far: one headphone price?
Interaction 2 – STT -> GPT: one headphone price?
Twilio -> Audio completed mark (1280): ea654842-6155-4e89-a724-dca99bdbd21b
Warning: Double function arguments returned by OpenAI: {"model": "airpods"}{"model": "airpods pro"}{"model": "airpods max"}
Interaction 2: GPT -> TTS: Let me check the price, one moment.
GPT -> called checkPrice function
**C:\Python Important\call-gpt-main\node_modules\openai\error.js:43
return new BadRequestError(status, error, message, headers);
^

BadRequestError: 400 'checkPrice' is not one of ['system', 'assistant', 'user', 'function'] - 'messages.8.role'**
at APIError.generate (C:\Python Important\call-gpt-main\node_modules\openai\error.js:43:20)
at OpenAI.makeStatusError (C:\Python Important\call-gpt-main\node_modules\openai\core.js:251:33)
at OpenAI.makeRequest (C:\Python Important\call-gpt-main\node_modules\openai\core.js:290:30)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async GptService.completion (C:\Python Important\call-gpt-main\services\gpt-service.js:55:20)
at async GptService.completion (C:\Python Important\call-gpt-main\services\gpt-service.js:114:9) {
status: 400,
headers: {
'access-control-allow-origin': '*',
'alt-svc': 'h3=":443"; ma=86400',
'cf-cache-status': 'DYNAMIC',
'cf-ray': '877c5ba83ad621e3-KHI',
connection: 'keep-alive',
'content-length': '203',
'content-type': 'application/json',
date: 'Sun, 21 Apr 2024 09:30:54 GMT',
'openai-organization': 'user-gntjrfo4a4rpvvm5p9udpbnq',
'openai-processing-ms': '40',
'openai-version': '2020-10-01',
server: 'cloudflare',
'set-cookie': '__cf_bm=iKylCws.LiQAZjpYO7fLbZyeMgMRa8pUZT7aUFV6k9Q-1713691854-1.0.1.1-kKpI9bMcsYZXFTmzd2m5KozjqvGMkwwPYslFKJnFMLX.Fw4md0fCxEe0B0RWxPXenKhGQw6MkE7maRZ_osYOpg; path=/; expires=Sun, 21-Apr-24 10:00:54 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, _cfuvid=DFu1pwBHgbgfvXVgiE_thT8EI4OpFrDNR6GdEkJB8zA-1713691854483-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None',
'strict-transport-security': 'max-age=15724800; includeSubDomains',
'x-ratelimit-limit-requests': '500',
'x-ratelimit-limit-tokens': '300000',
'x-ratelimit-remaining-requests': '499',
'x-ratelimit-remaining-tokens': '299618',
'x-ratelimit-reset-requests': '120ms',
'x-ratelimit-reset-tokens': '76ms',
'x-request-id': 'req_669c8e2baa1c400655a23fd6ab91b696'
},
error: {
message: "'checkPrice' is not one of ['system', 'assistant', 'user', 'function'] - 'messages.8.role'",
type: 'invalid_request_error',
param: null,
code: null
},
code: null,
param: null,
STT -> Speech was already final when UtteranceEnd recevied
Twilio -> Audio completed mark (195): 3898e4b5-4a15-4b41-871f-fd3c47b8ee77
Interaction 0: GPT -> TTS: Hi there! Thanks for calling in today. β€’
Interaction 0: GPT -> TTS: How can I assist you with your AirPods selection? β€’
Interaction 0: TTS -> TWILIO: Hi there! Thanks for calling in today. β€’
Interaction 0: GPT -> TTS: Do you prefer headphones that go in your ear β€’
Interaction 0: GPT -> TTS: or over the ear?
GPT -> user context length: 5
Interaction 0: TTS -> TWILIO: How can I assist you with your AirPods selection? β€’
Interaction 0: TTS -> TWILIO: Do you prefer headphones that go in your ear β€’
Interaction 0: TTS -> TWILIO: or over the ear?
Twilio -> Audio completed mark (467): 0ece6179-11ff-4a59-a4e9-eda039dabe4b
Twilio -> Audio completed mark (596): accf6e83-7e3c-4af4-b8d4-27bf3eb5ffa2
Twilio -> Audio completed mark (697): 9c2d998d-a0b5-43d6-860a-b4fd98ea9bf7
Twilio -> Audio completed mark (754): bb3228de-18f9-4019-b31d-f4119f48bc44
Interaction 1 – STT -> GPT: Can you tell me
Interaction 1: GPT -> TTS: Absolutely! β€’
Interaction 1: TTS -> TWILIO: Absolutely! β€’
Interaction 1: GPT -> TTS: Would you like to know more about the in-ear AirPods and AirPods Pro, β€’
Twilio -> Interruption, Clearing stream
Twilio -> Audio completed mark (1109): bd21b254-3f01-49b1-8119-08d3dcddffa0
Interaction 1: GPT -> TTS: or the over-the-ear AirPods Max?
GPT -> user context length: 7
Interaction 1: TTS -> TWILIO: Would you like to know more about the in-ear AirPods and AirPods Pro, β€’
Twilio -> Interruption, Clearing stream
Interaction 1: TTS -> TWILIO: or the over-the-ear AirPods Max?
Twilio -> Audio completed mark (1171): 44c7d4e8-1820-42e1-8456-8087fe4b8b92
UtteranceEnd received before speechFinal, emit the text collected so far: one headphone price?
Interaction 2 – STT -> GPT: one headphone price?
Twilio -> Audio completed mark (1280): ea654842-6155-4e89-a724-dca99bdbd21b
Warning: Double function arguments returned by OpenAI: {"model": "airpods"}{"model": "airpods pro"}{"model": "airpods max"}
Interaction 2: GPT -> TTS: Let me check the price, one moment.
GPT -> called checkPrice function
C:\Python Important\call-gpt-main\node_modules\openai\error.js:43
return new BadRequestError(status, error, message, headers);
^

BadRequestError: 400 'checkPrice' is not one of ['system', 'assistant', 'user', 'function'] - 'messages.8.role'
at APIError.generate (C:\Python Important\call-gpt-main\node_modules\openai\error.js:43:20)
at OpenAI.makeStatusError (C:\Python Important\call-gpt-main\node_modules\openai\core.js:251:33)
at OpenAI.makeRequest (C:\Python Important\call-gpt-main\node_modules\openai\core.js:290:30)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async GptService.completion (C:\Python Important\call-gpt-main\services\gpt-service.js:55:20)
at async GptService.completion (C:\Python Important\call-gpt-main\services\gpt-service.js:114:9) {
status: 400,
headers: {
'access-control-allow-origin': '*',
'alt-svc': 'h3=":443"; ma=86400',
'cf-cache-status': 'DYNAMIC',
'cf-ray': '877c5ba83ad621e3-KHI',
connection: 'keep-alive',
'content-length': '203',
'content-type': 'application/json',
date: 'Sun, 21 Apr 2024 09:30:54 GMT',
'openai-organization': 'user-gntjrfo4a4rpvvm5p9udpbnq',
'openai-processing-ms': '40',
'openai-version': '2020-10-01',
server: 'cloudflare',
'set-cookie': '__cf_bm=iKylCws.LiQAZjpYO7fLbZyeMgMRa8pUZT7aUFV6k9Q-1713691854-1.0.1.1-kKpI9bMcsYZXFTmzd2m5KozjqvGMkwwPYslFKJnFMLX.Fw4md0fCxEe0B0RWxPXenKhGQw6MkE7maRZ_osYOpg; path=/; expires=Sun, 21-Apr-24 10:00:54 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, _cfuvid=DFu1pwBHgbgfvXVgiE_thT8EI4OpFrDNR6GdEkJB8zA-1713691854483-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None',
'strict-transport-security': 'max-age=15724800; includeSubDomains',
'x-ratelimit-limit-requests': '500',
'x-ratelimit-limit-tokens': '300000',
'x-ratelimit-remaining-requests': '499',
'x-ratelimit-remaining-tokens': '299618',
'x-ratelimit-reset-requests': '120ms',
'x-ratelimit-reset-tokens': '76ms',
'x-request-id': 'req_669c8e2baa1c400655a23fd6ab91b696'
},
error: {
message: "'checkPrice' is not one of ['system', 'assistant', 'user', 'function'] - 'messages.8.role'",
type: 'invalid_request_error',
param: null,
code: null
type: 'invalid_request_error'
}

STT -> Speech was already final when UtteranceEnd recevied
Twilio -> Audio completed mark (195): 3898e4b5-4a15-4b41-871f-fd3c47b8ee77
Interaction 0: GPT -> TTS: Hi there! Thanks for calling in today. β€’
Interaction 0: GPT -> TTS: How can I assist you with your AirPods selection? β€’
Interaction 0: TTS -> TWILIO: Hi there! Thanks for calling in today. β€’
Interaction 0: GPT -> TTS: Do you prefer headphones that go in your ear β€’
Interaction 0: GPT -> TTS: or over the ear?
GPT -> user context length: 5
Interaction 0: TTS -> TWILIO: How can I assist you with your AirPods selection? β€’
Interaction 0: TTS -> TWILIO: Do you prefer headphones that go in your ear β€’
Interaction 0: TTS -> TWILIO: or over the ear?
Twilio -> Audio completed mark (467): 0ece6179-11ff-4a59-a4e9-eda039dabe4b
Twilio -> Audio completed mark (596): accf6e83-7e3c-4af4-b8d4-27bf3eb5ffa2
Twilio -> Audio completed mark (697): 9c2d998d-a0b5-43d6-860a-b4fd98ea9bf7
Twilio -> Audio completed mark (754): bb3228de-18f9-4019-b31d-f4119f48bc44
Interaction 1 – STT -> GPT: Can you tell me
Interaction 1: GPT -> TTS: Absolutely! β€’
Interaction 1: TTS -> TWILIO: Absolutely! β€’
Interaction 1: GPT -> TTS: Would you like to know more about the in-ear AirPods and AirPods Pro, β€’
Twilio -> Interruption, Clearing stream
Twilio -> Audio completed mark (1109): bd21b254-3f01-49b1-8119-08d3dcddffa0
Interaction 1: GPT -> TTS: or the over-the-ear AirPods Max?
GPT -> user context length: 7
Interaction 1: TTS -> TWILIO: Would you like to know more about the in-ear AirPods and AirPods Pro, β€’
Twilio -> Interruption, Clearing stream
Interaction 1: TTS -> TWILIO: or the over-the-ear AirPods Max?
Twilio -> Audio completed mark (1171): 44c7d4e8-1820-42e1-8456-8087fe4b8b92
UtteranceEnd received before speechFinal, emit the text collected so far: one headphone price?
Interaction 2 – STT -> GPT: one headphone price?
Twilio -> Audio completed mark (1280): ea654842-6155-4e89-a724-dca99bdbd21b
Warning: Double function arguments returned by OpenAI: {"model": "airpods"}{"model": "airpods pro"}{"model": "airpods max"}
Interaction 2: GPT -> TTS: Let me check the price, one moment.
GPT -> called checkPrice function
**C:\Python Important\call-gpt-main\node_modules\openai\error.js:43
return new BadRequestError(status, error, message, headers);
^

BadRequestError: 400 'checkPrice' is not one of ['system', 'assistant', 'user', 'function'] - 'messages.8.role'
at APIError.generate (C:\Python Important\call-gpt-main\node_modules\openai\error.js:43:20)**
at OpenAI.makeStatusError (C:\Python Important\call-gpt-main\node_modules\openai\core.js:251:33)
at OpenAI.makeRequest (C:\Python Important\call-gpt-main\node_modules\openai\core.js:290:30)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async GptService.completion (C:\Python Important\call-gpt-main\services\gpt-service.js:55:20)
at async GptService.completion (C:\Python Important\call-gpt-main\services\gpt-service.js:114:9) {
status: 400,
headers: {
'access-control-allow-origin': '*',
'alt-svc': 'h3=":443"; ma=86400',
'cf-cache-status': 'DYNAMIC',
'cf-ray': '877c5ba83ad621e3-KHI',
connection: 'keep-alive',
'content-length': '203',
'content-type': 'application/json',
date: 'Sun, 21 Apr 2024 09:30:54 GMT',
'openai-organization': 'user-gntjrfo4a4rpvvm5p9udpbnq',
'openai-processing-ms': '40',
'openai-version': '2020-10-01',
server: 'cloudflare',
'set-cookie': '__cf_bm=iKylCws.LiQAZjpYO7fLbZyeMgMRa8pUZT7aUFV6k9Q-1713691854-1.0.1.1-kKpI9bMcsYZXFTmzd2m5KozjqvGMkwwPYslFKJnFMLX.Fw4md0fCxEe0B0RWxPXenKhGQw6MkE7maRZ_osYOpg; path=/; expires=Sun, 21-Apr-24 10:00:54 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, _cfuvid=DFu1pwBHgbgfvXVgiE_thT8EI4OpFrDNR6GdEkJB8zA-1713691854483-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None',
'strict-transport-security': 'max-age=15724800; includeSubDomains',
'x-ratelimit-limit-requests': '500',
'x-ratelimit-limit-tokens': '300000',
'x-ratelimit-remaining-requests': '499',
'x-ratelimit-remaining-tokens': '299618',
'x-ratelimit-reset-requests': '120ms',
'x-ratelimit-reset-tokens': '76ms',
'x-request-id': 'req_669c8e2baa1c400655a23fd6ab91b696'
},
error: {
message: "'checkPrice' is not one of ['system', 'assistant', 'user', 'function'] - 'messages.8.role'",
type: 'invalid_request_error',
param: null,
code: null
Node.js v21.7.3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.