retellai/retell-client-js-sdk

Check out Retell AI Web Call Guide for how to use this SDK.

retell-client-js-sdk's Issues

How can we have a handle over the output stream

The SDK provides us with a handle for the input stream (sdk.stream), but I would also like to have the output stream.

Client SDK doesn't work on mobile networks

When using the latest version of the library that uses RTC I'm noticing that phones running on mobile networks (specifically I'm testing with Verizon) are unable to connect but are able to connect when on WiFi. I believe this is likely do to SSL / TURN not being setup correctly with the newer LiveKit dependency...

Check out the documentation: https://docs.livekit.io/home/self-hosting/deployment/

Elsewhere I see documenation states that this can cause issues with mobile networks when not setup correctly.

Knowledge folder is missing?

I think this link had useful info,
https://github.com/RetellAI/retell-client-js-sdk/blob/main/knowledge/audio-basics

Would love to understand more about optimal bitrates to be used!

Thank you,

The socket connection closes after some time

My socket connection closes after some time, for a reason unknown to me. I used your client-js-sdk and the example in the demo repository

const webClient = new RetellWebClient();

export default function Home() {
    const [ callActive, setCallActive ] = useState<boolean>(false);

    useEffect(() => {
        // Setup event listeners
        webClient.on("conversationStarted", () => {
            console.log("conversationStarted");
        });

        webClient.on("audio", (audio: Uint8Array) => {
            console.log("There is audio");
        });

        webClient.on("conversationEnded", ({ code, reason }) => {
            console.log("Closed with code:", code, ", reason:", reason);
            setCallActive(false); // Update button to "Start" when conversation ends
        });

        webClient.on("error", (error) => {
            console.error("An error occurred:", error);
            setCallActive(false); // Update button to "Start" in case of error
        });

        webClient.on("update", (update) => {
            // Print live transcript as needed
            console.log("update", update);
        });
    }, []);

    const toggleConversation = async () => {
        if (callActive) {
            webClient.stopConversation();
        } else {
            const registerCallResponse = await registerCall();
            if (registerCallResponse.call_id) {
                webClient
                    .startConversation({
                        callId: registerCallResponse.call_id,
                        sampleRate: 24000,
                        enableUpdate: true,
                    })
                    .catch(console.error);
                setCallActive(true); // Update button to "Stop" when conversation starts
            }
        }
    };

    const registerCall = async (): Promise<{ call_id: string }> => {

        const res = await axios.post<{ call_id: string }>("http://my-server.com/api/v1/calls/register-call/", {
            audio_encoding: "s16le",
            audio_websocket_protocol: "web",
            sample_rate: 24000
        });

        return res.data;
   }
   ...
}

[Bug] unable to stop conversation using sdk.stopConversation()

Using this code from the sample demo.
const toggleConversation = async () => {
if (isCalling) {
console.log("stopping conversation")
try {
webClient.stopConversation();
} catch (err) {
console.log(err);
// throw new Error(err);
}
} else {
const registerCallResponse = await registerCall(agentId);
console.log(registerCallResponse)
if (registerCallResponse.call_id) {
console.log("starting conversation")
webClient
.startConversation({
callId: registerCallResponse.call_id,
sampleRate: registerCallResponse.sample_rate,
enableUpdate: true,
})
.catch(console.error);
setIsCalling(true); // Update button to "Stop" when conversation starts
}
}
};

Able to successfully start the conversation but the conversation never stops and the event listeners for conversationEnded or error don't get called. So, unable to stop a web conversation. Is there anything I am missing here?

Audio event cuts off early/randomly

Sometimes while an agent is speaking, the audio buffer will stop prematurely. Feels like the event just closes itself randomly.

Receive function call results from client js sdk

Is there a way to receive the JSON output on the client from calling a function? Let's say my retell python backend defines a function (which does something other than ending the call) that gets called by the LLM, how can I access the function from this JS SDK on the client?

New version with livekit seems to have performance issues / latency

I was trying out the latest release that uses livekit that was pushed last night and have been noticing intermittent performance issues that don't exist with the previous version.

Specifically the voices seem to sometimes drop out and slowed down, like it's struggling with bad latency that we didn't see with the previous version of the api. Recordings of the call seem to be fine when we play them after ending the call, so the latency in the playback seems to be happening on the client side.

Reproduction of this is pretty simple we are running essentially the same code from the setup document (the issue happens regardless of whether we have emitRawAudioSamples on or off).

Is this something you have been observing in your testing? Should we revert back to the previous version?

stopConversation() doesn't stop the conversation?

retell-client-js-sdk@^1.3.2

I refresh the page instead.

Ref: https://discord.com/channels/1186424376300679209/1222558901753483295

import { RetellWebClient } from 'retell-client-js-sdk';

function WebCall() {
  let retellWebClient = new RetellWebClient();
  const [isCallActive, setIsCallActive] = useState(false);

  retellWebClient.on(ClientEventNames.conversationEnded, ({ code, reason }) => {
    console.log('Conversation ended with code "%d" and reason "%s"', code, reason);
    setIsCallActive(false);
  });

  retellWebClient.on(ClientEventNames.error, error => {
    console.error('Conversation error occurred', error);
    setIsCallActive(false);
  });

  async function toggleCall() {
    if (isCallActive) {
      retellWebClient.stopConversation();
      // stop doesn't work, so refresh the page instead
      window.location.reload();
    } else {
      // ... start the call
    }
  }