Voice recorder button throws an exception when qnabot is embedded on website

[chanbi-aws]

Voice recorder button throws an exception when qnabot is embedded on website

[dougtoppin]

@chanbi-aws thanks for your report, we will investigate and get back to you

[michaelin-96]

Hi @chanbi-aws, I assume you are using v5.4.5 from your latest post. I recommend trying to upgrade to the latest version v5.5.0. I wasn't able to replicate this issue when deploying qnabot v5.5.0. Let me know if you still come across issues! Thanks.

[michaelin-96]

After discussion with @chanbi-aws, this seems to be a bug from lex-web-ui, which our QnABot solution does not manage.

[chanbi-aws]

This is a different issue to the 'invalid ssml'.

[bobpskier]

@michaelin-96 @chanbi-aws If the LLM (Bedrock) response is generating output to be used in a voice based response through QnABot, the handlers managing the request/response with Bedrock will need to manage output in a voice based SSML manner. That is, the Bedrock integration will need to provide a voice based response in valid SSML if that is supported. LexWebUi is not going to manage output from the Lex bot and perform correction to account for incorrectly formatted SSML. The integration in QnABot needs to identify that the request was made in voice mode and that the output should be in voice mode using valid SSML.

Is the response payload being provided to LexWebUi when the error occurs available to investigate further?

Is there a public endpoint that can be exercised to investigate?

[chanbi-aws]

The issue occurs prior to any request to the LLM. The error is thrown when you click the 'mic' icon to record your voice input. Maybe this is an issue that needs to be raised with https://github.com/aws-samples/aws-lex-web-ui.

[bobpskier]

@chanbi-aws Is the embedded version of LexWebUi being used or was the full version from https://github.com/aws-samples/aws-lex-web-ui installed? Sounds like it is the full version based on the use of parent.html but want to confirm. If its the full version then this should be submitted against LexWebUi. Note a number of problems with voice mode were resolved in LexWebUi 0.20.2 which was released on 2023-11-28. So first step is going to be to upgrade to the latest LexWebUi at least greater than or equal to 0.20.2.

[michaelin-96]

@bobpskier This comment should refer to this ticket 694.
In terms of the mic issue in the embedded page, I recommend using the latest lex build (20.6), otherwise this may be a bug for lex-web-ui. Closing this ticket out now

@michaelin-96 @chanbi-aws If the LLM (Bedrock) response is generating output to be used in a voice based response through QnABot, the handlers managing the request/response with Bedrock will need to manage output in a voice based SSML manner. That is, the Bedrock integration will need to provide a voice based response in valid SSML if that is supported. LexWebUi is not going to manage output from the Lex bot and perform correction to account for incorrectly formatted SSML. The integration in QnABot needs to identify that the request was made in voice mode and that the output should be in voice mode using valid SSML.

Is the response payload being provided to LexWebUi when the error occurs available to investigate further?

Is there a public endpoint that can be exercised to investigate?

[chanbi-aws]

I will raise an issue with aws-lex-web-ui. Updating the stack with the master.yaml (https://github.com/aws-samples/aws-lex-web-ui/blob/master/templates/master.yaml) successfully upgrades to v0.20.6, but when the UI initialises with version 0.20.5. Browser console output: "lex-web-ui.min.js:29 successfully initialized lex web ui version: 0.20.5"