aws-samples/chime-live-translation-transcription-polly

WebApp Proof of Concept - Live Translation + Polly

Live Translation using AI (Amazon Chime + Amazon Transcribe + Amazon Translate + Amazon Polly)

Architecture Diagram

CONTEXT

1. This was a prototype to show the "Art of the Possible" of what the AWS services can be paired together
2. This is NOT a production SaaS product
3. This was used and testesd using a desktop Chrome browser. Since this prototype, we invite pull requests for further bug fixes and/or browser support.

Getting Started

1. yarn; yarn start

License

This library is licensed under the MIT-0 License. See the LICENSE file.

Contributing

We welcome community contributions and pull requests. See CONTRIBUTING.md for information on how to set up a development environment and submit code.

Content Security Legal Disclaimer

The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.

Operational Metrics Collection

This solution collects anonymous operational metrics to help AWS improve the quality and features of the solution. Data collection is subject to the AWS Privacy Policy (https://aws.amazon.com/privacy/). To opt out of this feature, simply remove the tag(s) starting with “uksb-” or “SO” from the description(s) in any CloudFormation templates or CDK TemplateOptions.

chime-live-translation-transcription-polly's Issues

Value null at 'mediaSampleRateHertz' failed to satisfy constraint: Member must not be null

Expected behavior and actual behavior

ANSWER:

Expected:

1. Sign in to the app
2. Add a request ID in the "Request ID" filed and click "Join"
3. Get prompted for browser access to mic and camera
4. Click "Transcribe"
5. Start speaking, and what is said is transcribed in the Transcription window

Actual behaviour on Firefox:

1. Sign in to the app
2. Add a request ID in the "Request ID" filed and click "Join"
3. Get prompted for browser access to mic and camera
4. Click "Transcribe"
5. Error window pops up with the following message:

An error occurred while recording: {"Message":"1 validation error detected: Value null at 'mediaSampleRateHertz' failed to satisfy constraint: Member must not be null"}

Steps to reproduce the problem

ANSWER:

1. In a terminal, clone the repo and navigate to the "chime-live-translation-transcription-polly" directory
2. run "yarn"
3. run "yarn start" to start up the local server
4. In a Firefox browser, navigate to "localhost:3000"
5. Sign in to the app
6. Add a request ID in the "Request ID" filed and click "Join"
7. Get prompted for browser access to mic and camera
8. Click "Transcribe"

Specifications like the version of the project, operating system, or hardware

ANSWER:

Project version:
Used main branch of the project with latest commit ID 7c936d2

OS:
Mac [Ventura 13.3.1]

Browser details:
Firefox version 102.10.0esr

Observations

• From what I can see the error is originating from the following line:

const command = new StartStreamTranscriptionCommand({ LanguageCode: language, MediaEncoding: 'pcm', MediaSampleRateHertz: sampleRate, AudioStream: audioStream, });

In Firefox, the value "sampleRate" passed for the MediaSampleRateHertz is null

• seems like sampleRate is being set here:

sampleRate = mediaStream.getAudioTracks()[0].getSettings().sampleRate;

However, with Firefox, the object returned by "mediaStream.getAudioTracks()[0].getSettings()" does not include "sampleRate"

• Overriding the sampleRate value and setting it to 44100 directly resolves the error, but then the app does not pick up any audio and no transcription is generated. After 15 seconds, the streaming session times out due to inactivity