fspoettel/moneypenny

🤖 never tired, slightly incoherent robo-secretary. Built with the Google Cloud Speech-to-Text API.

moneypenny

Automatic transcription tool built around the Google Speech-To-Text API. It turns audio/video files into timestamped, .srt-formatted transcripts and has support for speaker separation and automatic punctuation.

Self-Hosting

This service is self-hostable via dokku. It requires the following addons to be present and linked to the app:

• redis
• postgres

In order to create the database tables for this service, open psql via dokku postgres:connect <your_app> and run the instructions in ./src/db/pg/create.sql

After that, you can connect to your container and run yarn add-user --email {an_email} --password {password_will_be_hashed} to add the first user to the service.

You will also have to add the content of nginx.example.conf to /home/dokku/moneypenny/nginx.conf.d/upload.conf to increase nginx's default timeouts since the STT API can take a very long time to process input. This would ideally be circumvented via a queue for async processing. I have not got around to implement that yet.

Setting up Google Cloud

This service uses the Google Cloud Speech-To-Text API to transcribe media files. This API requires you to upload media files to Google Cloud Storage before being able to transcribe them. In order to setup your Google Cloud account, you'll need to:

• (env: GOOGLE_PROJECT_ID) Activate the Speech-To-Text API for the Google Cloud project you are using for this service
• (env: GOOGLE_BUCKET) Create a private Google Cloud Storage bucket
• (env GOOGLE_CLIENT_EMAIL & GOOGLE_PRIVATE_KEY) Create a service user with at least the following set of permissions:

• storage.objects.create
• storage.objects.delete
• storage.objects.get
• storage.objects.list

After setting the credentials as environment variables via dokku config:set (see above and .env.example), you should be able to use the service.

Configuration

See .env.example for required and optional .env variables.

• See here for Google setup instructions
• UPLOAD_LIMIT_MB sets the upload limit. This value will also have to be reflected in the nginx config (see above)
• SESSION_SECRET (required) see here
• SENTRY_DSN (optional)

Development

Mocking Google STT responses

Replace recognize() call in transcribe.js with src/lib/__mocks__/googleCloud.js implementation

moneypenny's Issues

Allow choosing file ending for transcripts

• .txt
• .srt

Remember language selection

Deploy & install instructions

Allow choosing audio input format/params

Offer option to split .srt by character limit

Offer option to split .srt files by punctuated sentences

Validate input based on STT language features

The backend handles mismatching inputs already. It would be nice if the frontend updates form state based on the constraints of the STT API:

• disable choosing split by punctuation if language does not support it
• disable enabling diarization if language does not support it
• disable enabling profanity filter if language does not support it

Automatically downsample files exceeding 48khz sample rate

Support other STT providers

• IBM Watson
• Amazon Transcribe

blob.text() API is not available in Safari

https://developer.mozilla.org/en-US/docs/Web/API/Blob/text

Auth mechanism