This was a quick experiment to see how Google Cloud Speech's transcription output looks for long sound files. As input, I used a ~63 minute FLAC file of an episode from the podcast Acquired (thanks Ben!).
The code is a modified example from Google to run an asynchronous speech processing request. I had to call asyncRecognize because this is the only option if the audio length is greater than 1 minute. Furthermore, the format of the file must be LINEAR16 signed-integer little-endian encoded raw... see the audio format section, and resources sections below for processing tips.
This script currently only supports Google Cloud Speech for long recordings (>1m), and with the stated limitations above. I may or may not modify it later to include other speech vendor examples. For a more comprehensive tool to do this, see Speech Recognition.
- The output was not at all what I expected, and fairly disappointing; take a glance below.
- I've had much better results with a short bit of speech, say 20s on their test page here: https://cloud.google.com/speech/
- One possible issue is that the post-processing on the podcast audio created artifacts that led to inaccuracy. There is some indication this could be true based on their best practices, e.g. "All noise reduction processing should be disabled.".
- Another concern is that Google Cloud Speech will try to return up to 30 guesses... unclear how long each guess can be. In the output below they are short phrases, but in my tests with 20 second long phrases on their site, I see much longer sentences per guess.
- See "maxAlternatives" deep down in their API docs here: https://cloud.google.com/speech/reference/rest/v1beta1/RecognitionConfig
- Based on the little I've seen here, and the fact that most of Google's Cloud Speech API is focused on <1m long recordings... my guess is that long form transcription is not really Google's goal. They want to build something that one can use to create listening bots (like Siri or Alexa), and not something like a transcription service. Please correct me if you know more and find this to be an incorrect conclusion.
The result for this 5 minute section of the podcast was:
{
"response": {
"@type": "type.googleapis.com/google.cloud.speech.v1beta1.AsyncRecognizeResponse",
"results": [
{
"alternatives": [
{
"confidence": 0.31786352,
"transcript": "Mississauga"
}
]
},
{
"alternatives": [
{
"confidence": 0.92226779,
"transcript": "weather for Bakersfield College"
}
]
}
]
},
"done": true,
"name": "4920998656671129691",
"metadata": {
"lastUpdateTime": "2016-08-24T18:38:09.049576Z",
"@type": "type.googleapis.com/google.cloud.speech.v1beta1.AsyncRecognizeMetadata",
"startTime": "2016-08-24T18:33:03.320765Z",
"progressPercent": 100
}
}
For the entire hour long podcast it was:
{
"response": {
"@type": "type.googleapis.com/google.cloud.speech.v1beta1.AsyncRecognizeResponse",
"results": [
{
"alternatives": [
{
"confidence": 0.31786352,
"transcript": "Mississauga"
}
]
},
{
"alternatives": [
{
"confidence": 0.92226779,
"transcript": "weather for Bakersfield College"
}
]
},
{
"alternatives": [
{
"confidence": 0.578178,
"transcript": "Bed Bath & Beyond"
}
]
},
{
"alternatives": [
{
"confidence": 0.70186234,
"transcript": "ESPN"
}
]
},
{
"alternatives": [
{
"confidence": 0.2595036,
"transcript": "Bakersfield High School"
}
]
},
{
"alternatives": [
{
"confidence": 0.93311942,
"transcript": "White Castle"
}
]
},
{
"alternatives": [
{
"confidence": 0.81802225,
"transcript": "Kelly Services in Columbia Maryland"
}
]
},
{
"alternatives": [
{
"confidence": 0.51821041,
"transcript": "weather Chicago"
}
]
},
{
"alternatives": [
{
"confidence": 0.30766535,
"transcript": "Yahoo"
}
]
},
{
"alternatives": [
{
"confidence": 0.31368122,
"transcript": "Mexican grocery store"
}
]
},
{
"alternatives": [
{
"confidence": 0.90448982,
"transcript": "Facebook"
}
]
},
{
"alternatives": [
{
"confidence": 0.26522613,
"transcript": "sexy"
}
]
},
{
"alternatives": [
{
"confidence": 0.69060469,
"transcript": "restaurants in Los Angeles"
}
]
},
{
"alternatives": [
{
"confidence": 0.49876788,
"transcript": "American toxicology"
}
]
},
{
"alternatives": [
{
"confidence": 0.63318396,
"transcript": "Dumb Ways to Die"
}
]
},
{
"alternatives": [
{
"confidence": 0.59668612,
"transcript": "facebook.com"
}
]
},
{
"alternatives": [
{
"confidence": 0.95454544,
"transcript": "661 Sheldon"
}
]
},
{
"alternatives": [
{
"confidence": 0.95454544,
"transcript": "weather"
}
]
},
{
"alternatives": [
{
"confidence": 0.3689521,
"transcript": "open Facebook"
}
]
},
{
"alternatives": [
{
"confidence": 0.57749414,
"transcript": "restaurants"
}
]
},
{
"alternatives": [
{
"confidence": 0.4529644,
"transcript": "dictionary.com"
}
]
},
{
"alternatives": [
{
"confidence": 0.95454544,
"transcript": "browser"
}
]
},
{
"alternatives": [
{
"confidence": 0.83804256,
"transcript": "transgender"
}
]
},
{
"alternatives": [
{
"confidence": 0.63055462,
"transcript": "Dublin Toyota"
}
]
}
]
},
"done": true,
"name": "6402258854541648563",
"metadata": {
"lastUpdateTime": "2016-08-24T06:47:00.016904Z",
"@type": "type.googleapis.com/google.cloud.speech.v1beta1.AsyncRecognizeMetadata",
"startTime": "2016-08-24T05:40:58.702306Z",
"progressPercent": 100
}
}
- Visit the Google Developer Console and create a new project.
- For that project, click "Enable API" and add "Google Cloud Speech API"
- For that project, click "Credentials" and create an "API Key".
- Copy the key, and set it as the env variable GOOGLE_API_KEY to run the script.
To process Audio greater than
# Install SOX on OSX
brew install sox --with-lame --with-flac --with-libvorbis
# Convert FLAC file I got from Ben to PCM LINEAR16 format
sox input/ben-podcast-waze.flac --channels=1 --bits=16 --rate=44100 --encoding=signed-integer --endian=little input/ben-podcast-waze.raw
# Optional: trim it down to try a smaller sample.
sox --rate 44100 --bits 16 --encoding signed-integer input/ben-podcast-waze.raw input/5m-ben-podcast-waze.raw trim 0 05:00
- Visit https://console.cloud.google.com/storage/browser
- Upload files and make them public.
- URL is of the format: gs://yourbucketname/yourfilename, e.g: gs://example-content/5m-ben-podcast-waze.raw
pip install --upgrade google-api-python-client
# Kick off job to process file hosted on Goole Storage
python google_speech.py -u gs://yourbucketname/yourfilename.raw
# Check status of job
python google_speech.py -n somenumber
Audio cleanup:
- http://apple.stackexchange.com/questions/137108/how-can-i-add-support-for-flac-files-in-sox
- http://stackoverflow.com/questions/38926114/flac-file-with-google-cloud-speech-api-fails
- http://stackoverflow.com/questions/9667081/how-do-you-trim-the-audio-files-end-using-sox
Google dev:
Google speech:
- https://cloud.google.com/speech/
- https://cloud.google.com/speech/reference/rest/v1beta1/RecognitionAudio
- https://cloud.google.com/speech/limits
- https://cloud.google.com/speech/support
Example code:
Podcast transcription:
Apache License, Version 2.0 (the "License"): http://www.apache.org/licenses/LICENSE-2.0
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent