bbc/stt-align-node

node version of stt-align https://github.com/bbc/stt-align by Chris Baume - R&D.

Stt-align-node

See The alignment problem in the docs for more background of the problem this module set out to address.

Originally developed as a node version of python's stt-align by Chris Baume - BBC R&D.

Setup - development

git clone [email protected]:bbc/stt-align-node.git

cd stt-align-node

npm install

Setup - in production

npm install @bbc/stt-align-node

---

Usage

Other then to realign STT results with accurate text, this modules can also be used to perform related oprations in the same domain, such as benchmarking STT.

Function	Description	type
alignSTT	Realign STT json with accurate text. by transposing words from accurate text to timecodes of STT.	json
diffsList	return a diff json of STT vs accurate text	json
diffsListAsHtml	return a diff of STT vs accurate text as HTML	html
diffsCount	return a diff of STT vs accurate text as HTML	json
calculateWordDuration	return a diff of STT vs accurate text as HTML	Number

See See README in example-usage folder as well as code examples for more.

---

System Architecture

Node version of stt-align by Chris Baume - R&D.

In pseudo code overview of alignSTT:

• input, output as described in the example usage.

  • Accurate base text transcription, string.
  • Array of word objects transcription from STT service.

• Align words

  • normalize words, by removing capitalization and punctuation and converting numbers to letters

  • generate array list of words from base text, and array list of words from stt transcript.

    • get opcodes using difflib comparing two arrays
    • for equal matches, add matched STT word objects segment to results array base text index position.
    • Then iterate to result array to replace STT word objects text with words from base text

  • interpolate missing words

    • calculates missing timecodes
    • first optimization
      • using neighboring words to do a first pass at setting missing start and end time when present
    • Then Missing word timings are interpolated using interpolation library 'everpolate.

Development env

• node 10
• npm 6.1.0

Build

npm run build

bundles the code with react, into a ./build folder.

build demo

npm run build:demo

Demo is in docs folder

Publish demo to github pages

npm run deploy:ghpages

Tests

npm run test:watch

• add more tests

Deployment

Deploy to npm

npm run publish:public

stt-align-node's Issues

README and examples issues

A good list of things to sort out in the README and rest of the repo by @gridinoc

• 1. const alignJSONText = require('./index.js'); -> no index.js, if you require src/index.js still dies as there is no function exported by default
• 2. "See /lib/example-usage for an example that you can run with npm run example." there is no /lib
• 3. babel-loader and gh-pages should be devDependencies
• 4. the demo is useless, I get a page asking for files I don't have
• 5. package.json: "main": "lib/index.js", well, no /lib again

Leaving it as a note for when I get a chance to do a PR

first and last words don't have timings issue - edge case

possible fix via @Laurian

basically I pad the word array to be sure the 1st and last word has timing, otherwise it will extrapolate timing on words that lack timing and are on the edge, getting by that timings outside the paragraph boundaries (even negative timings on 1st paragraph)

words = [
      {
        text: 'STARTSTART',
        start: words[0].start - (0.08475 + 0.05379 * 'STARTSTART'.length),
        end: words[0].start,
      },
      ...words,
      {
        text: 'ENDEND',
        start: words[words.length - 1].end,
        end: words[words.length - 1].end + 0.08475 + 0.05379 * 'ENDEND'.length,
      },
    ];

    const resultAligned = alignSTT(
      {
        words,
      },
      `STARTSTART ${text} ENDEND`
    );