jan-ka/sams-still-alive

experiment to have speech synthesis sing to still alive

sams-still-alive's Issues

switch to typescript

ToneJS is native typescript. No reason to not also use it for this project. As long as parcel keeps working or a better replacement can be found.

favicon not found

Browsers look for the favicon at the base url, not the path. There are two options to go about this:

1. find out how to let parcel ignore a linked asset
2. ascertain that having the favicon path with a hash is ok if we have to set it anyway

• means also probably getting rid of parcel-reporter-static-files-copy

pointers for solving "no voices"

Some linux variants appear to come with no default voices. There are easy ways to fix this. Adding a small section about this and linking it in the app should go a long way.

Reported working:

• Microsoft Windows 11
• Apple macOS 12.4
• Ubuntu 20.04

Known broken:

• Manjaro (? Which version)
• Ubuntu 18
• iOS 15.5

lyrics writer

display the lyrics while they are sung instead of the placeholder text

switch to pug for template rendering

this should cut down a bit on repitition and perhaps also allows to remove @cimonitor/parcel-transformer-package-version again by injecting the package version through pug vars.

app drops out of rendering too early when "no voices"

If a browser reports "no voices", the control structure breaks too early. Leaving parts of the scene dangling around. This needs a cleanup and possibly a rework of how scenes are switched. (Maybe something more elaborate than a really primitive state machine)

playback has ended event

I can't find any way to react to the end of playing the song. This makes handling the UI for that case rather clunky.

Transport emits an end event, but not when the current playback has ended. Is this a configuration issue? Library limitation?

Complete notes for singing

Only the first part of the song is prepared for singing. The rest of the song should be mapped, for completeness sake.

As @tonejs/midi is now loaded anyway, maybe we can do the mapping locally too.

still_alive_lyrics.midi.zip

handle stage internally

Currently stage is stored in the underlying element of ControlButton. Might as well be stored in the Class itself

sams-still-alive/src/ControlButton.ts

Line 15 in 7d9f580

//TODO: Handle internally?

investigate which notes are played at the wrong timing

Each time the song hits Drum 1, there's an error about wrong timing.

Error: Start time must be strictly greater than previous start time
    at assert (index.b71e74eb.js:12994:27)
    at OmniOscillator.start (index.b71e74eb.js:19471:31)
    at MembraneSynth._triggerEnvelopeAttack (index.b71e74eb.js:25119:25)
    at MembraneSynth.triggerAttack (index.b71e74eb.js:24884:14)
    at MembraneSynth.triggerAttackRelease (index.b71e74eb.js:25032:14)
    at Part.callback (index.b71e74eb.js:640:27)
    at Part._tick (index.b71e74eb.js:28127:30)
    at ToneEvent._tick (index.b71e74eb.js:27794:18)
    at TransportEvent.invoke (index.b71e74eb.js:20396:18)
    at index.b71e74eb.js:19917:60 2685.428 'standard kit' 

The error is caught and deranked to warn

sams-still-alive/src/Caroline.ts

Line 83 in 7d9f580

//TODO: investigate which notes are played at the wrong timing

use notes to change singing

Currently, a note just triggers the SpeechSynthesis. Maybe there's a way to coax the synthesis a bit more to actual singing by dynamically playing with the pitch and rate.

The finest “resolution” is possibly on the word level, but this also probably means it sounds dumb, since the SpeechSynthesis can no longer guesstimate the sentences' pronunciation. Worth a try, thought.

Speech hard to understand when multiple instruments play

Some instruments seem far louder than others, or perhaps they get louder as they accumulate, making it very hard to pick out the voice. The music should be about equal in perceived volume to the voice.

Speech synthesis through audio alone

It would be fun if there was one voice that we could default to, even if it is utterly terrible.
There should be enough material from early speech synthesis experiments that could not rely on samples.
Even "just" having a tonal approximation of speech would be fun if we don't need to define it through midi, even thought we can potentially accomplish this through notejs.

This is a strictly experimental and optional step.

custom elements for components

src/VoiceSelect.ts and src/ControlButton.ts are currently handled through external classes, the same effect could be achieved by using custom elements instead