dascritch/cpu-audio

An audio WebComponent to provide timecoded links and other features to an audio tag

CPU-Audio WebComponent

An audio WebComponent to provide an user-interface, timecoded links and some other features to an <audio> tag.

Purpose

A generic WebComponent crafted with a hashtag observer for <audio> tags with fancy interface, hyperlinks, chaptering, playlist and share buttons.

When you load a page :

1. if your URL has a hash with a timecode (https://example.com/page#tagID&t=10m), the service will position the named <audio controls> at this timecode (here, #TagID at 10 minutes) ;
2. or, if a audio source (qualified by its url) was played in your website, and was unexpectedly exited, the service will reposition its <audio controls> at the same timecode.

During the page life :

• if an <audio controls> anchor is linked with a timecode, as <a href="#sound&t=10m">, the service will play this tag at this timecode ;
• no cacophony : when a <audio controls> starts, it will stop any other <audio controls> in the page ;
• if you have a <cpu-controller> in your page, it will clone the playing <cpu-audio> interface.

This link starts the upper player at 20:45, and a link can limit playing between a start (0:05) and end (0:08) marks

Features

TL;DR ? See it in action

• hyperlink <audio> tags to a specific time, using not well-known Media Fragment standards ;
• standards first, future-proof ;
• only one single file to deploy ;
• pure vanilla, no dependencies to any framework, so usable in any JS framework ;
• progressive enhancement, can works even without proper WebComponent support ;
• add an UI, customizable via attributes and CSS variables ;
• exists in different graphic themes ;
• responsive liquid design ;
• recall the player where it was unexpectedly left (click on a link when playing) ;
• play only one sound in the page ;
• playlist with auto-advance ;
• play only a range between 2 timestamps ;
• chapters, using standards WebVTT ;
• alternate navigation for a finest precision on smartphones ;
• global <cpu-controller> .

It could have been done via polyfills or frameworks, but I wanted a plain standard, vanilla javascript, easy to install and configure.

HOW TO install

A unique and lightweight js file to install, without any dependencies.

• How to install, deploy and customize on your server
• You can try playing with our live configurator tool
• See it running in our demonstration site
• See basic examples and applications with advanced use cases
• A React .jsx example

Keyboard functions

When the interface got the focus, you can use those keys :

• Space : play/pause audio
• Enter : play/pause audio (only on play/pause button)
• ← : -5 seconds
• → : +5 seconds
• ↖ : back to start
• End : to the end, finish playing, ev. skip to the sound in playlist
• Escape : back to start, stop playing
• ↑ and ↓ : move focus between entries in panels (Enter to select)

For handheld users, a long press on the timeline will show you another interface for a more precise navigation (Desktop users can try it via a right click on it).

Some links

• Demonstration site
  • Features
  • How to install
  • Live configuration
• JS installable code
• Code repository, Latest stable release
• Frequently Asked Questions
• Blog posts about its creation and development : Série cpu-audio sur dascritch.net
• How to cite a podcast, now you can support time positions URL
• Main author : Xavier "dascritch" Mouton-Dubosc
• Licence LGPL 3

Participate

• Contribute in any way
• Where we are now
• Tests
• Bugs, issues, tickets and features
• What to do, next

Credits

Thank you to my lovely friends :

• Thomas Parisot for suggestions
• Loïc Gerbaud, Guillaume Lemoine and Guillaume de Jabrun for bug-hunting
• Benoît Salles and Michel Poulain for testing
• @mariejulien about CONTRIBUTING.md
• Christophe Porteneuve of Delicious Insights to kick my ass on modern javascript.
• scombat for a wonderful wrapper of cpu-audio.js in React
• Éric Daspet for its remarks on documentation
• Gabi Boyer for helping me avery a lot to debug iOS issues

Really sorry, NerOcrO

Keeping in touch

• professional http://dascritch.com
• blog http://dascritch.net
• twitter : @dascritch

cpu-audio's Issues

Support MSE API or DASH for non-bufferred playing

from dascritch/ondemiroir-audio-tag#22

May help issues when switching rapidly position at low bandwith.

• MediaSource specification at MDN
• https://www.w3.org/TR/2013/WD-media-source-20130129/#attributes-1
• Enhancement is becoming urgent, as a correct method to makes progressive downloads complying with very big media files over HTTP2Push #39
• Maybe I have to use DASH format :( https://hacks.mozilla.org/2015/07/streaming-media-on-demand-with-media-source-extensions/

Well, not useful. As the #39 bug is mainly a configuration or a protocol missing.
See also http://blog.eleven-labs.com/fr/video-live-dash-hls/

Show only a playing instance of cpu-audio

Suppose :

• on a page, you have
  • a <cpu-controller>
  • numerous <cpu-audio>, in a playlist

We may want to show only the active <cpu-audio>, hiding the others <cpu-audio>.

May be it is a wrong idea, maybe using css vars is enough.

Parameters to not put the share button

As in CPU.pm header , I've done an ugly CSS rule

Create a real playlist service (decentralized)

from dascritch/ondemiroir-audio-tag#42

See dascritch/plugin-dotclear-cpu-audio#4

Using
https://developer.mozilla.org/en-US/docs/Web/API/Navigator/registerProtocolHandler/Web-based_protocol_handlers
https://www.caniuse.com/#search=registerProtocolHandler
for having decentralized service

Remove any notation in links except t=<seconds>

keep it simple, stupid

canonical attribute must accept hash

A very stupid issue in element_cpu.js, function update_time is building the canonical URL with the CURRENT id element. This is not critical, but really a stupid idea. I should have wrote that in a rush.

    update_time(event) {
        let timecode = convert.SecondsInTime(this.audiotag.currentTime);
        let link_to = absolutize_url(this.audiotag.dataset.canonical)+'#';
        // WHAAAT ? I should have done something wrong there
        link_to += this.audiotag.id ? (this.audiotag.id+'&') : '';
        link_to += 't='+timecode;

The best idea is to look if a valid ID is already in URL's hash, out of other parameters

Rewrite and split index.html presentation page

Too many audio elements, Chrome takes hours to load it

Clean html : use <progress> for timeline

Instead of ugly <div> with a percent width...

• migrate from <div> to <progress> for the loading line and the reading line
• with min="0" and max="" at total length, instead to ugly play with percentage
• have undeterminate state until we have source length
• use styling tricks https://css-tricks.com/html5-progress-element/ https://css-tricks.com/styling-cross-browser-compatible-range-inputs-css/

Notes : <input type="range"> is too heavy to customize, and nor Chrome, Safari, Firefox can differ the track before the thumb. there is somewhere a way with ::before and attr(value) , but I can't find it anymore

Playing time range using start-point,end-point notation

dascritch/ondemiroir-audio-tag#25

cf https://www.w3.org/2008/WebVideo/Fragments/WD-media-fragments-spec/#smpte-time

t=10,20   # => results in the time interval [10,20)
t=,20     # => results in the time interval [0,20)
t=10      # => results in the time interval [10,end)

Needs to

• improve hash url parser
• put event on timecode to stop player, or using progress
• stops player even in no WebComponent browsers
• remove it on changing position, out of end position
• adding visual element

Restore i18n

from dascritch/timecodehash#12

Was not really feasible due to HTML Import, but as that feature doesn't seems to be in consensus and removed due to security concerns, we can even use my https://github.com/dascritch/__ library, but only via the dist/cpu-audio.js

Use a playlist for continuous playing at a page

from dascritch/ondemiroir-audio-tag#47

if an <cpu-audio playlist="name"> finishes, search in the same page for next <cpu-audio playlist="name"> (even without controls ?) and start playing

• define the attribute finally
• make a API to add in playlist
• expose playlist in document.CPU API
• playlists in a page, play next audio in playlist parametrable via API
• make event at end to move next in playlist
• add to stacked audio timecode for end
• change container to use named playlists
• show playlist in <cpu-controller>

Public methods should have a javadoc

Yep, i know.

I said once that i ended with javadoc...

Create a live configurator page

• form for html source
• form for css variables
• support reset event for CSS edition
• a dummy "button" event switches pan
• include in .MD
• separate style modifications events
• group widths on width breakpoints
• show only changed css parameters

Autoplay policy (pending)

As in Chrome cf https://developers.google.com/web/updates/2017/09/autoplay-policy-changes ,
Firefox Nightly (63) is testing blocking autoplay due to abuse. We're both impacted https://bugzilla.mozilla.org/show_bug.cgi?id=1476853

Note : To do the tests, don't reload the browser, but follow the links. Yes, it's ugly.

Use QUnit v2

Alas, my conversion failed with "global failures : no tests were run" ????
It seems a regular problem with QUnit v2.

Live parametrables should offer hide="" options

Prepare design and key bindings for RTL localizations

Arabic, Hebraic and others RTL (Right To Left) languages are usually using right-to-left timelines. A lot of abstraction work is to do there :

• check RTL support for native players
• reverse button groups
• reverse chapters list
• reverse timeline
• reverse key bidings (the most difficult ?)

Rename source files

prefix with numbers to know their order

On removing <audio> tag from DOM, remove <cpu-audio>

from dascritch/ondemiroir-audio-tag#8

See https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver . We can also use it for any property change (song name, etc...)

Support incoming navigator.share API

from dascritch/ondemiroir-audio-tag#55

reference :

• mozilla/standards-positions#27
• https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/nwmTLZKFFIA/discussion
• https://www.chromestatus.com/features/5668769141620736
• https://developer.mozilla.org/en-US/docs/Web/API/Navigator/share

Only on Chrome android. Wait for another implementation.

Integrate Dotclear plugin generation

Put a soundwave form image alongside the timeline

from dascritch/ondemiroir-audio-tag#52

• Calculated by the browser ? may be too heavy and impact battery usage
• Use a remote image with the soundwave form ?
• Use a remote data with the soundwave form, rendered as SVG ?

Need a CPU.is_playing_audiotag()

To check if a audiotag element is the same one playing, we sometimes compare their ID, sometime using isEqualNode() . There is a lot of confusion and is not really DRY.

examples :

(self.audiotag.id === CPU_Audio.current_audiotag_playing.id) 

or

audiotag.isEqualNode(CPU_Audio.current_audiotag_playing)

We need a utility method on the global API, to simplify the code.

Use a real CSS compressor to remove comments

Supporting annotations à la SoundCloud

dascritch/ondemiroir-audio-tag#29

There is an official standard for that :
https://hypothes.is/blog/annotation-is-now-a-web-standard/

There is also a standard for server-to-server dialogs on annotations : https://en.wikipedia.org/wiki/Salmon_(protocol)

In playlist, next song must restart at 0

In a playlist context, if the next sound piece was already started and not pointing at 0:00, ending its predecessor will continue the play, instead to restart it at its begin.

CSS breakpoints via container queries

Actually, breakpoints are hard-coded : 640px , 480px, and 320px

CSS variables are not able to be used as breakpoints, as they are... uh... variables.

There are some proposal to the standards, as
CSS @element queries and CSS Environment variables, and some tests in browsers.

Wait & see.

Share toolbar is too wide

Only happening in cpu.pm , on cpu-controller (header, small to medium width)

querySelector_apply too largely called

The third parameter of querySelector_apply(selector, callback, subtree) is facultative. When it is not defined, it will query on the whole document tree.

But it MAY BE a very wrong idea o some places :

cpu_controller.class.js, connectedCallback() :

querySelector_apply('.canonical', element_prevent_link_on_same_page);

should be limited to the shadowDOM only

No more hotlink to my website, gh-pages instead

Buggy when playable media not declared

reported Mac OS X Safari 12 by @hadrienl

Preview position when hovering an internal link

On a host page, when hovering/focus a link that goes to a moment of a player, show the target link on the timeline of the targeted player.

• will need a special API on cpu-audio element :
  • highlight_moment
  • highlight_hide
  • highlight_range
  • highlight_chapter
• add a event on host page on hovering/focus <a href="">
  • check it links on the same page
  • check it links to a cpu-audio
  • prepare to highlight_hide when pointer goes out or focus out
• add a event on webcomponent on hovering/focus <a href=""> (for chapter list)
• also hightlight on <cpu-controller>

May be linked to #40

Using touchstart event to emulate hover

dascritch/ondemiroir-audio-tag#37

something like ???

.{{classname}} [title]:hover:after {
 display: block;
 position : absolute;
 top : -100%;
 content: attr(title);
 font-size:10px;
}

Better way to change time position on mobile

When I'm a mobile user
I want to have a more precise way to position on the timeline
Because i'm not very precise with my finger on a 2 hours long timeline

The user have actually 2 ways to reposition the player at a certain timeline, out of hyperlinks :

• With the keyboard ← and →, which (obviously) doesn't exists on handheld
• By touching the timeline, but it is really imprecise.

Why not make appears a special selector, as

<input type="time" min="0" max="1:1:1" step="1" /> 

As most of browsers on handheld take care to have a more handly selector for this kind of <input>.

Caveats :

• Does the step="1" attribute which triggers the seconds selector is really accepted everywhere ?
• input and change events are triggered too early or too late ?
• If a media is longer than 24 hours, you won't be able to select upper than the first 24 hours

Explode index.html

to use github pages features completely
Also, Chrome loads and caches so takes to much time when there is too much <audio> tags for him.

Regression on hash events on unnamed audio tag names

https://cpu.dascritch.net/post/2018/05/18/Bonjour-%C3%A0-toi%2C-Enfant-du-Futur-Imm%C3%A9diat-%3A-Des-chiffres-et-des-lettres#CPU-Audio-tag-0&t=97 is no more working.

https://dascritch.net/post/2019/01/17/CPU-Ex0102-Bonjour-%C3%A0-toi%2C-Enfant-du-Futur-Imm%C3%A9diat-%3A-Le-Seum-et-le-Grub#CPU-Audio-tag-0&t=2026

CPU-Audio-tag-0 is auto generated , and not seen in time

Redispatch events to reduce paints

Better aspect for loading timeline

from dascritch/ondemiroir-audio-tag#53
Taken from the pdf reader of Firefox 55

#loadingBar .progress.indeterminate .glimmer {
    position: absolute;
    top: 0;
    left: 0;
    height: 100%;
    width: calc(100% + 150px);
    background: repeating-linear-gradient(135deg, #bbb 0, #999 5px, #999 45px, #ddd 55px, #ddd 95px, #bbb 100px);
    animation: progressIndeterminate 950ms linear infinite;

prevent_link_on_same_page doesn't prevent if a hash is targeted

my bad....

Hide fallback chapters lists when player display chapters

We may be interested to display a chapter lists by default (by example, in http://cpu.pm , we have #chapters) , waiting for th player to display his own.

• prepare a way to hide it from an attribute in the chapters list ( data-cpu-playlist-for="audio-id" )
• prepare a way to hide it from an attribute in the <cpu-audio> call ( playlist-fallback="playlist-id" )
• hide only if a non-empty chapter list is loaded

Or use a simplier class added to host document.body, namely cpu_tag_«${audiotag.id}»_chaptered. Most flexible and easy to style.

Expandable mode="" on read

Why not having a player that expands when clicked on play for the first time ?

The player would have a button or compact aspect , then expands as default if the player is triggered to play.

it won't shrink later while the page is opened.

Add a fade effect when start/stop parametrable via CSS variable

should permit smoother transitions.
Could also be done via css transition effect ?

Creating an API to add sharing functions

from dascritch/ondemiroir-audio-tag#1

Some css vars settings won't be useful

Try those personnalization settings :

body {
	--cpu-background : transparent;
	--cpu-color : currentColor;
	--cpu-playing-background : transparent;
	--cpu-playing-color : currentColor;
}

When the mouse is over a link or button, the result is dirt, cannot be read.

Try to correct the issue if it's possible (filters ?), or document the impossibility.

Use passive event listeners

https://developers.google.com/web/tools/lighthouse/audits/passive-event-listeners

First check that those events don't have preventDefault()

Create a live chapter editor page

• Having a player
• Choosing an audio source URL
• Create and naming chapters
• Moving chapters, sorted by time
• Deleting chapters
• Showing HTML generated source
• Showing HTML generated fragment
• Showing markdown generated source
• Showing wiki generated source
• Integrate in .md file

Add a11y role="alert|progressbar"

Quoted from https://24ways.org/2018/wcag-for-people-who-havent-read-the-update/ :

if your user needs to know, “What was the error?”, add role="alert”;
if you user needs to know, “How far through it are we?”, add role="log" (for a chat window) or role="progressbar" (for, well, a progress bar).

<a id="elapse"> tag may be linked to "undefined"

fallback is missing there

Active chapter references in host page

Make references to named chapters in host page, to highlight during the play. It may be as a data-cpu-chapter="" attribute, or as a link <a href="#t=....">, but is there a specific notation of anchors in anchors ? ;)

Have a local .md → .html generator

Github system sometimes doesn't convert some pages, as https://dascritch.github.io/cpu-audio/CONTRIBUTING
Overfrustrating