Add playback progress of current song

[bjoernpetersen]

Add playback progress of current song

[BjoernPetersen]

I have often considered adding something like this, but I'm still unsure if and how to do it.

First of all, the benefits are clear. This information would allow clients and the server to show playback progress, and it would also make things like lyrics plugins possible.

One feature which will not be implemented in the foreseeable future is skipping to a particular time in a song.

Client API

Progress vs. timestamp

I think adding progress instead of a timestamp to the API would be better, here's why:
As long as the player is playing, a timestamp would be marginally better, because it makes up for network delay. As soon as we factor in pauses, we'll run into problems. What to send to the client / show within the client while the player is paused? Also, when the player is unpaused, the "start timestamp" needs to be adjusted by the time the player was paused. Just keeping track of the playback progress server-side and sending it to the client seems like the better choice to me.

Song duration

Be aware that the song progress as calculated by the bot may be greater than the song duration (see below). It will probably never be exposed as such by the API, but the bot will rather report the maximum duration as the current progress for several seconds.
Additionally, Song.duration is optional/nullable.

Server-side

This is where things get tricky. At the baisc level, applicable to all plugin scenarios, the bot needs to keep track of the playback progress "manually" and react to player state changes, particularly pauses.
Manual tracking is neccessary, because, as far as the bot core is concerned, a PlaybackFactory is basically a blackbox in order to keep plugin restrictions to a minimum.

Accuracy issues

Some PlaybackFactory implementations are susceptible to external delays and inputs which can not (or not accurately) be detected:

• Playing YouTube-videos with mpv will always introduce some buffering delays (basically undetectable)
• The Spotify plugin plays songs using the official Spotify client, in which the user can pause the current song (detectable, but only delayed)

Such events would lead to an inaccurate playback progress with the "manual" progress tracking.

Enhancing accuracy

Some playback implementations can directly access the current playback progress. Providing an optional feedback channel to report that progress would probably greatly enhance the progress accuracy compared to the "manual" tracking method. Off the top of my head, this should be possible for the current Spotify- and mpv-PlaybackFactory implementations, covering just about all of today's bot configurations.

There already is a feedback channel for each Playback object to report state changes. It's currently limited to play/pause/error events, but it could probably be extended to allow Playback objects to accurately report their playback progress.

Should we add it?

From the server/plugin-perspective, I think it's doable with (hopefully) not too much overhead and without complicating plugin-development too much, but probably with some inaccuracy. In any case it'll take a while until everything is implemented.

From your more client-oriented perspective, do you agree with everything so far? I'd like to hear your thoughts on this.

[ivoberger]

You're raising valid points and I do agree that having a progress is a better solution. My thinking came mainly from the client-side polling of the player status and the accompanying update delay.
I'd be in favor of adding a nullable progress field to the current SongEntry and then plugin support can be added gradually. If it's not there yet for some plugin it can simply be left out (which of course also applies when there is no duration). Also I think somewhat inaccurate progress is still a better UX than none at all as long as the deviations aren't too large.