Encoder and decoder stats

[w3c]

Encoder and decoder stats

[pthatcherg]

I like the idea of the output of the decoder (DecodedAudioPacket or DecodedVideoFrame) having a few useful things like .decodeTime . Similarly for EncodedVideoFrame and .encodeTime and .qp.

[chcunningham]

Per-frame encode/decode durations. Real-time applications may use this to decrease the quality of the media stream if the encoder or decoder is taking too long.

I think knowing about "too long" cases may be possible without these statistics. The app can observe the delta between pushing frames in and getting corresponding (same timestamp) frames out. Making this observation in javascript is probably superior to providing the codecs view of the world, as the codec doesn't know how much delay the app may encounter between the codec being "done" with the frame and the app actually receiving it from the output callback.

Per-frame quality information. For example, getting the QP value of a video frame.

My naive model is that QP will be an input encoder.configure(), and that every output would then have that QP (making it not interesting to add as an attribute). Is that a reasonable model?

[padenot]

My naive model is that QP will be an input encoder.configure(), and that every output would then have that QP (making it not interesting to add as an attribute). Is that a reasonable model?

Not really. For a really common example, using x264 in CRF mode (which is recommended: "This is the recommended rate control mode for most uses. ") is essentially changing the QP per frame. Here is a good article I've bookmarked that talks about it in a bit more depth: https://slhck.info/video/2017/02/24/crf-guide.html#crf-versus-constant-qp (direct link to the section about this, but the rest is interesting).

[chcunningham]

Thanks @padenot, I'll give that a read. For QP discussion, we have #56. Happy to have separate issues for the codec specific needs.

For the original issue, we haven't had demand for such stats from actual users and I expect the measurements I outlined are good enough if not better. Given ongoing privacy discussion about timing attacks, I vote to close this ancient issue until we have have motivating use cases. Please feel free to reopen if there's more to discuss.