Refine encoder configuration of bitrate and/or quantization

[w3c]

Refine encoder configuration of bitrate and/or quantization

[sandersdan]

We'll need to document what options are available from common platform decoders, for this and for other tuning parameters. I'll create a tag for that.

[sandersdan]

Android: starting in L (API level 21), Android encoders optionally support CBR, CQ, and VBR modes. It is possible to query in advance to check if a mode is supported, so we have a choice of fallback options.

I'm not quite sure how an app achieves a target bitrate using this mechanism, especially on Android where it cannot be controlled per-frame. It may be best to expose the options separately, then the WebCodecs implementation can do its best given what the codec implementation offers.

[chcunningham]

Re: title update, we agree the app should be in control. Just a matter of deciding on the knobs.

[chcunningham]

Re: milestone, this is a very important issue, but its not a launch blocker. We will add knobs for QC, CBR, VBR etc, and we have some research still to do on how best to express these concepts (e.g. is QC done in a per-codec dictionary, or in the top level config?). The existing bitrate knob is here to stay. Later additions of other knobs may interact with it (e.g. CBR vs VBR would affect the time window where target bitrate is computed), but additions are not breaking changes.

[chcunningham]

triage note: marked 'extension', as discussion proposes to add new members to config dict.

[Yahweasel]

I would love to see constant-quality modes, and feel that that option is more important than everything in between (ABR window sizes, bitrate ranges, etc). For just constant quality, I'd like to suggest communicating the name in a way that's a bit less codec-y. "Quantization" doesn't mean anything if you aren't a digital media expert. How about just "quality", with codec-specific meaning, but a codec-agnostic range from 0 to 1 (or 0 to 100), where 1 (or 100) means lossless? The mapping could be centrally defined as part of the codec registry, or could be implementation-defined. CCQ modes could be done by specifying both bitrate and quality, and neither of those need to expose codec details to be useful.

[chcunningham]

Sounds interesting.

You mentioned constant-quality modes... we already have config.bitrateMode w/ values of "variable" and "constant". Would bitrateMode + a new quality knob meet your needs?

I didn't follow your last sentence about CCQ... what does that stand for? Can you give a example of how this would work?

[Yahweasel]

Variable bitrate mode is good. I personally just prefer constant quality modes, e.g. CRF in H.264 or constant quantizer in other encodings. The thing with ABR/VBR modes is that they're still bitrate-centric, and an average bitrate in one encoding yields a very different quality than the same average bitrate in another encoding. Although it's not meaningful to equate constant quality modes across different encoders, it's even less meaningful to equate bitrates if what you care about is quality.

re CCQ, sorry, I meant constrained constant quality modes, which are usually abbreviated ambiguously to "CQ", not CCQ. I always say "CCQ" because "CQ" really should just mean "constant quality", even though it never does :) . e.g. in VP9 you can set a quantizer but also a constraint on the bitrate.

[hlevring]

I think this is incredibly important for the video recording use case. @chcunningham I don't think webcodec will be practical for more serious recording use cases until it gets exactly that nob to set a quality constraint or target a quality.

For example in our case, the content we record many times involves both some full motion camera scenes and then some scenes which are mostly static static content combined with a camera PiP. Just having a bitrate option forces you to set the bitrate very high to get decent quality on the full motion scenes, but then you basically waste a bunch of bits on all the scenes with static content + pip. - For recording you really need some ability to target some type of quality factor or QP.

Any idea around ETA or how this is currently prioritized?

[dalecurtis]

#633 and #642 both attempt to address this. Support for VPX/AV1 QP mode should be forthcoming in Chrome. It'll be harder to get H.264 QP support since it varies by platform / api.

@Djuffin

[hlevring]

Thanks for pointing to those issues. For our use case H.264 is the only codec we can use for now, but I will follow the development and hope it will land sooner than later.

[aboba]

Is this issue still open, or has it been addressed by #633 and #642?

[Djuffin]

Closing it as implemented