decord._ffi.base.DECORDError from VideoReader

[dmlc]

decord._ffi.base.DECORDError from VideoReader

[THUSIGSICLAB]

I meet the same problem

[prismformore]

Try num_threads=1

[btekin]

Setting num_threads=1 in VideoReader resolves the issue for me.

[innerlee]

Which version of decord, and could you share a corrupted video (ideally a small-sized one)?

Edit: sorry, saw the video link above

[innerlee]

BTW how to deal with corrupted frames? Raise an error or a warning? Return a black frame?

[Lijun-Yu]

It depends on what the user cares about, e.g. object detection (just warning and skip would be fine), tracking (maybe reuse the previous frame), optical flow (maybe an interpolated frame). I guess it would be better to offer options for users to choose from. Some packages I have tried:

• OpenCV (cv2.VideoCapture): skip without warning in Python, but FFmpeg prints obscure warnings (like the ones above such as [NULL @ 0x7fde3829b200]). It is not desired when you care about the frame index to align with annotation.
• Pims with PyAV: return a black frame without warning in Python, also warnings from FFmpeg. It affects the detection and tracking performance.
• MoviePy: return the previously available frame without warning.
• AVI-R: option to skip or return previously available frame; option for warning.

[innerlee]

I tested the video. The pts are all -9223372036854775808, i.e., AV_NOPTS_VALUE. The current (master) logic of decord relies on pts. One improvement of the logic is to use dts when pts is not available.

However, even when replace the null pts with dts, decord still cannot decode this video.

That's what I know by far.

[zhreshold]

@Lijun-Yu corrupted packet can cause more complicated situtations, for example, a single corrupted frame can affect all consequential frame decoding until the next keyframe. Multiple corrupted frame across the entire video can make it completely useless during training.

decord is intended for training models and the best hander for corrupted frame is to throw the error and let user determine by

• catch the DecordError and throw away the entire video
• try catch and locate the corrupted frame range, and only access the valid part of the video(this can be improved by decord by returning valid range maybe?)

Any thoughts?

[Lijun-Yu]

@zhreshold Agreed, corrupted frames should not be used in training. But there can still be situations where people want to decode a corrupted video for inference.

I guess for the VideoReader, it could be enough to just throw the error and let users decide whether to throw away the video/batch/batch or try to recover it. Locating the corrupted frame range may require decoding the whole video, which does not seem efficient in this direct access context. But it could be better to include documentation stating that errors can be raised in certain conditions.

For the VideoLoader, it may be better for users to control its behavior by choosing from some options, e.g. skip the batch, skip the video, or try to recover it first and skip if failed. Or the errors raised from its internal loading procedure could cause a mess. Just some initial thoughts.