netcazean/vmaf

Perceptual video quality assessment based on multi-method fusion.

VMAF - Video Multi-Method Assessment Fusion

VMAF is a perceptual video quality assessment algorithm developed by Netflix. This software package includes a stand-alone C library libvmaf and its wrapping Python library. The Python library also provides a set of tools that allows a user to train and test a custom VMAF model. Read this tech blog post for an overview, this post for the tips of best practices, and this post for our latest efforts on speed optimization, new API design and the introduction of a codec evaluation-friendly NEG mode.

Also included in libvmaf are implementations of several other metrics: PSNR, PSNR-HVS, SSIM, MS-SSIM and CIEDE2000.

News

• (2020-12-7) Check out our latest tech blog on speed optimization, new API design and the introduction of a codec evaluation-friendly NEG mode.
• (2020-12-3) We are releasing libvmaf v2.0.0. It has a new fixed-point and x86 SIMD-optimized (AVX2, AVX-512) implementation that achieves 2x speed up compared to the previous floating-point version. It also has a new API that is more flexible and extensible.
• (2020-7-13) We have created a memo to share our thoughts on VMAF's property in the presence of image enhancement operations, its impact on codec evaluation, and our solutions.
• (2020-2-27) We have changed VMAF's license from Apache 2.0 to BSD+Patent, a more permissive license compared to Apache that also includes an express patent grant.

Frequently Asked Questions

Refer to the FAQ page.

Usages

The software package offers a number of ways to interact with the VMAF implementation.

• The command-line tool vmaf provides a complete algorithm implementation, such that one can easily deploy VMAF in a production environment. Additionally, the vmaf tool provides a number of auxillary metrics such as PSNR, SSIM and MS-SSIM.
• The C library libvmaf provides an interface to incorporate VMAF into your code, and tools to integrate other feature extractors into the library.
• The Python library offers a full array of wrapper classes and scripts for software testing, VMAF model training and validation, dataset processing, data visualization, etc.
• VMAF is now included as a filter in FFmpeg, and can be configured using: ./configure --enable-libvmaf. Refer to the Using VMAF with FFmpeg page.
• VMAF Dockerfile generates a docker image from the Python library. Refer to this document for detailed usages.
• To build VMAF on Windows, follow this instruction.

Datasets

We also provide two sample datasets including the video files and the properly formatted dataset files in Python. They can be used as sample datasets to train and test custom VMAF models.

Models

Besides the default VMAF model which predicts the quality of videos displayed on a 1080p HDTV in a living-room-like environment, VDK also includes a number of additional models, covering phone and 4KTV viewing conditions. Refer to the models page for more details.

Confidence Interval

Since June 2018, we have introduced a way to quantify the level of confidence that a VMAF prediction entails. Each VMAF prediction score now can come with a 95% confidence interval, which quantifies the level of confidence of the prediction. Refer to the VMAF Confidence Interval page for more details.

Matlab Functionality

Besides the C and Python part of the software package, we also introduced a number of algorithms that are implemented in Matlab. For example, users can calculate ST-RRED, ST-MAD, SpEED-QA, and BRISQUE. For more details, see the Matlab Usage page for more details.

Report of VMAF Bad Cases

Over time, we have received feedbacks on when VMAF's prediction does not reflect the expected perceptual quality of videos, either they are corner cases where VMAF fails to cover, or new application scenarios which VMAF was not initially intended for. In response to that, we have created a Google form to allow users to upload their video samples and describe their cases. The bad cases are valuable for improving future versions of VMAF. Users can opt in or out for sharing their sample videos publicly. Please submit the bad cases through this link.

References

Refer to the references page.

External Resources

Refer to this page for a list of 3rd-party tools built on top of VMAF.