rolinh/vqmt

VQMT: Video Quality Measurement Tool. Fast implementations of the following objective image quality metrics: PSNR, SSIM, MS-SSIM, VIFp, PSNR-HVS and PSNR-HVS-M.

VQMT - Video Quality Measurement Tool

This software provides fast implementations of the following objective metrics:

• PSNR: Peak Signal-to-Noise Ratio,
• SSIM: Structural Similarity,
• MS-SSIM: Multi-Scale Structural Similarity,
• VIFp: Visual Information Fidelity, pixel domain version
• PSNR-HVS: Peak Signal-to-Noise Ratio taking into account Contrast Sensitivity Function (CSF),
• PSNR-HVS-M: Peak Signal-to-Noise Ratio taking into account Contrast Sensitivity Function (CSF) and between-coefficient contrast masking of DCT basis functions.

In this software, the above metrics are implemented in C++ with the help of OpenCV and are based on the original Matlab implementations provided by their developers. The source code of this software can be compiled on any platform and only requires the OpenCV library (core and imgproc modules). This software allows performing video quality assessment without using Matlab and shows better performance than Matlab in terms of run time.

PREREQUISITE

The OpenCV library (http://opencv.willowgarage.com/wiki/) needs to be installed to be able to compile this code. Only the core and imgproc modules are required.

BUILD

CMake is required in order to build VQMT. A Makefile is provided to ease the building step:

make

This command has the effect of creating the build directory, calling cmake within it and building VQMT. The binary may then be found in build/bin/Release.

USAGE

vqmt (or VQMT.exe on Windows) OriginalVideo ProcessedVideo Height Width NumberOfFrames ChromaFormat Output Metrics

• OriginalVideo: the original video as raw YUV video file, progressively scanned, and 8 bits per sample
• ProcessedVideo: the processed video as raw YUV video file, progressively scanned, and 8 bits per sample
• Height: the height of the video
• Width: the width of the video
• NumberOfFrames: the number of frames to process
• ChromaFormat: the chroma subsampling format. 0: YUV400, 1: YUV420, 2: YUV422, 3: YUV444
• Output: the name of the output file(s)
• Metrics: the list of metrics to use

Available metrics:

• PSNR: Peak Signal-to-Noise Ratio (PNSR)
• SSIM: Structural Similarity (SSIM)
• MSSSIM: Multi-Scale Structural Similarity (MS-SSIM)
• VIFP: Visual Information Fidelity, pixel domain version (VIFp)
• PSNRHVS: Peak Signal-to-Noise Ratio taking into account Contrast Sensitivity Function (CSF) (PSNR-HVS)
• PSNRHVSM: Peak Signal-to-Noise Ratio taking into account Contrast Sensitivity Function (CSF) and between-coefficient contrast masking of DCT basis functions (PSNR-HVS-M)

Example:

VQMT.exe original.yuv processed.yuv 1088 1920 250 1 results PSNR SSIM MSSSIM VIFP

will create the following output files in CSV (comma-separated values) format:

• results_pnsr.csv
• results_ssim.csv
• results_msssim.csv
• results_vifp.csv

Notes:

• SSIM comes for free when MSSSIM is computed (but you still need to specify it to get the output)
• PSNRHVS and PSNRHVSM are always computed at the same time (but you still need to specify both to get the two outputs)
• When using MSSSIM, the height and width of the video have to be multiple of 16
• When using VIFP, the height and width of the video have to be multiple of 8

COPYRIGHT

Permission is hereby granted, without written agreement and without license or royalty fees, to use, copy, modify, and distribute the software provided and its documentation for research purpose only, provided that this copyright notice and the original authors' names appear on all copies and supporting documentation. The software provided may not be commercially distributed. In no event shall the Ecole Polytechnique Fédérale de Lausanne (EPFL) be liable to any party for direct, indirect, special, incidental, or consequential damages arising out of the use of the software and its documentation. The Ecole Polytechnique Fédérale de Lausanne (EPFL) specifically disclaims any warranties. The software provided hereunder is on an "as is" basis and the Ecole Polytechnique Fédérale de Lausanne (EPFL) has no obligation to provide maintenance, support, updates, enhancements, or modifications.

REFERENCES

• Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, April 2004.
• Z. Wang, E.P. Simoncelli, and A.C. Bovik, "Multiscale structural similarity for image quality assessment," in IEEE Asilomar Conference on Signals, Systems and Computers, November 2003, vol. 2, pp. 1398–1402.
• H.R. Sheikh and A.C. Bovik, "Image information and visual quality," IEEE Transactions on Image Processing, vol. 15, no. 2, pp. 430-444, February 2006.
• K. Egiazarian, J. Astola, N. Ponomarenko, V. Lukin, F. Battisti, and M. Carli, "New full-reference quality metrics based on HVS," in Proceedings of the Second International Workshop on Video Processing and Quality Metrics, 2006.
• N. Ponomarenko, F. Silvestri, K. Egiazarian, M. Carli, J. Astola, and V. Lukin, "On between-coefficient contrast masking of DCT basis functions," in Proceedings of the Third International Workshop on Video Processing and Quality Metrics for Consumer Electronics, January 2007.

vqmt's Issues

Upgrade

Any plan to upgrade PSNR-HVS and PSNR-HVSM to PSNR-HA and PSNR-HMA?

Question: Version of VQMT code

VQMT official project shares its code at https://mmspg.epfl.ch/files/content/sites/mmspl/files/shared/VQMT/VQMT_Sources.rar

So does the VQMT code used here is in sync with official codebase?

Python wrappers

I would like to use this through Python libraries.

Improve arguments parsing

Argument parsing is currently very weak and not user friendly. This needs to be improved. Requiring the Boost library just for this is probably overkill so maybe docopt.cpp provides a good alternative for options parsing.

i cannot use vqmt

when I use "vqmt -h"
return error : no such file or directory

Fail to build for arches other than i386/amd64

Is the include for sys/io.h in inc/VideoYUV.hpp is necessary ?
Removing this include doesn't break the build for i386 and amd64 and allow to build for arm64, armel, armhf and powerpc.

Does this have anything to do with the version of opencv?

PSNR-HVS-M function fails on this image comparison.

PSNR_HVS_M calculation fails with these two images (after decoding the jpg to bmp, padding if necessary) and comparing the two.

The error bubbles up from the compute / maskeff function in PSNRHVS.cpp, where the line "float pop = vari(z)" returns 0. Then the subsequent calculation divides by pop, which of course is NaN, then it is all downhill from there.

No pipe support

Add please support for named pipes, so one could use ffmpeg for comparison. And not waste time and disk space for comparison.

Other Quality Assessment algorithms

• aizvorski/video-quality#8
  • VIF (sometimes called VIF-P or VIFP), Visual Information Fidelity
  • RECO, Relative Polar Edge Coherence
  • NIQE, Natural Image Quality Evaluator
  • 3SSIM, 3-Component Structural Similarity Metric
  • VQUAD-HD
  • VQM
  • UIQ, Universal Image Quality
  • MSSIM, Motion SSIM
  • VMAF

How do I use this similar function using Python

Need support using Python if possible

readOneFrame error

I am gettign the following error when I run vqmt

readOneFrame: cannot read 1280 bytes from the input file, unexpected EOF.

Am I doing something wrong

I launched the tool using ./vqmt video1.mp4 video2.mp4 720 1280 80 1 results SSIM

Adding License to SPDX Listing?

Given the license and source code have remained stable since 2013, would it be acceptable to have the license submitted for inclusion on the SPDX list (see. https://spdx.org/licenses/) per the guidelines (see. https://github.com/spdx/license-list-XML/blob/master/DOCS/license-inclusion-principles.md)?

The aim would be to aid in compliance with your terms (particularly the non-commercial requirement) through detection in SBOM generation.

MSSSIM height check

Can't compare images smaller than 160px.

bash -c "vqmt tmp.y4m 58.convert.y4m 144 144 1 3 tmp PSNR SSIM MSSSIM VIFP PSNRHVS PSNRHVSM"
OpenCV Error: Assertion failed (s >= 0) in setSize, file /home/svasilev/opencv/opencv-3.3.0/modules/core/src/matrix.cpp, line 310
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/svasilev/opencv/opencv-3.3.0/modules/core/src/matrix.cpp:310: error: (-215) s >= 0 in function setSize

files.zip

Got this files with following commands

convert 110086_x.png -resize 144x144 tmp.png
convert tmp.png -quality 58 -sampling-factor 1x1 58.convert.jpg
convert 58.convert.jpg 58.convert.png
ffmpeg -y -loglevel panic -i tmp.png -pix_fmt yuv444p tmp.y4m
ffmpeg -y -loglevel panic -i 58.convert.png -pix_fmt yuv444p 58.convert.y4m

160 and bigger goes well.

which paper can refer?

PSNRHVSM width and height check

My test video has resolution 2580x1080. If I run PSNRHVSM (or PSNRHVS) metric computation vqmt crashes

OpenCV Error: Assertion failed (0 <= _colRange.start && _colRange.start <= _colRange.end && _colRange.end <= m.cols) in Mat, file /build/opencv/src/opencv-3.4.0/modules/core/src/matrix.cpp, line 501
terminate called after throwing an instance of 'cv::Exception'
  what():  /build/opencv/src/opencv-3.4.0/modules/core/src/matrix.cpp:501: error: (-215) 0 <= _colRange.start && _colRange.start <= _colRange.end && _colRange.end <= m.cols in function Mat

After I convert sample to 2584x1080 (dimensions dividable by 8) the computation goes well. So, dimensions of video should be dividable by 8. In daala tool dump_psnrhvs has no such restriction.

readOneFrame: cannot read 3840 bytes from input file, unexpected EOF.

I am trying to compare two mp4 videos using the command:
./vqmt bbb_source.mp4 bbb_h264_ultrafast_crf20.mp4 2160 3840 38072 1 results PSNR SSIM MSSSIM VIFP
both files are with 3840x2160px resolution and they do not have audio tracks, the pixel format is yuv420p and they the duration is 38072 frames.

When I checked the results I saw couple of result files. It appears I have results only for the first 50 frames:

cat results_psnr.csv
frame,value
0,7.295123
1,7.671892
2,7.670082
3,7.663648
4,7.693020
5,7.671082
6,7.698746
7,7.670381
8,7.689044
9,7.710546
10,7.693920
11,7.681756
12,7.647830
13,7.693138
14,7.679656
15,7.669047
16,7.713078
17,7.772104
18,7.759742
19,7.755799
20,7.698050
21,7.643544
22,7.720232
23,7.744545
24,7.738848
25,7.729941
26,7.746346
27,7.743903
28,7.741730
29,7.745615
30,7.716219
31,7.683934
32,7.743063
33,7.745136
34,7.740988
35,7.729583
36,7.716391
37,7.727443
38,7.740869
39,7.720695
40,7.733064
41,7.715613
42,7.679746
43,7.703423
44,7.715934
45,7.715479
46,7.753919
47,7.677543
48,7.707615
49,7.651045
50,7.539806