thatnerduknow/vmaf-gui

A GUI built using C# to make Netflix's VMAF easier to use

VMAF-GUI

Replacement

I'm currently working on replacing the c# version of this project into rust+react using the tauri framework. View the progress of this effort in the following project: https://github.com/users/ThatNerdUKnow/projects/3/views/1 This should resolve most of the issues of the current version as all processing is done in-memory, which means that we don't have to decode videos to YUV files on the filesystem. Also, I'm able to get progress updates on vmaf score calculation!

About

VMAF is a tool developed by Netflix to grade the quality of a compressed video file in comparison to the Original. However, VMAF is only available through the commandline or as an extension to FFMPEG. If you're like me, you find the functionality of VMAF very useful but very time consuming and confusing to use at first. This tool I have built simply adds a gui to the program to make it easy to use.

Build

Using visual studio Just click the .sln file then compile for your system. In the build folder you'll have to provide your own copy of ffmpeg, vmaf, and the model files for vmaf in a subfolder named model.

vmaf-gui's Issues

"System.FormatException: The input string format is incorrect"

Using 2.1.3, I'm getting this error. I've linked the screenshot of it (parts of it are in French)

Both files are in .mp4, the source is in H264 and the compressed one is in AV1

"There was a problem running VMAF"

Tried to compare two H.264 MKV files. Got this error.

Closing the application doesn't end the process and a few other bugs

Hello,

Straight to the point:

• Once the VMAF score has been presented and the log.xml opened and shown to the user, exiting VMAF-GUI by pressing the 'x' doesn't end the process. The exe is still running.
• Exiting midway through a comparison closes the application, but leaves the ffmpeg process running and continuing the decompress the file(s).

Selectable OS Threads

Currently the vmaf invocation will always use 4 threads. Setting threads to 0 uses only a single thread. We'd have to detect the number of is threads and inject it into the vmaf invocation

Possible to add an output to a plot where the vmaf values are plotted on a graph or a table?

Something like this.

Episode | Length | Size | Q:Avg | Q:75% | Q:25% | Q:01%
--------+---------+-------+-------+-------+-------+------
S02E01 | 55:48 | 340M | 94.30 | 95.53 | 92.46 | 83.12
S02E02 | 54:09 | 323M | 94.20 | 95.41 | 92.17 | 81.80
S02E03 | 55:10 | 288M | 94.03 | 95.21 | 92.21 | 82.81
S02E04 | 54:43 | 293M | 93.68 | 95.04 | 91.71 | 82.71
S02E05 | 56:57 | 301M | 94.02 | 95.13 | 92.45 | 83.68
S02E06 | 58:08 | 323M | 94.19 | 95.40 | 92.48 | 83.94
S02E07 | 55:19 | 297M | 94.04 | 95.42 | 92.29 | 84.77
S02E08 | 56:03 | 332M | 93.97 | 95.49 | 91.79 | 81.31

Add selectable vmaf models

as suggested in issue #6
Users should be able to select from the provided vmaf models

Would it be possible to add a pause and ETA feature?

Hello,

Firstly, thank you for writing such a straight-forward GUI for VMAF. It's incredibly useful. Using a 12GB x265 file as "Source", and a 2.5GB x265 file as "Compressed", the decompressing is taking a significant amount of time, which is expected. The uncompressed file is already at 306GB!

If possible, it would be useful to have a few extra features:

• The progress bar appears to be static with larger files. Over 500GB should show something in the progress bar.
• If possible, a very approximate ETA would be highly useful. Even a piece of text stating that 'an x GB source file is likely to take x minutes' so the user has a rough idea of how long the process is likely to take.
• A 'pause' button would also be useful, and as of writing this sentence the 12GB source file is now at 516GB and I need to remove some files in order to free up space.
• Is it possible to automatically delete the temp directory after the log is displayed?
• I had to exit the application as the temp file reached over 540GB, leaving me with about 30GB of space. However, does stop the application, but doesn't stop ffmpeg from continuing to process the file. The only way to stop ffmpeg is to force-close. I will create a new bug report.

What's your view on these requests?

Regards,
Piers

2.0 examines first 3120 frames and exits

Tried today's release out. It examines the first 3120 frames and exits
log.zip
.

Huge .y4m file size (~20–30 ✕ original)

The issue

source.y4m size: 284.3 G (305 255 244 908 bytes)

compressed.y4m size: 88.5 G (94 981 849 088 bytes)

Source size: 14.7 G (15 749 803 095 bytes)

Compressed size: 3.4 G (3 670 213 684 bytes)

Additional info

Eventually, vmafgui exits with

---------------------------
There was a problem with VMAF
---------------------------
Error reading YUV frame data.



problem reading pictures

problem flushing context


---------------------------
OK   
---------------------------

the .y4m files are not deleted.

HUGE temporary files left behind

The application seems to do the VMAF analysis in two steps: decompress the video(s?) first into temporary files, then do the VMAF analysis over the decompressed video(s).

When I do a VMAF run, a "temp" folder is created inside the application path, this is where the decompressed files are stored. After the VMAF run is done, and/or the application is exited, the temporary files in this "temp" folder remain there, they are NOT deleted.

This would not be of much concern, if the files weren't INSANELY huge. For reference: a 4:22 minute 1080p/24fps reference video that was encoded from 100 MB into a 75 MB distorted 1080p/24fps video left behind two 25 GB files (=50 GB in total) in the "temp" folder.

I haven't tested it, but this would mean "temporary" files of nearly 500 GB could be generated (and left behind) for a simple VMAF run over a 45 minute 1080p/24fps video clip.

If this is really the case, then the app should make sure...
A) that this space is actually available on the drive where the application is located, or
B) that the user can select a different location (e.g. different drive) for those temporary files, and/or
C) that "temporary" files are actually temporary and get deleted after a VMAF analysis run completes.

Alternatively, FFMPEG offers the ability to decompress "on the fly", to my understanding without any 500 GB intermediate temporary files. If I do CLI VMAF runs, I call FFMPEG like this:

ffmpeg.exe -threads 0 -i V:\Distorted.mp4 -i V:\Reference.mov -threads 0 -lavfi libvmaf="model='path=C\\\:/ffmpeg/vmaf/vmaf_v0.6.1.json':log_fmt=xml:log_path=V\\\:/Distorted.vmaf.xml" -f null -

This assumes that...

• the bin location for ffmpeg.exe is set in PATH
• reference and distorted videos are on drive V:
• the VMAF model files are installed in C:\ffmpeg\vmaf\*.json
• the output log file is supposed to go next to the distorted file on V:

Like this, I can have FFMPEG chug through a video file entirely and do VMAF analysis without generating (and leaving behind) any temporary files, especially not 500 GB worth of them.

Maybe that's an option for this application as well?

Stuck on 'Decompressing source'

No idea why it happens just does.

Keeps getting stuck on decompressing source and then nothing happens

Source video
is a .mp4
Compressed video is a .webm

Maybe has something to do with that, not sure tho.

VMAF-GUI doesnt generate output

Hi,
VMAF-GUI doesn't actually generate a output file. It only displays the one it comes with and if i delete that i can't write a new one.

It is creating both source and compressed .yuv files in its temp directory, so i don't think its a permission issue.

EDIT:
I am using the latest version 1.2.4 extracted from 7z File