hate5six/hate5sync

code for computing necessary sync offset for a/v in obs

hate5sync's Issues

Shouldn't the algorithm use np.argmax to find biggest jump in brightness?

Please help me understand, because I got very confused by this part of the README.md (which corresponds to lines 37-41 of the script):

We could look for the max brightness value, but quick tests indicate that finding the max change in brightness is more accurate. It makes sense: if you look at the above plot the peak isn't a sharp point, it spikes up then increases for a few frames before dropping down. We care about the initial jump. For each consecutive brightness values we'll compute the successive differences, then find the frame that contained the biggest jump:

pairs_bright = list(((b-a) for a,b in zip(brightness, brightness[1:])))
peak_video = np.argmin(pairs_bright)

The prose says "biggest jump", but the code seems to look for the biggest fall. So then I thought that I didn't know what the list brightness stored, so I went back:

We use open-cv to convert each frame to the LAB colorspace. All we care about the L channel, which captures lightness (intensity). L ends up being a matrix the size of the video frame (1920x1080). We normalize it so each value in the matrix is between 0-1. Then we compute the average value of that normalized matrix to get the average brightness of that image. Store that in a list called brightness and repeat the process on the next frame.

Per Wikipedia article on Lab color space:

The lightness value, L*, also referred to as "Lstar," defines black at 0 and white at 100.

And the graph from the README seems to correspond to the notation black at 0.0 and white at 1.0 (not 100 because normalized):

I even tried running some numbers, similar to the graph in python repl myself:

>>> import numpy as np
>>> brightness=[0.92, 0.91, 0.92, 0.91, 0.93, 0.91, 0.91, 0.91, 0.91, 1.0, 1.01, 1.02, 0.91, 0.92, 0.92, 0.88, 0.88, 0.92, 0.92, 0.92 ]
>>> pairs_bright = list(((b-a) for a,b in zip(brightness, brightness[1:])))
>>> np.argmin(pairs_bright)
11
>>> np.argmax(pairs_bright)
8

argmax returns the index of where the jump from 0.91 to 1.0 happens and argmin returns the index of where the drop from 1.02 to 0.91 happens.

So my question is: shouldn't the algorithm use np.argmax to find, quote, "the biggest jump"?

Error: invalid value encountered in true_divide L_norm = L/np.max(L)

Hey I'm getting this error when running this script on the attached video.

Trying to sync my elgato game capture footage to my microphone.

video-file.mp4

My series of errors includes this:

E:/hate5sync.py:30: RuntimeWarning: invalid value encountered in true_divide
L_norm = L/np.max(L)
Traceback (most recent call last):
File "E:/hate5sync.py", line 78, in
peak_audio = compute_audio_peak(video_f

ile, fps)
File "E:/hate5sync.py", line 52, in compute_audio_peak
ffmpeg.input(video_file).output(audio_stream).run()
AttributeError: module 'ffmpeg' has no attribute 'input'