hate5six / hate5sync Goto Github PK
View Code? Open in Web Editor NEWcode for computing necessary sync offset for a/v in obs
code for computing necessary sync offset for a/v in obs
Please help me understand, because I got very confused by this part of the README.md (which corresponds to lines 37-41 of the script):
We could look for the max brightness value, but quick tests indicate that finding the max change in brightness is more accurate. It makes sense: if you look at the above plot the peak isn't a sharp point, it spikes up then increases for a few frames before dropping down. We care about the initial jump. For each consecutive brightness values we'll compute the successive differences, then find the frame that contained the biggest jump:
pairs_bright = list(((b-a) for a,b in zip(brightness, brightness[1:]))) peak_video = np.argmin(pairs_bright)
The prose says "biggest jump", but the code seems to look for the biggest fall. So then I thought that I didn't know what the list brightness
stored, so I went back:
We use open-cv to convert each frame to the LAB colorspace. All we care about the L channel, which captures lightness (intensity). L ends up being a matrix the size of the video frame (1920x1080). We normalize it so each value in the matrix is between 0-1. Then we compute the average value of that normalized matrix to get the average brightness of that image. Store that in a list called brightness and repeat the process on the next frame.
Per Wikipedia article on Lab color space:
The lightness value, L*, also referred to as "Lstar," defines black at 0 and white at 100.
And the graph from the README seems to correspond to the notation black at 0.0 and white at 1.0 (not 100 because normalized):
I even tried running some numbers, similar to the graph in python repl myself:
>>> import numpy as np
>>> brightness=[0.92, 0.91, 0.92, 0.91, 0.93, 0.91, 0.91, 0.91, 0.91, 1.0, 1.01, 1.02, 0.91, 0.92, 0.92, 0.88, 0.88, 0.92, 0.92, 0.92 ]
>>> pairs_bright = list(((b-a) for a,b in zip(brightness, brightness[1:])))
>>> np.argmin(pairs_bright)
11
>>> np.argmax(pairs_bright)
8
argmax
returns the index of where the jump from 0.91 to 1.0 happens and argmin
returns the index of where the drop from 1.02 to 0.91 happens.
So my question is: shouldn't the algorithm use np.argmax
to find, quote, "the biggest jump"?
Hey I'm getting this error when running this script on the attached video.
Trying to sync my elgato game capture footage to my microphone.
My series of errors includes this:
E:/hate5sync.py:30: RuntimeWarning: invalid value encountered in true_divide
L_norm = L/np.max(L)
Traceback (most recent call last):
File "E:/hate5sync.py", line 78, in
peak_audio = compute_audio_peak(video_f
ile, fps)
File "E:/hate5sync.py", line 52, in compute_audio_peak
ffmpeg.input(video_file).output(audio_stream).run()
AttributeError: module 'ffmpeg' has no attribute 'input'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.