sudheerachary/mesh-flow-video-stabilization

Online video stabilization using a novel MeshFlow motion model

mesh-flow-video-stabilization's Issues

How I fixed some bugs and ran the code successfully

I faced some issues when running the code with newer versions of python and OpenCV. This is how I fixed them and managed to run the code successfully:
Make the following adjustment in Stabilization.py:

line  23: print(method.__name__+' has taken: '+str(end_time-start_time)+' sec')
line 265: print('Time elapsed: ', str(time.time()-start_time))
line  60: VERTICAL_BORDER = int((HORIZONTAL_BORDER*old_gray.shape[1])/old_gray.shape[0])
line  66: x_paths = np.zeros((int(old_frame.shape[0]/PIXELS), int(old_frame.shape[1]/PIXELS), 1))
line  67: y_paths = np.zeros((int(old_frame.shape[0]/PIXELS), int(old_frame.shape[1]/PIXELS), 1))

Make the following adjustment in MeshFlow.py:

line  43: cols, rows = int(old_frame.shape[1]/PIXELS), int(old_frame.shape[0]/PIXELS)
line  80: x_motion_mesh[key] = x_motion[key]+temp_x_motion[key][int(len(temp_x_motion[key])/2)]
line  85: y_motion_mesh[key] = y_motion[key]+temp_y_motion[key][int(len(temp_y_motion[key])/2)]

Error reading the video

Hello, when I run:
python Stabilization.py ../data/shaky-5.avi
or
python Stabilization.py ../data/small-shaky-5.avi
I get the following error:

OpenCV Error: Assertion failed (scn == 3 || scn == 4) in ipp_cvtColor, file /home/travis/miniconda/conda-bld/conda_1486587069159/work/opencv-3.1.0/modules/imgproc/src/color.cpp, line 7456
Traceback (most recent call last):
File "Stabilization.py", line 252, in
x_motion_meshes, y_motion_meshes, x_paths, y_paths = read_video(cap)
File "Stabilization.py", line 21, in timed
result = method(*args, **kwargs)
File "Stabilization.py", line 52, in read_video
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)
cv2.error: /home/travis/miniconda/conda-bld/conda_1486587069159/work/opencv-3.1.0/modules/imgproc/src/color.cpp:7456: error: (-215) scn == 3 || scn == 4 in function ipp_cvtColor

How can I solve this? Should I use another version of opencv? Thank you!

cvxpy version

Which is the exact version of cvxpy?
I get a ModuleNotFoundError when importing 'canonInterface' in cvxpy/utilities/quadratic.py.

an error or imperfection when the frame don't have any feature point

In mesh_flow.ipynb , an error occur when one frame has no feature point:
# calculate optical flow
p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)

This problem occur definitely when a video has a blank or black frame, can you provide a more rebust code for this solution.

Thank you very much!

error

x_paths = np.zeros((old_frame.shape[0]/PIXELS, old_frame.shape[1]/PIXELS, 1))

TypeError: 'float' object cannot be interpreted as an integer

while running this code i got error

numpy.core.multiarray failed to import

SyntaxError: invalid syntax

Tried to run: python Stabilization.py shaky-5.avi

with the shaky-5.avi in the same src directory, and got this:
File "Stabilization.py", line 23
print method.name+' has taken: '+str(end_time-start_time)+' sec'
^
SyntaxError: invalid syntax

Is the stabilized video cropped?

empty H matrix

Hi,
when I run './data/shaky-5.avi', i get the following error:
File "/home/ubuntu/Video_Stabilization/Mesh-Flow-Video-Stabilization/src/MeshFlow.py", line 21, in point_transform
a = H[0,0]*pt[0] + H[0,1]*pt[1] + H[0,2]
TypeError: 'NoneType' object is not subscriptable
it seems that openCV doesn't always manage to find a homography matrix, due to a very small number of features found. this, of course, can be fixed by changing the 'feature_params', but I was wondering if anyone encountered this with the original code.
I run with Python 3.

thanks, Offir

list index out of range

hi there, when I am trying to run the code after installing all the prerequisites, it shows me this error. please help me out.!!

/home/darkratio/.virtualenvs/cv/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
/home/darkratio/.virtualenvs/cv/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
Traceback (most recent call last):
File "Stabilization.py", line 248, in
file_name = sys.argv[1]

Please Help!

Is there any version written in PyTorch?

When I am trying to run the Mesh flow stabilization for real time video stabilization I am getting the below error

Is the parameter lamada_t in your implementation different from the one in the paper?

I find that in your implementation parameter lamada_t is fixed.But in the papaer MeshFlow: Minimum Latency Online Video Stabilization, lamada_t is an adaptive parameter. Why?

stable_video_change

Thanks for your sharing,why are the characters in the stable video obtained larger than in the original video?

Real-time stabilization

Can this code be implemented to work on real-time incoming frames? Thanks.

When I run this program and get the error in stabilization.py

file_name =(sys.argv[1])
IndexError: list index out of range
How to solve this Index error .Sir

Originally posted by @LeoSeinThanKyaw in 1cb9994#commitcomment-30972156

Does the code use a offline mode other than online mode ?

Camera path

Thank you for your code.
In your README you demonstrate plot of path's original video and stabilized. How do you get this paths?
I thought that it's path of frame's center - x. So x' = H*x, where H is matrix homography and (x'[0] / x'[2], x'[1] / x'[2]) is new coordinates (x, y) of center next frame. But I get very strange plot.
Could you explain me my mistake, please?

Does this work support real-time video stabilizaion?

Does this work support real-time video stabilizaion?
Thanks

It seems that this work doesn't support real time video stabilization ?

Code is full of bugs

in the Stabilization.py file:
line 23 - print statement missing parenthesis
line 66 - integer division produces float which is invalid as a key
line 67 - integer division produces float which is invalid as a key
line 265 - print statement missing parenthesis

in the MeshFlow.py file:
line 81 - integer division produces float which is invalid as a key
line 86 - integer division produces float which is invalid as a key

I've also noticed this stabilization method is insanely slow, a 10 minute video is taking hours to stabilize, I was expecting better performance so these results are quite disappointing.

still some bug when frame size width < height

I have modify Stabilization.py and MeshFlow.py according to the issues before and test data/*.avi is OK.

when test my video, the width < height, arised array bounds overflow. here is the error log.
frame_num 0 size (240, 134, 3)
Traceback (most recent call last):
File "src/Stabilization.py", line 306, in
generate_stabilized_video(cap, x_motion_meshes, y_motion_meshes, new_x_motion_meshes, new_y_motion_meshes, write_name)
File "src/Stabilization.py", line 23, in timed
result = method(args, **kwargs)
File "src/Stabilization.py", line 245, in generate_stabilized_video
new_frame = mesh_warp_frame(frame, new_x_motion_mesh, new_y_motion_mesh)
File "/data/video_stabilization/Mesh-Flow-Video-Stabilization/src/MeshFlow.py", line 175, in mesh_warp_frame
map_x[:, j] = map_x[:, PIXELSx_motion_mesh.shape[0]-1]
IndexError: index 239 is out of bounds for axis 1 with size 134

so I modify MeshFlow.py

# # repeat motion vectors for remaining frame in y-direction
# for i in range(PIXELS*x_motion_mesh.shape[0], map_x.shape[0]):
#         map_x[i, :] = map_x[PIXELS*x_motion_mesh.shape[0]-1, :]
#         map_y[i, :] = map_y[PIXELS*x_motion_mesh.shape[0]-1, :]

# # repeat motion vectors for remaining frame in x-direction
# for j in range(PIXELS*x_motion_mesh.shape[1], map_x.shape[1]):
#         map_x[:, j] = map_x[:, PIXELS*x_motion_mesh.shape[0]-1]
#         map_y[:, j] = map_y[:, PIXELS*x_motion_mesh.shape[0]-1]

# video weight < height
# repeat motion vectors for remaining frame in y-direction
for i in range(PIXELS*x_motion_mesh.shape[0], map_x.shape[0]):
        map_x[:, i] = map_x[:, PIXELS*x_motion_mesh.shape[0]-1]
        map_y[:, i] = map_y[:, PIXELS*x_motion_mesh.shape[0]-1]

# repeat motion vectors for remaining frame in x-direction
for j in range(PIXELS*x_motion_mesh.shape[1], map_x.shape[1]):
        map_x[j, :] = map_x[PIXELS*x_motion_mesh.shape[0]-1, :]
        map_y[j, :] = map_y[PIXELS*x_motion_mesh.shape[0]-1, :]

the video output is drawing, but the stab video has a white edge in the middle of the height and in the right of the width, and I have not figure out.