[Bug]: frame desynchronization

[nordost8]

[Bug]: frame desynchronization

[nordost8]

from vidgear.gears import WriteGear
import cv2
import time

PATH_TO_SAVE_MP4 = "result.mp4"

target_frame_rate = 24  # FPS
target_duration = 30 # 30 seconds

# define your parameters
output_params = {
    "-acodec": "aac",
    "-ar": 44100,
    "-b:a": "128k",
    "-vcodec": "libx264",
    "-pix_fmt": "yuv420p",
    "-preset": "ultrafast",
    "-r": target_frame_rate,
    "-g": target_frame_rate * 4,
    "-framerate": target_frame_rate,
    "-t": target_duration,
}

writer = WriteGear(output=PATH_TO_SAVE_MP4, **output_params)

target_frames = target_frame_rate * target_duration

# get paths fo all frames:
image_paths = [f'./frames/s_{nummer:05}.jpg' for nummer in range(1, 721)] # from 1 to 720 frame

for image_source in image_paths:
    frame = cv2.imread(image_source, cv2.IMREAD_UNCHANGED)
    try:
        writer.write(frame)
    except OSError as e:
        print(e)
        break
    except Exception as e:
        print(e)
    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break


writer.close()
time.sleep(2)

# add audio to video (I didn’t find any other way in documentation to combine audio and video with VideGear :( ):
ffmpeg_command = [
    "-y",
    "-i",
    "result.mp4",
    "-i",
    "example_audio.mp3",
    "-c:v",
    "copy",
    "-c:a",
    "copy",
    "-map",
    "0:v:0",
    "-map",
    "1:a:0",
    "-t",
    str(target_duration),
    "-shortest",
    "result_with_audio.mp4",
]

writer.execute_ffmpeg_cmd(ffmpeg_command)

[nordost8]

video: https://youtu.be/fbqZMc8VtiI (you can see and hear the desync from about 8 seconds). Full project .zip: https://drive.google.com/file/d/1M8tPckgFph9cv2d30SQYZxlZwvePVeSk/view?usp=drive_link

If you use the native command "ffmpeg -r 24 -i s_%05d.jpg -c:v libx264 -preset ultrafast -pix_fmt yuv420p result.mp4" instead of the part code with "writer" (the code with the addition of audio remains unchanged), then there is no problem.

p.s. At 10 seconds you can see about 252 frames, instead of 240 (I expected to see exactly 240). You can check this more clearly with Sony Vegas or another program that allows you to watch video frame by frame

[abhiTronix]

@rankovaZoria Kindly provide code and video(and corresponding audio) files, so it can be tested.

[abhiTronix]

I need exact code you used to extract frames and then combine them, and all parameters(such as framerate) as such to replicate the behavior.

[abhiTronix]

define your parameters

output_params = {
"-acodec": "aac",
"-ar": 44100,
"-b:a": "128k",
"-vcodec": "libx264",
"-pix_fmt": "yuv420p",
"-preset": "ultrafast",
"-r": target_frame_rate,
"-g": target_frame_rate * 4,
"-framerate": target_frame_rate,
"-t": target_duration,
}

@rankovaZoria You're using custom parameters and expecting it to behave like "ffmpeg -r 24 -i s_%05d.jpg -c:v libx264 -preset ultrafast -pix_fmt yuv420p result.mp4". Explain how?

Secondly, where's -input_framerate parameter as mentioned in docs: https://abhitronix.github.io/vidgear/latest/gears/writegear/compression/usage/#using-compression-mode-with-controlled-framerate, You need exactly same input framerate defined as it was used to separate frames into images.

add audio to video (I didn’t find any other way in documentation to combine audio and video with VideGear :( ):

ffmpeg_command = [
"-y",
"-i",
"result.mp4",
"-i",
"example_audio.mp3",
"-c:v",
"copy",
"-c:a",
"copy",
"-map",
"0:v:0",
"-map",
"1:a:0",
"-t",
str(target_duration),
"-shortest",
"result_with_audio.mp4",
]

writer.execute_ffmpeg_cmd(ffmpeg_command)

If you use the native command "ffmpeg -r 24 -i s_%05d.jpg -c:v libx264 -preset ultrafast -pix_fmt yuv420p result.mp4" instead of the part code with "writer" (the code with the addition of audio remains unchanged), then there is no problem.

@rankovaZoria Are you serious? You can use search function in docs and literally there are multiple examples to merge audio and video:

• https://abhitronix.github.io/vidgear/latest/gears/writegear/compression/usage/#using-compression-mode-with-live-audio-input
• https://abhitronix.github.io/vidgear/latest/gears/writegear/compression/advanced/cciw/#using-writegear-to-merge-audio-with-video
• https://abhitronix.github.io/vidgear/latest/help/writegear_ex/#using-writegears-compression-mode-to-add-external-audio-file-input-to-video-frames

p.s. At 10 seconds you can see about 252 frames, instead of 240 (I expected to see exactly 240). You can check this more clearly with Sony Vegas or another program that allows you to watch video frame by frame

That's because you're using custom parameters without knowing how they works in the first place, GOP value and framerate should match, also, input framerate(-r) must be defined using -input_framerate parameter. And still if you're confused, there's a complete doc: https://abhitronix.github.io/vidgear/latest/gears/writegear/compression/advanced/cciw/#usage-examples and https://abhitronix.github.io/vidgear/latest/gears/writegear/compression/advanced/cciw/#using-writegear-to-merge-audio-with-video and https://abhitronix.github.io/vidgear/latest/help/writegear_ex/#using-writegears-compression-mode-to-add-external-audio-file-input-to-video-frames

[abhiTronix]

@rankovaZoria, also, next time, kindly read the documents properly before raising an issue here.