Synchronize audio and video with FFmpeg

The more I work with FFmpeg, the more I discover its capabilities and realize how powerful it is. So far I covered many functionalities of FFmpeg besides the conventional format conversions. With FFmpeg you can do screen casting, podcast recording, cutting files and even mixing audio and video files. Yet, there is much more to FFmpeg. In this article, I explain how to synchronize audio and video with FFmpeg.

To overcome the synchronization problem, first, we have to know the nature of it. Essentially all the audio and video out of sync issues boil down to either,

• The audio is ahead of the video (or the video behind the audio)
• The audio is behind the video (or the video ahead of the audio)

A slightly confusing part is both problems have a very similar solution that only differs in nuisances. That’s why I separate the solutions as follows.

Fixing audio ahead of the video

In this case, we have to delay the playing of the audio.

For that we run ffmpeg like this,

$ ffmpeg -i file.mkv -itsoffset 3 -i file.mkv -c:a copy -c:v copy -map 0:v:0 -map 1:a:0 out.mkv

Don’t panic because of the lengthy command. It gets easy once you learn how each switch works.

• -i – stands for input. It’s used to pass an input file to FFmpeg. In our case, we have two files audio.mp3 and file.mkv, so we use two -i.
• -itsoffset – this is an important parameter in this command because it instructs FFmpeg to add a delay of three seconds to the input streams. 3 literally means start the second input after three seconds of when the first input started.
• -c:a copy – tells FFmpeg for audio codec just copy it. If we want to can specify other codes like -c:a libmp3lame.
• -c:v copy – is the same as the audio codec, except it is for video.
• -map 0:v:0 – instructs FFmpeg to copy/map the video of the first input to the output file.
• -map 1:a:0 – same as above, except it copies the audio of the second input to the output file.

What happens is the second input which we take its audio starts with three seconds delay.

Here the assumption is the audio is ahead of the video for three seconds. If you don’t know the exact delay, unfortunately, there’s no easy way rather than trial and error. With a couple of attempts, you should get a good result. The good news is, you can be as granular as milliseconds even.

Fixing audio behind the video

For this fix, we have to start playing audio earlier. In other words, we have to start the video with a delay.

For that, we can use the above command and just tweak a little bit,

$ ffmpeg -i file.mkv -itsoffset 3 -i file.mkv -c:a copy -c:v copy -map 0:a:0 -map 1:v:0 out.mkv

Note that the only differences are the -map parts. We instructed FFmpeg to copy the video stream of the second input (the one that starts with three seconds delay) to the output file and copy the audio from input one (the one that plays with no delay).

That essentially means play the audio first and then play the video with three seconds delay.

Pass timestamp to -itsoffset

For the simplicity sake, in all the given examples, we pass 3 to -itsoffset. But the parameter accepts timestamp as granula as milliseconds. For example,

$ ffmpeg -i file.mkv -itsoffset 00:00:00.300 -i file.mkv -c:a copy -c:v copy -map 0:a:0 -map 1:v:0 out.mkv

In the example, we added three hundred milliseconds delay to the video.

Source

• https://superuser.com/questions/982342/in-ffmpeg-how-to-delay-only-the-audio-of-a-mp4-video-without-converting-the-au

Inline/featured images credits

• FFmpeg logo on Wikimedia
• Audio/video graphs by W. Jason Woodrow