Apr 24

Converting ANY video with ffmpeg (letterboxing/pillarboxing)

Category: Linux,multimedia  tengo @ 5:17 am

When you use ffmpeg to transcode videos from various sources and in various sizes, formats and aspect ratios to a given destination format, you can’t rely on ffmpeg alone to produce the expected results. In this post we will have a look at how we can dynamically letterbox or pillarbox (black bars on the sides of the video, instead of top/bottom) any input video so that we can fit the video source into a fixed dimension target frame without breaking aspect ratio by squeezing the image.

my $input_width;
my $input_height;

# probe file for aspect
open(FFMPEG, “ffmpeg -i ‘$inFile’ 2>&1 |”) or $error .= “Couldn’t read from ffmpeg: $!\n”;
while (<FFMPEG>) {
if($_ =~ /Video:.+ (\d+)x(\d+)/){
$input_width = $1;
$input_height= $2;
}
}
if(!$input_width && !$input_height){
die “Could not fully get input video dimensions: $input_width x $input_height\n”;
}

Now, let’s define the target video dimensions:

my $output_width = 480;
my $output_height= 320;

# we need this for the ffmpeg command… ugly but works
my $padone = ‘-padtop’;
my $padtwo = ‘-padbottom’;

Then, here comes the magic that calculates the thickness of the black bars. This assumes that the video is actually full screen or less/letterboxed. If the video is less wide than the target frame size, the below routine will give a negative letterboxing result (which we will correct later on and use the value for pillarboxing):

# calculate padding (for black bar letterboxing/pillarboxing)
my $input_aspect= $input_width / $input_height;
my $conv_height = int( ($output_width / $input_aspect) );
$conv_height -= 1 if $conv_height % 2 == 1;
my $conv_pad = int( (($output_height – $conv_height) / 2.0) );
$conv_pad -= 1 if $conv_pad % 2 == 1;

Now we know the thickness of the black bars, but we must have a look at the video’s dimensions to actually know if we need to letterbox of pillarbox. Note the use of the value 1.333. This is the aspect ratio of your used target frame size. We use 480×320 which results in an aspect of 1.333 (480/320) and is the basis for a “breakingpoint” in our calculation:

my $aspect_mode;
if($input_aspect < 1.333){
$aspect_mode = ‘pillarboxing’;
$conv_pad *= -1; # negative to positive values
$padone = ‘-padleft’; # padding on sides
$padtwo = ‘-padright’;
}else{# default
$aspect_mode = ‘letterboxing’;
}

One last hack for the ffmpeg command:

# need for our command..
my $wxh = $output_width .’x’. $conv_height;

That’s it! Now let’s send these results to our ffmpeg command to start transcoding:

open(FFMPEG, “ffmpeg -y -i ‘$inFile’ -f flv -s $wxh $padone $conv_pad $padtwo $conv_pad -ar …”);

(the “-s” parameter informs ffmpeg about the target size, $padone and $padtwo change, depending on letterboxing or pillarboxing, to “-padtop/-padbottom” and “-padleft/-padright” while $conv_pad holds the symmetrical value for the black bars thickness.)

Further: An interesting algorithm to detect aspect ratio can be found in this thread:

my $aspect = $w/$h;
my $size;
if (abs($aspect – 16/9) < 0.02) {
$size = “320×180″;
} elsif (abs($aspect – 4/3) < 0.02) {
$size = “320×240″;
} else {
die “Weird aspect ratio: ${w}x${h} = $aspect\n”;
}

One last but important note:
When feeding the letterboxing flags to ffmpeg via the command line, padtop, padbottom etc. need to go after the output file name. Otherwise they would be interpreted as information to interpret the input file! So for example do: ffmpeg -i <input 16:9 file> -s 320×240 <output video> -padbottom 12 -padtop 12

One Response to “Converting ANY video with ffmpeg (letterboxing/pillarboxing)”

  1. Ffmpeg incorrect top pad size | Zen of Linux says:

    [...] discovered this while developing the code to transcode any video with letterboxing/pillarboxing with ffmpeg.) Related PostsSome package notesChaining commandsSimple video transcoding with [...]