Jul 08

Comparison of lossless video compression options

Category: Linux,multimedia   — Published by goeszen on July 8, 2015 at 11:47 am

Here it is, an unscientific comparison of lossless video encoding / compression options on Linux with avconv:

Input material was 20 seconds of digitized, interlaced PAL VHS video encoded as HuffYUV video.

Regarding the table of files below,

filename file size rough speed
output_huffyuv.mkv  (-vcodec copy) 200,7MB ~fps=79
output_huffyuv.mkv.7z 152,8MB na9
output_huffyuv.mkv.rar 162,5MB na9
output_huffyuv.mkv.zip 162,6MB na9
output_huffyuv.mkv.gz 163,9MB na9
output_ffvhuff_interlaced_masked.mkv 199,7MB ~fps=52
output_huffyuv_interlaced_masked.mkv 199,7MB ~fps=57
output_ffv1.mkv 126,3MB na9
output_ffv1_interlaced.mkv 126,3MB na9
output_ffv1_interlaced_masked.mkv 124,2MB ~fps=21
output_ffv1_interlaced_masked.mkv.7z 123,3MB na9
output_ultrafastx264.mkv 148,3MB na9
output_ultrafastx264_interlaced.mkv 148,0MB na9
output_ultrafastx264_interlaced_masked.mkv 145,1MB ~fps=50
output_veryslowx264.mkv 125,4MB na9
output_veryslowx264_interlaced.mkv 124,8MB  ~fps=1.0
output_veryslowx264_interlaced_masked.mkv 122,2MB ~fps=1.0
output_zlib_interlaced_masked.mkv 395,1MB ~fps=7.3

Results:

  • ffv1, which is a favorite with long time preservation people and archives, is the winner. It is second in resulting size (2nd smallest file)
    while being reasonably fast.
  • properly handling interlaced input material reduces file size, although by less than 1%.
  • x264 lossless compression is an interesting but CPU intensive (when resulting file size should be comparable) alternative to other lossless codecs
  • "-vcodec zlib" actually managed to produce a file ~100MB bigger than the HuffYUV encoded one
  • if you're into archiving, compressing the resulting file might be an option to bring down size yet again

for comparison, some lossy encodes:

$ avconv -i VHS_raw.avi -vcodec libx264 -q:v 3 -acodec aac -b:a 140k -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" -strict -2 output_x264+aac_interlaced_masked.mp4

output_x264+aac_interlaced_masked.mp4        9,5MB        ~fps=7.9

    frame=  501 fps=7.9 q=-1.0 Lsize=    9297kB time=00:00:20.01 bitrate=3805.9kbits/s dup=1 drop=0
video:8941kB audio:343kB subtitle:0 global headers:0kB muxing overhead 0.138680%
[libx264 @ 0x13b6d60] frame I:8     Avg QP:24.11  size: 35167
[libx264 @ 0x13b6d60] frame P:488   Avg QP:26.47  size: 18071
[libx264 @ 0x13b6d60] frame B:5     Avg QP:26.67  size: 10947
[libx264 @ 0x13b6d60] consecutive B-frames: 98.0%  2.0%  0.0%  0.0%
[libx264 @ 0x13b6d60] mb I  I16..4: 18.5% 61.7% 19.8%
[libx264 @ 0x13b6d60] mb P  I16..4: 16.8% 13.1%  3.7%  P16..4: 40.6% 17.3%  7.9%  0.0%  0.0%    skip: 0.6%
[libx264 @ 0x13b6d60] mb B  I16..4:  1.0%  1.0%  0.5%  B16..8: 30.7%  3.4%  0.4%  direct:50.9%  skip:12.0%  L0:46.1% L1:40.1% BI:13.8%
[libx264 @ 0x13b6d60] field mbs: intra: 95.5% inter:93.5% skip:78.7%
[libx264 @ 0x13b6d60] 8x8 transform intra:40.0% inter:81.3%
[libx264 @ 0x13b6d60] coded y,uvDC,uvAC intra: 45.2% 99.8% 96.8% inter: 31.7% 97.2% 62.7%
[libx264 @ 0x13b6d60] i16 v,h,dc,p:  5% 43%  4% 48%
[libx264 @ 0x13b6d60] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  6% 31% 25%  6%  5%  3%  9%  4% 11%
[libx264 @ 0x13b6d60] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 52% 17%  3%  4%  2%  6%  1%  8%
[libx264 @ 0x13b6d60] i8c dc,h,v,p: 46% 46%  3%  5%
[libx264 @ 0x13b6d60] ref P L0: 40.0% 17.2% 19.7%  7.7% 10.1%  5.2%
[libx264 @ 0x13b6d60] ref B L0: 48.5% 15.0% 28.5%  8.0%
[libx264 @ 0x13b6d60] ref B L1: 73.5% 26.5%

$ avconv -i VHS_raw.avi -vcodec libx264 -preset medium -acodec aac -b:a 140k -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" -strict -2 output_x264+aac+presetmedium_interlaced_masked.mp4

output_x264+aac+presetmedium_interlaced_masked.mp4        9,5MB        ~fps=8.2

    frame=  501 fps=8.2 q=-1.0 Lsize=    9297kB time=00:00:20.01 bitrate=3805.9kbits/s dup=1 drop=0
video:8941kB audio:343kB subtitle:0 global headers:0kB muxing overhead 0.138680%
[libx264 @ 0x25ead80] frame I:8     Avg QP:24.11  size: 35167
[libx264 @ 0x25ead80] frame P:488   Avg QP:26.47  size: 18071
[libx264 @ 0x25ead80] frame B:5     Avg QP:26.67  size: 10947
[libx264 @ 0x25ead80] consecutive B-frames: 98.0%  2.0%  0.0%  0.0%
[libx264 @ 0x25ead80] mb I  I16..4: 18.5% 61.7% 19.8%
[libx264 @ 0x25ead80] mb P  I16..4: 16.8% 13.1%  3.7%  P16..4: 40.6% 17.3%  7.9%  0.0%  0.0%    skip: 0.6%
[libx264 @ 0x25ead80] mb B  I16..4:  1.0%  1.0%  0.5%  B16..8: 30.7%  3.4%  0.4%  direct:50.9%  skip:12.0%  L0:46.1% L1:40.1% BI:13.8%
[libx264 @ 0x25ead80] field mbs: intra: 95.5% inter:93.5% skip:78.7%
[libx264 @ 0x25ead80] 8x8 transform intra:40.0% inter:81.3%
[libx264 @ 0x25ead80] coded y,uvDC,uvAC intra: 45.2% 99.8% 96.8% inter: 31.7% 97.2% 62.7%
[libx264 @ 0x25ead80] i16 v,h,dc,p:  5% 43%  4% 48%
[libx264 @ 0x25ead80] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  6% 31% 25%  6%  5%  3%  9%  4% 11%
[libx264 @ 0x25ead80] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 52% 17%  3%  4%  2%  6%  1%  8%
[libx264 @ 0x25ead80] i8c dc,h,v,p: 46% 46%  3%  5%
[libx264 @ 0x25ead80] ref P L0: 40.0% 17.2% 19.7%  7.7% 10.1%  5.2%
[libx264 @ 0x25ead80] ref B L0: 48.5% 15.0% 28.5%  8.0%
[libx264 @ 0x25ead80] ref B L1: 73.5% 26.5%
[libx264 @ 0x25ead80] kb/s:3656.34

$ avconv -i VHS_raw.avi -vcodec libx264 -preset faster -acodec aac -b:a 140k -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" -strict -2 output_x264+aac+presetmedium_interlaced_masked.mp4

output_x264+aac+presetfaster_interlaced_masked.mp4        8,3MB        ~fps=22

    frame=  501 fps= 22 q=-1.0 Lsize=    8147kB time=00:00:20.01 bitrate=3335.1kbits/s dup=1 drop=0
video:7791kB audio:343kB subtitle:0 global headers:0kB muxing overhead 0.158292%
[libx264 @ 0x1825920] frame I:8     Avg QP:24.11  size: 30787
[libx264 @ 0x1825920] frame P:488   Avg QP:26.45  size: 15739
[libx264 @ 0x1825920] frame B:5     Avg QP:26.82  size: 10032
[libx264 @ 0x1825920] consecutive B-frames: 98.0%  2.0%  0.0%  0.0%
[libx264 @ 0x1825920] mb I  I16..4: 23.0% 57.8% 19.2%
[libx264 @ 0x1825920] mb P  I16..4: 15.0% 19.5%  2.2%  P16..4: 55.7%  5.5%  0.8%  0.0%  0.0%    skip: 1.2%
[libx264 @ 0x1825920] mb B  I16..4:  2.9%  3.8%  0.4%  B16..8: 38.1%  3.9%  0.2%  direct:45.9%  skip: 4.7%  L0:53.7% L1:41.2% BI: 5.1%
[libx264 @ 0x1825920] field mbs: intra: 94.0% inter:94.6% skip:72.7%
[libx264 @ 0x1825920] 8x8 transform intra:53.3% inter:51.2%
[libx264 @ 0x1825920] coded y,uvDC,uvAC intra: 47.2% 99.5% 89.3% inter: 21.9% 97.0% 64.5%
[libx264 @ 0x1825920] i16 v,h,dc,p: 20% 44% 15% 21%
[libx264 @ 0x1825920] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 38% 32%  4%  3%  2%  5%  2%  7%
[libx264 @ 0x1825920] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 50% 13%  4%  4%  2%  7%  2% 11%
[libx264 @ 0x1825920] i8c dc,h,v,p: 48% 40%  5%  7%
[libx264 @ 0x1825920] ref P L0: 38.5% 30.6% 17.5% 13.4%
[libx264 @ 0x1825920] ref B L0: 37.2% 23.4% 25.3% 14.2%
[libx264 @ 0x1825920] ref B L1: 55.3% 44.7%
[libx264 @ 0x1825920] kb/s:3185.91

$ avconv -i F_VHSC_1992_01_05_raw_edit.avi -vcodec libx264 -preset faster -q:v 3 -acodec aac -b:a 140k -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" -strict -2 output_x264+aac+presetfaster+q3_interlaced_masked.mp4

output_x264+aac+presetfaster+q3_interlaced_masked.mp4        8,3MB        ~fps=22

    frame=  501 fps= 22 q=-1.0 Lsize=    8147kB time=00:00:20.01 bitrate=3335.1kbits/s dup=1 drop=0
video:7791kB audio:343kB subtitle:0 global headers:0kB muxing overhead 0.158292%
[libx264 @ 0x1a67bc0] frame I:8     Avg QP:24.11  size: 30787
[libx264 @ 0x1a67bc0] frame P:488   Avg QP:26.45  size: 15739
[libx264 @ 0x1a67bc0] frame B:5     Avg QP:26.82  size: 10032
[libx264 @ 0x1a67bc0] consecutive B-frames: 98.0%  2.0%  0.0%  0.0%
[libx264 @ 0x1a67bc0] mb I  I16..4: 23.0% 57.8% 19.2%
[libx264 @ 0x1a67bc0] mb P  I16..4: 15.0% 19.5%  2.2%  P16..4: 55.7%  5.5%  0.8%  0.0%  0.0%    skip: 1.2%
[libx264 @ 0x1a67bc0] mb B  I16..4:  2.9%  3.8%  0.4%  B16..8: 38.1%  3.9%  0.2%  direct:45.9%  skip: 4.7%  L0:53.7% L1:41.2% BI: 5.1%
[libx264 @ 0x1a67bc0] field mbs: intra: 94.0% inter:94.6% skip:72.7%
[libx264 @ 0x1a67bc0] 8x8 transform intra:53.3% inter:51.2%
[libx264 @ 0x1a67bc0] coded y,uvDC,uvAC intra: 47.2% 99.5% 89.3% inter: 21.9% 97.0% 64.5%
[libx264 @ 0x1a67bc0] i16 v,h,dc,p: 20% 44% 15% 21%
[libx264 @ 0x1a67bc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 38% 32%  4%  3%  2%  5%  2%  7%
[libx264 @ 0x1a67bc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 50% 13%  4%  4%  2%  7%  2% 11%
[libx264 @ 0x1a67bc0] i8c dc,h,v,p: 48% 40%  5%  7%
[libx264 @ 0x1a67bc0] ref P L0: 38.5% 30.6% 17.5% 13.4%
[libx264 @ 0x1a67bc0] ref B L0: 37.2% 23.4% 25.3% 14.2%
[libx264 @ 0x1a67bc0] ref B L1: 55.3% 44.7%
[libx264 @ 0x1a67bc0] kb/s:3185.91

avconv versions:

    $ avconv -i VHS_raw.avi -c:v libx264 -preset veryslow -qp 0 -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" output_veryslowx264_interlaced_masked.mkv
ffmpeg version 1.2.6-7:1.2.6-1~trusty1 Copyright (c) 2000-2014 the FFmpeg developers
built on Apr 26 2014 18:52:58 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
configuration: --arch=amd64 --disable-stripping --enable-avresample --enable-pthreads --enable-runtime-cpudetect --extra-version='7:1.2.6-1~trusty1' --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --enable-bzlib --enable-libdc1394 --enable-libfreetype --enable-frei0r --enable-gnutls --enable-libgsm --enable-libmp3lame --enable-librtmp --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-vaapi --enable-vdpau --enable-libvorbis --enable-libvpx --enable-zlib --enable-gpl --enable-postproc --enable-libcdio --enable-x11grab --enable-libx264 --shlibdir=/usr/lib/x86_64-linux-gnu --enable-shared --disable-static
libavutil      52. 18.100 / 52. 18.100
libavcodec     54. 92.100 / 54. 92.100
libavformat    54. 63.104 / 54. 63.104
libavdevice    53.  5.103 / 53.  5.103
libavfilter     3. 42.103 /  3. 42.103
libswscale      2.  2.100 /  2.  2.100
libswresample   0. 17.102 /  0. 17.102
libpostproc    52.  2.100 / 52.  2.100
[avi @ 0x91d140] non-interleaved AVI
Guessed Channel Layout for  Input Stream #0.1 : stereo
Input #0, avi, from 'VHS_raw.avi':
Metadata:
encoder         : Lavf54.63.104
Duration: 00:31:23.02, start: 0.000000, bitrate: 80040 kb/s
Stream #0:0: Video: huffyuv (HFYU / 0x55594648), yuv422p, 720x576, 25.01 tbr, 25.01 tbn, 25.01 tbc
Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
[libx264 @ 0x92cb00] interlace + weightp is not implemented
[libx264 @ 0x92cb00] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
[libx264 @ 0x92cb00] profile High 4:4:4 Predictive, level 4.0, 4:2:2 8-bit
[libx264 @ 0x92cb00] 264 - core 142 r2389 956c8d8 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=16 deblock=1:0:0 analyse=0x3:0x133 me=umh subme=9 psy=0 mixed_ref=1 me_range=24 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=tff bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=cqp mbtree=0 qp=0
Output #0, matroska, to 'output_veryslowx264_interlaced_masked.mkv':
Metadata:
encoder         : Lavf54.63.104
Stream #0:0: Video: h264 (H264 / 0x34363248), yuv422p, 720x576, q=-1--1, 1k tbn, 25.01 tbc
Stream #0:1: Audio: vorbis (oV[0][0] / 0x566F), 48000 Hz, stereo, fltp
Stream mapping:
Stream #0:0 -> #0:0 (huffyuv -> libx264)
Stream #0:1 -> #0:1 (pcm_s16le -> libvorbis)

Helpful links:

Other lossless video codecs which I haven't tested

https://en.wikipedia.org/wiki/Lagarith (avconv is only able to decode it, currently)
https://en.wikipedia.org/wiki/JPEG_2000#Motion_JPEG_2000
https://de.wikipedia.org/wiki/LCL-Codec  (?, same as "-vcodec zlib" ?)
https://de.wikipedia.org/wiki/CorePNG (said to be inferior / slower / bigger files to HuffYUV)

$ avconv -codecs | grep "S " | grep "DEV"
ffmpeg version 1.2.6-7:1.2.6-1~trusty1 Copyright (c) 2000-2014 the FFmpeg developers
built on Apr 26 2014 18:52:58 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
configuration: --arch=amd64 --disable-stripping --enable-avresample --enable-pthreads --enable-runtime-cpudetect --extra-version='7:1.2.6-1~trusty1' --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --enable-bzlib --enable-libdc1394 --enable-libfreetype --enable-frei0r --enable-gnutls --enable-libgsm --enable-libmp3lame --enable-librtmp --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-vaapi --enable-vdpau --enable-libvorbis --enable-libvpx --enable-zlib --enable-gpl --enable-postproc --enable-libcdio --enable-x11grab --enable-libx264 --shlibdir=/usr/lib/x86_64-linux-gnu --enable-shared --disable-static
libavutil      52. 18.100 / 52. 18.100
libavcodec     54. 92.100 / 54. 92.100
libavformat    54. 63.104 / 54. 63.104
libavdevice    53.  5.103 / 53.  5.103
libavfilter     3. 42.103 /  3. 42.103
libswscale      2.  2.100 /  2.  2.100
libswresample   0. 17.102 /  0. 17.102
libpostproc    52.  2.100 / 52.  2.100
DEVIL. asv1                 ASUS V1
DEVIL. asv2                 ASUS V2
DEVI.. ayuv                 Uncompressed packed MS 4:4:4:4
DEVI.S bmp                  BMP (Windows and OS/2 bitmap)
DEV.LS dirac                Dirac (decoders: dirac libschroedinger ) (encoders: libschroedinger )
DEVI.S dpx                  DPX image
DEV..S ffv1                 FFmpeg video codec #1
DEVI.S ffvhuff              Huffyuv FFmpeg variant
DEV..S flashsv              Flash Screen Video v1
DEV..S gif                  GIF (Graphics Interchange Format)
DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_vdpau ) (encoders: libx264 libx264rgb )
DEVI.S huffyuv              HuffYUV
DEVILS jpeg2000             JPEG 2000 (decoders: j2k libopenjpeg ) (encoders: j2k libopenjpeg )
DEVILS jpegls               JPEG-LS
DEVI.S pam                  PAM (Portable AnyMap) image
DEVI.S pbm                  PBM (Portable BitMap) image
DEVI.S pcx                  PC Paintbrush PCX image
DEVI.S pgm                  PGM (Portable GrayMap) image
DEVI.S pgmyuv               PGMYUV (Portable GrayMap YUV) image
DEV..S png                  PNG (Portable Network Graphics) image
DEVI.S ppm                  PPM (Portable PixelMap) image
DEV..S qtrle                QuickTime Animation (RLE) video
DEVI.S r10k                 AJA Kona 10-bit RGB Codec
DEVI.S r210                 Uncompressed RGB 10-bit
DEVI.S rawvideo             raw video
DEVI.S sgi                  SGI image
DEV.LS snow                 Snow
DEVI.S sunrast              Sun Rasterfile image
DEVI.S targa                Truevision Targa image
DEVI.S tiff                 TIFF image
DEVI.S utvideo              Ut Video
DEVI.S v210                 Uncompressed 4:2:2 10-bit
DEVI.S v410                 Uncompressed 4:4:4 10-bit
DEVI.S xbm                  XBM (X BitMap) image
DEVI.S xwd                  XWD (X Window Dump) image
DEVI.S zlib                 LCL (LossLess Codec Library) ZLIB
DEV..S zmbv                 Zip Motion Blocks Video

Leave a Reply

=