Jul 08
Comparison of lossless video compression options
Here it is, an unscientific comparison of lossless video encoding / compression options on Linux with avconv:
Input material was 20 seconds of digitized, interlaced PAL VHS video encoded as HuffYUV video.
Regarding the table of files below,
- "_interlaced" means avconv was passed the interlaced flag: -flags +ilme+ildct
- and "_masked" means the "noisy" VHS overscan area has been masked with -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1"
- Machine was a Intel® Core™2 CPU 4300 @ 1.80GHz × 2
- Reading and writing happened on an SSD drive.
| filename | file size | rough speed |
| output_huffyuv.mkv (-vcodec copy) | 200,7MB | ~fps=79 |
| output_huffyuv.mkv.7z | 152,8MB | na9 |
| output_huffyuv.mkv.rar | 162,5MB | na9 |
| output_huffyuv.mkv.zip | 162,6MB | na9 |
| output_huffyuv.mkv.gz | 163,9MB | na9 |
| output_ffvhuff_interlaced_masked.mkv | 199,7MB | ~fps=52 |
| output_huffyuv_interlaced_masked.mkv | 199,7MB | ~fps=57 |
| output_ffv1.mkv | 126,3MB | na9 |
| output_ffv1_interlaced.mkv | 126,3MB | na9 |
| output_ffv1_interlaced_masked.mkv | 124,2MB | ~fps=21 |
| output_ffv1_interlaced_masked.mkv.7z | 123,3MB | na9 |
| output_ultrafastx264.mkv | 148,3MB | na9 |
| output_ultrafastx264_interlaced.mkv | 148,0MB | na9 |
| output_ultrafastx264_interlaced_masked.mkv | 145,1MB | ~fps=50 |
| output_veryslowx264.mkv | 125,4MB | na9 |
| output_veryslowx264_interlaced.mkv | 124,8MB | ~fps=1.0 |
| output_veryslowx264_interlaced_masked.mkv | 122,2MB | ~fps=1.0 |
| output_zlib_interlaced_masked.mkv | 395,1MB | ~fps=7.3 |
Results:
- ffv1, which is a favorite with long time preservation people and archives, is the winner. It is second in resulting size (2nd smallest file)
while being reasonably fast. - properly handling interlaced input material reduces file size, although by less than 1%.
- x264 lossless compression is an interesting but CPU intensive (when resulting file size should be comparable) alternative to other lossless codecs
- "-vcodec zlib" actually managed to produce a file ~100MB bigger than the HuffYUV encoded one
- if you're into archiving, compressing the resulting file might be an option to bring down size yet again
for comparison, some lossy encodes:
$ avconv -i VHS_raw.avi -vcodec libx264 -q:v 3 -acodec aac -b:a 140k -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" -strict -2 output_x264+aac_interlaced_masked.mp4
output_x264+aac_interlaced_masked.mp4 9,5MB ~fps=7.9
frame= 501 fps=7.9 q=-1.0 Lsize= 9297kB time=00:00:20.01 bitrate=3805.9kbits/s dup=1 drop=0
video:8941kB audio:343kB subtitle:0 global headers:0kB muxing overhead 0.138680%
[libx264 @ 0x13b6d60] frame I:8 Avg QP:24.11 size: 35167
[libx264 @ 0x13b6d60] frame P:488 Avg QP:26.47 size: 18071
[libx264 @ 0x13b6d60] frame B:5 Avg QP:26.67 size: 10947
[libx264 @ 0x13b6d60] consecutive B-frames: 98.0% 2.0% 0.0% 0.0%
[libx264 @ 0x13b6d60] mb I I16..4: 18.5% 61.7% 19.8%
[libx264 @ 0x13b6d60] mb P I16..4: 16.8% 13.1% 3.7% P16..4: 40.6% 17.3% 7.9% 0.0% 0.0% skip: 0.6%
[libx264 @ 0x13b6d60] mb B I16..4: 1.0% 1.0% 0.5% B16..8: 30.7% 3.4% 0.4% direct:50.9% skip:12.0% L0:46.1% L1:40.1% BI:13.8%
[libx264 @ 0x13b6d60] field mbs: intra: 95.5% inter:93.5% skip:78.7%
[libx264 @ 0x13b6d60] 8x8 transform intra:40.0% inter:81.3%
[libx264 @ 0x13b6d60] coded y,uvDC,uvAC intra: 45.2% 99.8% 96.8% inter: 31.7% 97.2% 62.7%
[libx264 @ 0x13b6d60] i16 v,h,dc,p: 5% 43% 4% 48%
[libx264 @ 0x13b6d60] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 6% 31% 25% 6% 5% 3% 9% 4% 11%
[libx264 @ 0x13b6d60] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 52% 17% 3% 4% 2% 6% 1% 8%
[libx264 @ 0x13b6d60] i8c dc,h,v,p: 46% 46% 3% 5%
[libx264 @ 0x13b6d60] ref P L0: 40.0% 17.2% 19.7% 7.7% 10.1% 5.2%
[libx264 @ 0x13b6d60] ref B L0: 48.5% 15.0% 28.5% 8.0%
[libx264 @ 0x13b6d60] ref B L1: 73.5% 26.5%
$ avconv -i VHS_raw.avi -vcodec libx264 -preset medium -acodec aac -b:a 140k -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" -strict -2 output_x264+aac+presetmedium_interlaced_masked.mp4
output_x264+aac+presetmedium_interlaced_masked.mp4 9,5MB ~fps=8.2
frame= 501 fps=8.2 q=-1.0 Lsize= 9297kB time=00:00:20.01 bitrate=3805.9kbits/s dup=1 drop=0
video:8941kB audio:343kB subtitle:0 global headers:0kB muxing overhead 0.138680%
[libx264 @ 0x25ead80] frame I:8 Avg QP:24.11 size: 35167
[libx264 @ 0x25ead80] frame P:488 Avg QP:26.47 size: 18071
[libx264 @ 0x25ead80] frame B:5 Avg QP:26.67 size: 10947
[libx264 @ 0x25ead80] consecutive B-frames: 98.0% 2.0% 0.0% 0.0%
[libx264 @ 0x25ead80] mb I I16..4: 18.5% 61.7% 19.8%
[libx264 @ 0x25ead80] mb P I16..4: 16.8% 13.1% 3.7% P16..4: 40.6% 17.3% 7.9% 0.0% 0.0% skip: 0.6%
[libx264 @ 0x25ead80] mb B I16..4: 1.0% 1.0% 0.5% B16..8: 30.7% 3.4% 0.4% direct:50.9% skip:12.0% L0:46.1% L1:40.1% BI:13.8%
[libx264 @ 0x25ead80] field mbs: intra: 95.5% inter:93.5% skip:78.7%
[libx264 @ 0x25ead80] 8x8 transform intra:40.0% inter:81.3%
[libx264 @ 0x25ead80] coded y,uvDC,uvAC intra: 45.2% 99.8% 96.8% inter: 31.7% 97.2% 62.7%
[libx264 @ 0x25ead80] i16 v,h,dc,p: 5% 43% 4% 48%
[libx264 @ 0x25ead80] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 6% 31% 25% 6% 5% 3% 9% 4% 11%
[libx264 @ 0x25ead80] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 52% 17% 3% 4% 2% 6% 1% 8%
[libx264 @ 0x25ead80] i8c dc,h,v,p: 46% 46% 3% 5%
[libx264 @ 0x25ead80] ref P L0: 40.0% 17.2% 19.7% 7.7% 10.1% 5.2%
[libx264 @ 0x25ead80] ref B L0: 48.5% 15.0% 28.5% 8.0%
[libx264 @ 0x25ead80] ref B L1: 73.5% 26.5%
[libx264 @ 0x25ead80] kb/s:3656.34
$ avconv -i VHS_raw.avi -vcodec libx264 -preset faster -acodec aac -b:a 140k -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" -strict -2 output_x264+aac+presetmedium_interlaced_masked.mp4
output_x264+aac+presetfaster_interlaced_masked.mp4 8,3MB ~fps=22
frame= 501 fps= 22 q=-1.0 Lsize= 8147kB time=00:00:20.01 bitrate=3335.1kbits/s dup=1 drop=0
video:7791kB audio:343kB subtitle:0 global headers:0kB muxing overhead 0.158292%
[libx264 @ 0x1825920] frame I:8 Avg QP:24.11 size: 30787
[libx264 @ 0x1825920] frame P:488 Avg QP:26.45 size: 15739
[libx264 @ 0x1825920] frame B:5 Avg QP:26.82 size: 10032
[libx264 @ 0x1825920] consecutive B-frames: 98.0% 2.0% 0.0% 0.0%
[libx264 @ 0x1825920] mb I I16..4: 23.0% 57.8% 19.2%
[libx264 @ 0x1825920] mb P I16..4: 15.0% 19.5% 2.2% P16..4: 55.7% 5.5% 0.8% 0.0% 0.0% skip: 1.2%
[libx264 @ 0x1825920] mb B I16..4: 2.9% 3.8% 0.4% B16..8: 38.1% 3.9% 0.2% direct:45.9% skip: 4.7% L0:53.7% L1:41.2% BI: 5.1%
[libx264 @ 0x1825920] field mbs: intra: 94.0% inter:94.6% skip:72.7%
[libx264 @ 0x1825920] 8x8 transform intra:53.3% inter:51.2%
[libx264 @ 0x1825920] coded y,uvDC,uvAC intra: 47.2% 99.5% 89.3% inter: 21.9% 97.0% 64.5%
[libx264 @ 0x1825920] i16 v,h,dc,p: 20% 44% 15% 21%
[libx264 @ 0x1825920] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 38% 32% 4% 3% 2% 5% 2% 7%
[libx264 @ 0x1825920] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 50% 13% 4% 4% 2% 7% 2% 11%
[libx264 @ 0x1825920] i8c dc,h,v,p: 48% 40% 5% 7%
[libx264 @ 0x1825920] ref P L0: 38.5% 30.6% 17.5% 13.4%
[libx264 @ 0x1825920] ref B L0: 37.2% 23.4% 25.3% 14.2%
[libx264 @ 0x1825920] ref B L1: 55.3% 44.7%
[libx264 @ 0x1825920] kb/s:3185.91
$ avconv -i F_VHSC_1992_01_05_raw_edit.avi -vcodec libx264 -preset faster -q:v 3 -acodec aac -b:a 140k -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" -strict -2 output_x264+aac+presetfaster+q3_interlaced_masked.mp4
output_x264+aac+presetfaster+q3_interlaced_masked.mp4 8,3MB ~fps=22
frame= 501 fps= 22 q=-1.0 Lsize= 8147kB time=00:00:20.01 bitrate=3335.1kbits/s dup=1 drop=0
video:7791kB audio:343kB subtitle:0 global headers:0kB muxing overhead 0.158292%
[libx264 @ 0x1a67bc0] frame I:8 Avg QP:24.11 size: 30787
[libx264 @ 0x1a67bc0] frame P:488 Avg QP:26.45 size: 15739
[libx264 @ 0x1a67bc0] frame B:5 Avg QP:26.82 size: 10032
[libx264 @ 0x1a67bc0] consecutive B-frames: 98.0% 2.0% 0.0% 0.0%
[libx264 @ 0x1a67bc0] mb I I16..4: 23.0% 57.8% 19.2%
[libx264 @ 0x1a67bc0] mb P I16..4: 15.0% 19.5% 2.2% P16..4: 55.7% 5.5% 0.8% 0.0% 0.0% skip: 1.2%
[libx264 @ 0x1a67bc0] mb B I16..4: 2.9% 3.8% 0.4% B16..8: 38.1% 3.9% 0.2% direct:45.9% skip: 4.7% L0:53.7% L1:41.2% BI: 5.1%
[libx264 @ 0x1a67bc0] field mbs: intra: 94.0% inter:94.6% skip:72.7%
[libx264 @ 0x1a67bc0] 8x8 transform intra:53.3% inter:51.2%
[libx264 @ 0x1a67bc0] coded y,uvDC,uvAC intra: 47.2% 99.5% 89.3% inter: 21.9% 97.0% 64.5%
[libx264 @ 0x1a67bc0] i16 v,h,dc,p: 20% 44% 15% 21%
[libx264 @ 0x1a67bc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 38% 32% 4% 3% 2% 5% 2% 7%
[libx264 @ 0x1a67bc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 7% 50% 13% 4% 4% 2% 7% 2% 11%
[libx264 @ 0x1a67bc0] i8c dc,h,v,p: 48% 40% 5% 7%
[libx264 @ 0x1a67bc0] ref P L0: 38.5% 30.6% 17.5% 13.4%
[libx264 @ 0x1a67bc0] ref B L0: 37.2% 23.4% 25.3% 14.2%
[libx264 @ 0x1a67bc0] ref B L1: 55.3% 44.7%
[libx264 @ 0x1a67bc0] kb/s:3185.91
avconv versions:
$ avconv -i VHS_raw.avi -c:v libx264 -preset veryslow -qp 0 -t 20 -flags +ilme+ildct -vf "drawbox=0:0:5:576:black@1, drawbox=716:0:4:576:black@1, drawbox=0:571:720:5:black@1" output_veryslowx264_interlaced_masked.mkv
ffmpeg version 1.2.6-7:1.2.6-1~trusty1 Copyright (c) 2000-2014 the FFmpeg developers
built on Apr 26 2014 18:52:58 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
configuration: --arch=amd64 --disable-stripping --enable-avresample --enable-pthreads --enable-runtime-cpudetect --extra-version='7:1.2.6-1~trusty1' --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --enable-bzlib --enable-libdc1394 --enable-libfreetype --enable-frei0r --enable-gnutls --enable-libgsm --enable-libmp3lame --enable-librtmp --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-vaapi --enable-vdpau --enable-libvorbis --enable-libvpx --enable-zlib --enable-gpl --enable-postproc --enable-libcdio --enable-x11grab --enable-libx264 --shlibdir=/usr/lib/x86_64-linux-gnu --enable-shared --disable-static
libavutil 52. 18.100 / 52. 18.100
libavcodec 54. 92.100 / 54. 92.100
libavformat 54. 63.104 / 54. 63.104
libavdevice 53. 5.103 / 53. 5.103
libavfilter 3. 42.103 / 3. 42.103
libswscale 2. 2.100 / 2. 2.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 2.100 / 52. 2.100
[avi @ 0x91d140] non-interleaved AVI
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, avi, from 'VHS_raw.avi':
Metadata:
encoder : Lavf54.63.104
Duration: 00:31:23.02, start: 0.000000, bitrate: 80040 kb/s
Stream #0:0: Video: huffyuv (HFYU / 0x55594648), yuv422p, 720x576, 25.01 tbr, 25.01 tbn, 25.01 tbc
Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
[libx264 @ 0x92cb00] interlace + weightp is not implemented
[libx264 @ 0x92cb00] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 SlowShuffle
[libx264 @ 0x92cb00] profile High 4:4:4 Predictive, level 4.0, 4:2:2 8-bit
[libx264 @ 0x92cb00] 264 - core 142 r2389 956c8d8 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=16 deblock=1:0:0 analyse=0x3:0x133 me=umh subme=9 psy=0 mixed_ref=1 me_range=24 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=tff bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=cqp mbtree=0 qp=0
Output #0, matroska, to 'output_veryslowx264_interlaced_masked.mkv':
Metadata:
encoder : Lavf54.63.104
Stream #0:0: Video: h264 (H264 / 0x34363248), yuv422p, 720x576, q=-1--1, 1k tbn, 25.01 tbc
Stream #0:1: Audio: vorbis (oV[0][0] / 0x566F), 48000 Hz, stereo, fltp
Stream mapping:
Stream #0:0 -> #0:0 (huffyuv -> libx264)
Stream #0:1 -> #0:1 (pcm_s16le -> libvorbis)
Helpful links:
Other lossless video codecs which I haven't tested
https://en.wikipedia.org/wiki/Lagarith (avconv is only able to decode it, currently)
https://en.wikipedia.org/wiki/JPEG_2000#Motion_JPEG_2000
https://de.wikipedia.org/wiki/LCL-Codec (?, same as "-vcodec zlib" ?)
https://de.wikipedia.org/wiki/CorePNG (said to be inferior / slower / bigger files to HuffYUV)
$ avconv -codecs | grep "S " | grep "DEV"
ffmpeg version 1.2.6-7:1.2.6-1~trusty1 Copyright (c) 2000-2014 the FFmpeg developers
built on Apr 26 2014 18:52:58 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
configuration: --arch=amd64 --disable-stripping --enable-avresample --enable-pthreads --enable-runtime-cpudetect --extra-version='7:1.2.6-1~trusty1' --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --enable-bzlib --enable-libdc1394 --enable-libfreetype --enable-frei0r --enable-gnutls --enable-libgsm --enable-libmp3lame --enable-librtmp --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-vaapi --enable-vdpau --enable-libvorbis --enable-libvpx --enable-zlib --enable-gpl --enable-postproc --enable-libcdio --enable-x11grab --enable-libx264 --shlibdir=/usr/lib/x86_64-linux-gnu --enable-shared --disable-static
libavutil 52. 18.100 / 52. 18.100
libavcodec 54. 92.100 / 54. 92.100
libavformat 54. 63.104 / 54. 63.104
libavdevice 53. 5.103 / 53. 5.103
libavfilter 3. 42.103 / 3. 42.103
libswscale 2. 2.100 / 2. 2.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 2.100 / 52. 2.100
DEVIL. asv1 ASUS V1
DEVIL. asv2 ASUS V2
DEVI.. ayuv Uncompressed packed MS 4:4:4:4
DEVI.S bmp BMP (Windows and OS/2 bitmap)
DEV.LS dirac Dirac (decoders: dirac libschroedinger ) (encoders: libschroedinger )
DEVI.S dpx DPX image
DEV..S ffv1 FFmpeg video codec #1
DEVI.S ffvhuff Huffyuv FFmpeg variant
DEV..S flashsv Flash Screen Video v1
DEV..S gif GIF (Graphics Interchange Format)
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_vdpau ) (encoders: libx264 libx264rgb )
DEVI.S huffyuv HuffYUV
DEVILS jpeg2000 JPEG 2000 (decoders: j2k libopenjpeg ) (encoders: j2k libopenjpeg )
DEVILS jpegls JPEG-LS
DEVI.S pam PAM (Portable AnyMap) image
DEVI.S pbm PBM (Portable BitMap) image
DEVI.S pcx PC Paintbrush PCX image
DEVI.S pgm PGM (Portable GrayMap) image
DEVI.S pgmyuv PGMYUV (Portable GrayMap YUV) image
DEV..S png PNG (Portable Network Graphics) image
DEVI.S ppm PPM (Portable PixelMap) image
DEV..S qtrle QuickTime Animation (RLE) video
DEVI.S r10k AJA Kona 10-bit RGB Codec
DEVI.S r210 Uncompressed RGB 10-bit
DEVI.S rawvideo raw video
DEVI.S sgi SGI image
DEV.LS snow Snow
DEVI.S sunrast Sun Rasterfile image
DEVI.S targa Truevision Targa image
DEVI.S tiff TIFF image
DEVI.S utvideo Ut Video
DEVI.S v210 Uncompressed 4:2:2 10-bit
DEVI.S v410 Uncompressed 4:4:4 10-bit
DEVI.S xbm XBM (X BitMap) image
DEVI.S xwd XWD (X Window Dump) image
DEVI.S zlib LCL (LossLess Codec Library) ZLIB
DEV..S zmbv Zip Motion Blocks Video