Video decoders

Important: video decoding is a huge topic area of theories, algorithms, standards and formats. I don't even try to cover all, there are so many good books and resources on the web. What I'm interested in is implementation in C: because no matter what, at the end we have to write some code. So everything on this site, figs, text, descriptions are what helped me to understand and implement parts of different video decoders.

Beginnings

I wanted to decode and display compressed video in my homebrew OS. I had no prior knowledge on video compression standards - and obviously had no idea what I'm getting into..

I took a video file .mp4 from my old Nokia N80 and started to hex-dump it:

00000000: 00 00 00 1C 66 74 79 70 | 6D 70 34 32 00 00 00 00 | ....ftypmp42....
00000010: 6D 70 34 32 33 67 70 34 | 69 73 6F 6D 00 07 17 09 | mp423gp4isom....
00000020: 6D 64 61 74 00 00 01 B6 | 10 00 18 3D 61 7C D3 39 | mdat.......=a|.9
00000030: 95 63 2E BD 3D CE 89 9C | D4 CE 70 B1 06 F2 2C B8 | .c..=.....p...,.
00000040: 3E CC 00 75 A5 76 F0 E9 | 40 B3 1E 53 FD A7 98 7A | >..u.v..@..S...z
... 

So I started to read about and write a simple MPEG-4 decoder. Quite unfortunate, it's a highly extended and sophisticated version of previous video compression standards, like H.261, H.262/MPEG-2, but I simply didn't know that at this point.

Methods

Some freely available software (eg. MTS4EA Elementary Stream Analyzer Version 6.1.0 demo) have a few test-streams of various video codecs. I tried to decode these streams and compare the yuv files. A painful process but one can really learn how digital video compression works!

Decoder conformance testing

Some standards makes coded bitstreams along with uncompressed yuv files available for encoder/decoder testing. Again, comparing the yuv-output of your decoder (decoder under test, DUT) with these yuv files is the method.

Video codecs I worked on

Cronological order of my projects:

My recommendation: