http://halicery.com/2Image Decoders/JPEG/Progressive.html

Little about jpeg progressive

(C) 2011 A. Tarpai (tarpai76 at gmail)

This is old... but stands here for reference.

Ad the documentation of a progressive jpeg decoder.

A JPEG file consist of a series of markers, between SOI and EOI, and various lenght bitstreams belonging to each marker:

[marker]---------bits------->


Progressive jpeg has multiple 'scans' in SOS markers: each scan contains a subset of coded coefficients from the whole image, which adds to the previous, giving an option of increased detail level after decoding each scan (progressive display).

Here is a simple progressive jpeg file that has 3 scans (3 full images on different details):

  [SOI]  
    [SOF]-------->
    [DHT]-------->
    [DQT]------>
    [SOS]---------------------------------->
    [SOS]---------------------------------------------------->
    [SOS]--------------------------------------------->
  [EOI]

Progressive display

Baseline jpeg: MCU by MCU

BBBBBBBB......    BBBBBBBBBBBBBB    BBBBBBBBBBBBBB
..............    BBBBBBBBBBBBBB    BBBBBBBBBBBBBB
..............    BBBBBBBBBB....    BBBBBBBBBBBBBB
..............    ..............    BBBBBBBBBBBBBB
..............    ..............    BBBBBBBBBBBBBB

Progressive jpeg: scan by scan

IIIIIIIIIIIIII    PPPPPPPPPPPPPP    BBBBBBBBBBBBBB
IIIIIIIIIIIIII    PPPPPPPPPPPPPP    BBBBBBBBBBBBBB
IIIIIIIIIIIIII    PPPPPPPPPPPPPP    BBBBBBBBBBBBBB
IIIIIIIIIIIIII    PPPPPPPPPPPPPP    BBBBBBBBBBBBBB
IIIIIIIIIIIIII    PPPPPPPPPPPPPP    BBBBBBBBBBBBBB

An example

For simplicity, lets suppose a data unit (DU) consists of 4x4 coefficients (I don't have to draw that much), and forget about successive approximation for now, and sub-sampling also. In progressive jpeg each scan contains coded coefficients belonging to a 'band' (the Ss and Se parameter in the SOS header). This is the example 3 scans:

[SOS]ooooooooooooooooooooooooooooooooooo                     for band [0...0]
[SOS]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx   for band [1...5]
[SOS]zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz          for band [6...15]

and they fill each DU of the whole image in this way:

DU             DU after       DU after       DU after
empty   -->    scan 1    -->  scan 2    -->  scan 3
                                          
. . . .        o . . .        o x x x        o x x x
. . . .        . . . .        x x . .        x x z z
. . . .        . . . .        . . . .        z z z z
. . . .        . . . .        . . . .        z z z z

The first band: DC-band. Always in jpeg. After dequantization (zeroes remain zeroes) idct is performed. The result of the idct is pixel color values finally, something to display! The nature of the idct is, when only the DC coefficient is non zero, all values will be the same.

o . . .         a a a a
. . . .   idct  a a a a
. . . .   --->  a a a a
. . . .         a a a a


Giving a 4x4 pixel area of the final image with the same color. A kind of blocky image, but something to see even after the first scan!

Now.. lets look at the image level with image dimensions of 32 x 16 pixels. Lets forget about sub-sampling also (or look at the figs belonging to only one image color component).

The whole image's coefficient buffer - and that can require quite a lot of memory! Min. 2 bytes / pixel / color component, for a 800x600 color image that is almost 3 MB, and 6 MB using 32-bit integer coefficients! Unacceptable in many applications..

I. Full image coefficient buffer, progressive display

[SOS]ooooooooooooooooooooooooooooooooooo

    o... o... o... o... o... o... o... o...                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    o... o... o... o... o... o... o... o...                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....    convert      IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    o... o... o... o... o... o... o... o...   --------->    IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII    --> display
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    o... o... o... o... o... o... o... o...                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
    .... .... .... .... .... .... .... ....                 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII


[SOS]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    xx.. xx.. xx.. xx.. xx.. xx.. xx.. xx..                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    .... .... .... .... .... .... .... ....                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    .... .... .... .... .... .... .... ....                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    xx.. xx.. xx.. xx.. xx.. xx.. xx.. xx..                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    .... .... .... .... .... .... .... ....                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    .... .... .... .... .... .... .... ....    convert      PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx   --------->    PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP    --> display
    xx.. xx.. xx.. xx.. xx.. xx.. xx.. xx..                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    .... .... .... .... .... .... .... ....                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    .... .... .... .... .... .... .... ....                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    xx.. xx.. xx.. xx.. xx.. xx.. xx.. xx..                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    .... .... .... .... .... .... .... ....                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
    .... .... .... .... .... .... .... ....                 PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP

[SOS]zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz

    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz    convert      BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx   --------->    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB    --> display
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

II. Full image coefficient buffer, non-progressive display

[SOS]ooooooooooooooooooooooooooooooooooo
[SOS]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[SOS]zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz

    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz     convert     BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx    --------->   BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB    --> display
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

III. 1 MCU LINE coefficient buffer, non-progressive display

          JPEG stream data                                           BMP bitplane
    
[SOS]ooooo|.............................                    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[SOS]xxxxxxxx|..........................................    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[SOS]zzzzzz|....................................            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             |                                              ................................
             |   decode                                     ................................
            \ /                                             ................................
             V                                              ................................
                                                            ................................   
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 ................................
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz    convert      ................................
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz   ---------->   ................................
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 ................................
                                                            ................................
            coefficient buffer                              ................................
                                                            ................................
                                                            

[SOS].....|ooooooooo|...................                    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[SOS]........|xxxxxxxxxxx|..............................    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[SOS]......|zzzzzzzz|...........................            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             |                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             |                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
            \ /                                             BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             V                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            ................................   
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 ................................
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 ................................
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz   ---------->   ................................
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 ................................
                                                            ................................
                                                            ................................
                                                            ................................      


[SOS]...............|ooooooo|...........                    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[SOS]...................|xxxxxxxx|......................    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[SOS]...............|zzzzzzzzzzzz|...............           BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             |                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             |                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
            \ /                                             BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             V                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB   
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz   ---------->   BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 ................................
                                                            ................................
                                                            ................................
                                                            ................................
                                                            

[SOS]...........................|ooooooo|                   BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[SOS]...................................|xxxxxxxxxxxxxx|    BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[SOS]..............................|zzzzzzzzzzzzz|          BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             |                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             |                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
            \ /                                             BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
             V                                              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB    --> display
    oxxx oxxx oxxx oxxx oxxx oxxx oxxx oxxx                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    xxzz xxzz xxzz xxzz xxzz xxzz xxzz xxzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz   ---------->   BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
    zzzz zzzz zzzz zzzz zzzz zzzz zzzz zzzz                 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                                                            BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

Q: Why one MCU LINE and not a single MCU, as in your baseline jpeg decoder?
A: When sub-sampling comes into the picture - literally :) - single-component scan data does not follow the 'MCU order'.

The 'placement' of one coefficient block over the final image is different in these two. Here is the essence of it, as in the most common sub-sampling schema, showing the data units of the luma plane.

The Decoding order is the same in both scans:
    
    0 1 2 3 4 5 6 7 8 9 . . . . . . . . . . . . . . . .

Display order Interleaved:

        0 1 4 5 8 9 . .
        2 3 6 7 . . . .

Display order Single:

        0 1 2 3 4 5 6 7 
        8 9 . . . . . . 

My coefficient buffer is MCU-based. Just a decision.

Successive approximation

From the decoder point of view. Why? Because it's really not easy to just 'read backward' the figs in the standard, so here it is.

The most difficult part is how to handle runs and eob-s. As a thumbrule: every RUN skips zeroes - and zeroes only. Here is an example coefficient block from the previous scan: some coefficients have already a non-zero value (a,b and c). The k is the actual index during decoding. Lets suppose k points to 'a' and a new Huffman code comes.. a new coefficient with different RUN values:

. . . . . . a x 0 b 0 0 0 c 0 0 0 0 0 0 0 0 0 0 0 0 0 
            |
            k

- read new coef x
- read r 
- skip r zeroes AND read as many successive bits as non-zero on the way!

Examples:
                
. . . . . . a x 0 b 0 0 0 c 0 0 0 0 0 0 0 0 0 0 0 0 0 (r=0, 1 successive bit read)
            | |
            k k

. . . . . . a 0 x b 0 0 0 c 0 0 0 0 0 0 0 0 0 0 0 0 0 (r=1, 1 successive bit read)
            |   |
            k   k
                                
. . . . . . a 0 0 b x 0 0 c 0 0 0 0 0 0 0 0 0 0 0 0 0 (r=2, 2* successive bit read)
            |       |                      
            k       k

. . . . . . a 0 0 b 0 x 0 c 0 0 0 0 0 0 0 0 0 0 0 0 0 (r=3, 2* successive bit read)
            |         |                                
            k         k

                                             (Note 2*: corrected by JC Li, thanks!)
               ....

               
                                         
. . . . . . a 0 0 b 0 0 0 c 0 0 0 0 0 0 0 0 0 0 x 0 0 (r=15, 3 successive bit read)
            |                                   |
            k                                   k


ZRL:

. . . . . . a 0 0 b 0 0 0 c 0 0 0 0 0 0 0 0 0 0 0 0 0 (r=15, 3 successive bit read)
            |                                   |
            k                                   k
    
    - move 15 zeroes AND do nothing after! 

The next iteration moves k to the next position and the process repeats.

When an EOB code comes, it's a little simpler:

. . . . . . a 0 0 b 0 0 0 c 0 0 0 0 
            |
            k - - >
            
- find all non-zero and read successive bits
- 3 successive bits read
- done with this block. 

Some ASCII graphics..

+-----+
| MCU |
+-----+

DU

Everything is jpeg is based on 8x8 coefficient blocks, the data unit (DU), as the unit for the invers DCT. This is the 'resolution' of the jpeg compression.

DC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC

When sub-sampling is not used, each pixel has an associated value from the 3 YUV color components (color jpeg mostly works in the YUV color space).

     8x8 Y values              8x8 U values               8x8 V values

DC AC AC AC AC AC AC AC    DC AC AC AC AC AC AC AC    DC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC


This corresponds with an 8x8 pixel area of the image.

When - for example - both chroma components are half sub-sampled, each 2x2 pixel has its own luma value, but share the same chroma value. Because jpeg compression has a 'DU-resolution', this means 'each 2x2 luma-Y DU share the same chroma-U/V DU', and the structure looks like this and covers a 16x16 pixel area of the image:

     8x8 Y0 values             8x8 Y1 values           
     
DC AC AC AC AC AC AC AC    DC AC AC AC AC AC AC AC    
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC    
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC           8x8 U values                 8x8 V values       
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC                                                           
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC       DC AC AC AC AC AC AC AC      DC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC       AC AC AC AC AC AC AC AC      AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC       AC AC AC AC AC AC AC AC      AC AC AC AC AC AC AC AC
                                                         AC AC AC AC AC AC AC AC      AC AC AC AC AC AC AC AC
     8x8 Y2 values             8x8 Y3 values             AC AC AC AC AC AC AC AC      AC AC AC AC AC AC AC AC
                                                         AC AC AC AC AC AC AC AC      AC AC AC AC AC AC AC AC
DC AC AC AC AC AC AC AC    DC AC AC AC AC AC AC AC       AC AC AC AC AC AC AC AC      AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC       AC AC AC AC AC AC AC AC      AC AC AC AC AC AC AC AC
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC                                                           
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC                                                           
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC                                                           
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC   
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC   
AC AC AC AC AC AC AC AC    AC AC AC AC AC AC AC AC   



.. or simply

Y0 Y1
Y2 Y3  U  V


This is the MCU!

MCU-s are stored always from left-to-right, top-to-bottom in the jpeg stream, MCU by MCU:

| Y0 Y1 Y2 Y3 U V | Y0 Y1 Y2 Y3 U V | Y0 Y1 Y2 Y3 U V | Y0 Y1 Y2 Y3 U V .... 

The coefficients itself are compressed using Huffman-coding (see ...), in a zig-zag order, with zero-run codes, etc..

MCU-s in the jpeg stream repeats and covers the whole image (always full MCU-s), also from left-to-right, top-to-bottom:

    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU -->
--> MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 

The MCU consist of data units that form an independent coded block of samples from each color components and the structure is specified in the jpeg header (SOF):

This information is enough for the decoder to build up and process a given MCU configuration. Luckily, H and V has only values of 1..4, giving a maximum ration of 1:4 sub-sampling.

Here is an example 22-12-11 (YUV) sub-sampling (I have such an image). The MCU will be:

Y Y Y Y U U V

Each MCU codes a 16 x 16 pixel area of the original image (max H and V is 2). 
Y is not sub-sampled.
U is sub-sampled by 2 horizontally
V is sub-sampled by 2 both horizontally and vertically. 

As YUV-planes of one MCU (16 x 16 pixel)
        
    YY  (16 x 16 pixel)
    YY
    
    U
    U

    V

The YUV-planes of the whole image (when original image has a dimension of 160 x 48)

    YYYYYYYYYYYYYYYYYYYY    (10 x 6 MCU)
    YYYYYYYYYYYYYYYYYYYY
    YYYYYYYYYYYYYYYYYYYY
    YYYYYYYYYYYYYYYYYYYY
    YYYYYYYYYYYYYYYYYYYY
    YYYYYYYYYYYYYYYYYYYY

    UUUUUUUUUU
    UUUUUUUUUU
    UUUUUUUUUU
    UUUUUUUUUU
    UUUUUUUUUU
    UUUUUUUUUU

    VVVVVVVVVV
    VVVVVVVVVV
    VVVVVVVVVV

Levels of decoding

     coefficient decoding
              |
-------- coef buffer ---------
              |
      de-quantization
              |
            idct
              |
------ MCU-pixel buffer ----------
              |
   color-space conversion
            (bmp)

Levels of coefficient decoding

[SOS]01010101010101010101010100011110101110011010100011111110000000101010100101000111001

DU
    
    8x8 coefficient values

    DC AC AC AC AC AC AC AC
    AC AC AC AC AC AC AC AC
    AC AC AC AC AC AC AC AC
    AC AC AC AC AC AC AC AC
    AC AC AC AC AC AC AC AC
    AC AC AC AC AC AC AC AC
    AC AC AC AC AC AC AC AC
    AC AC AC AC AC AC AC AC

MCU 

    DU DU DU DU DU DU DU

COEFFLINE

    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
                 |
                 |
                 MCU --> color-conversion

COEFFIMAGE

    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
    MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU MCU 
                 |
                 |
                 MCU --> color-conversion

Note that color conversion is always MCU-based (ie. it doesn't matter how big the coefficient buffer is). Some examples of coefficient buffer size:


I've tried all 3...

Baseline and progressive

To be written..

Interleaved and single scans

To be written..


Sun Jun 17 18:48:06 UTC+0200 2018 © A. Tarpai