YUV conversion investigations

Because sometimes it's just amazing what lies behind one line of code

YUV intro

YUV comes from analogue TV.

First there was a B/W-system with one brightness signal (luminance, Y):

          B/W sender                      B/W receiver    
      +-----------------+                 +---------+
R --> | Wr -->          |                 |         | 
G --> | Wg -->    Y     | ~~~~~ Y ~~~~~~> |    Y    |  
B --> | Wb -->          |                 |         |
      +-----------------+                 +---------+

Y = Wr R + Wg G + Wb B

The overall brightness (luminance) is a weighted sum of the 3 RGB color components computed by the sender. Constants may vary, but Wr + Wb + Wg = 1, so full RGB (white) makes full luma brightness. Also Wg = 1 − Wb − Wr so only Wb and Wr can specify the conversion.

Color television system added two 2 chrominance signals, without changing Y, as red- and blue difference. The color TV has 3 electron guns luminating R-G-B triplets on the phosphorus screen surface.

         Color sender                    Color receiver
      +-----------------+                 +----------+
R --> | Wr -->    R-Y   | ~~~~~ V ~~~~~~> | Wr --> R |   
G --> | Wg -->    Y     | ~~~~~ Y ~~~~~~> | Wg --> G |   
B --> | Wb -->    B-Y   | ~~~~~ U ~~~~~~> | Wb --> B |   
      +-----------------+                 +----------+

Y = Wr R + Wg G + Wb B
U = B - Y
V = R - Y

The color receiver recovers the original RGB levels by:

R = Y + V
G = Y − Wb/Wg U − Wr/Wg V
B = Y + U

Explaning G from Y = Wr R + Wg G + Wb B:

Wg G = Y − Wb B − Wr R
G = ( Y − Wb B − Wr R ) / Wg
G = ( Y − Wb(Y+U) − Wr(Y+V) ) / Wg
G = ( Y − WbY − WbU − WrY − WrV ) / Wg
G = ( (1 − Wb − Wr) Y − Wb U − Wr V ) / Wg
G = ( Wg Y − WbU − WrV ) / Wg
G = Y − Wb/Wg U − Wr/Wg V

Note, that color receivers should use the same Wb/Wr constants as the sender.

But why color difference and not simply R/B? Compatibility: between senders and receivers.

So a B/W TV could still receive colored sending and show as B/W by simply not using the chroma signals:

+--------+                  +--------+
|        | ~~~~~ V          |        |
| color  | ~~~~~ Y ~~~~~~~> |  B/W   |
|        | ~~~~~ U          |        |
+--------+                  +--------+

And a color TV could still receive and display a B/W sending having chrominances set to zero:

+--------+                  +--------+    
|        |           0 ~~~> |        |    
|  B/W   | ~~~~~ Y ~~~~~~~> | color  |    
|        |           0 ~~~> |        |    
+--------+                  +--------+    

This really works, all RGB will be the same, the color TV shows the B/W picture:

R = Y + 0 = Y
G = Y − Wb/Wg U − Wr/Wg V = Y
B = Y + 0 = Y

YUV levels

Lets say analogue RGB levels are of [0..1]. Then Y is also [0..1]. U/V are Wb/Wr-dependent.

It may look like this:

1  ----- ~~~~~~~~~~~~~~~~~ ---------------------------------------------------------------------------------  1
         ~~~~~~~~~~~~~~~~~
         ~~~~~~~~~~~~~~~~~            ~~~~~~~~~~~~~~~~~ +Umax      
         ~~~~~~  Y  ~~~~~~            ~~~~~~~~~~~~~~~~~              ~~~~~~~~~~~~~~~~~ +Vmax       
         ~~~~~~~~~~~~~~~~~            ~~~~~~~~~~~~~~~~~              ~~~~~~~~~~~~~~~~~  
         ~~~~~~~~~~~~~~~~~            ~~~~~~~~~~~~~~~~~              ~~~~~~~~~~~~~~~~~
         ~~~~~~~~~~~~~~~~~            ~~~~~~  U  ~~~~~~              ~~~~~~  V  ~~~~~~
0  ----- ~~~~~~~~~~~~~~~~~ ---------- ~~~~~~~~~~~~~~~~~ ------------ ~~~~~~~~~~~~~~~~~ ---------------------  0
                                      ~~~~~~~~~~~~~~~~~              ~~~~~~~~~~~~~~~~~
                                      ~~~~~~~~~~~~~~~~~              ~~~~~~~~~~~~~~~~~        
                                      ~~~~~~~~~~~~~~~~~              ~~~~~~~~~~~~~~~~~ -Vmax
                                      ~~~~~~~~~~~~~~~~~ -Umax       
                                      
-1 --------------------------------------------------------------------------------------------------------- -1

Computing ±Umax and ±Vmax using ITU BT601 constants (Wb = 0.114, Wr = 0.299). The result is ±Umax = 1 − Wb and ±Vmax = 1 − Wr from

Y = Wr R + Wg G + Wb B
U = B − Y
V = R − Y

or

U = B − Wr R − Wg G − Wb B = B ( 1 − Wb ) − Wg G − Wr R
V = R − Wr R − Wg G − Wb B = R ( 1 − Wr ) − Wg G − Wb B

From these equations is easy to see the minimun and maximum possible values for U/V.

Note, that U/V range is bigger than 1:

2Umax = 1.772
2Vmax = 1.402

8-bit digital YUV

The analogue YUV values has to be converted to positive integer numbers and stored on 8 bits each. There are two main standards:

                           Full           Ranged
                         
Y [0..1]                 Y' [0..255]    Y' [16..235]
U [-Umax..Umax] ------>  Pb [0..255]    Cb [16..240]
V [-Vmax..Vmax]          Pr [0..255]    Cr [16..240]

Digital YUV calculations

Possible equations for full range:

Y' = 255Y
Pb = 255(U + Umax) / 2Umax
Pr = 255(V + Vmax) / 2Vmax

Equations for ranged digital YUV:

Y' = 219Y + 16
Cb = 224(U + Umax) / 2Umax + 16
Cr = 224(V + Vmax) / 2Vmax + 16

or

Pb = ( 255 / 2Umax ) U + 127.5
Pr = ( 255 / 2Vmax ) V + 127.5

Cb = ( 224 / 2Umax ) U + 128
Cr = ( 224 / 2Vmax ) V + 128

Note that the so called zero-point is a little different for the two types of chroma: 128 vs. 127.5. According the the standard both are computed with the value of 128. This will slightly overestimates the results and has to be clipped too for Y'PbPr.

Y' = [ 255 Y ]
Pb = CLIP[ (255/2Umax) U + 128 ]
Pr = CLIP[ (255/2Vmax) V + 128 ]

Y' = [ 219 Y ] + 16
Cb = [ (224/2Umax) U ] + 128
Cr = [ (224/2Vmax) V ] + 128

where [ ] denotes rounding to nearest integer.

Decoders: YUV to RGB

What's interesting from the decoder's point of view, how to convert the decoded digital YUV values (JPEG, MPEG, H.263..) to digital RGB values. JPEG, MPEG, H.264, etc. compress in the YUV domain. After decoding there are some digital YUV result to convert to RGB. Usually specified in the header what type of YUV the result is supposed to be: ranged or full, and what Wb/Wr coefficients used for computing Y (luminance).

The task is to find equations for

Y' [0..255]      R [0..255]       Y' [16..235]      R [0..255]
Pb [0..255] ---> G [0..255]       Cb [16..240] ---> G [0..255]
Pr [0..255]      B [0..255]       Cr [16..240]      B [0..255]

based on the analogue equations of

R = Y + V
G = Y - Wb/Wg U - Wr/Wg V
B = Y + U

First compute Y, U and V from digital YUV values going backwards. Note that 128 zero point is used below for converting from full range PbPr:

Y = Y' / 255
U = 2Umax/255 ( Pb - 128 )
V = 2Vmax/255 ( Pr - 128 )

and from CbCr:

Y = ( Y' - 16 ) / 219
U = 2Umax/224 ( Cb - 128 )
V = 2Vmax/224 ( Cr - 128 )

Because of the final 8-bit RGB values will be ×255 of the analogue RGB values, we can multiply these YUV values by 255 instead of the analogue RGB values. Lets say Y8, U8 and V8. Then the 8-bit RGB will be:

R8 = Y8 + V8
G8 = Y8 - Wb/Wg U8 - Wr/Wg V8
B8 = Y8 + U8

From full range PbPr:

Y8 = Y'
U8 = 2Umax ( Pb - 128 )
V8 = 2Vmax ( Pr - 128 )

and from CbCr:

Y8 = 255/219 ( Y' - 16 )
U8 = 255/224 2Umax ( Cb - 128 )
V8 = 255/224 2Vmax ( Cr - 128 )

Then computing R8-G8-B8 can be also written as

R8 = Y8 + 2Vmax ( Pr - 128 )
G8 = Y8 - Wb/Wg 2Umax ( Pb - 128 ) - Wr/Wg 2Vmax ( Pr - 128 )
B8 = Y8 + 2Umax ( Pb - 128 )

and

R8 = Y8 + 255/224 2Vmax ( Cr - 128 )
G8 = Y8 - 255/224 Wb/Wg 2Umax ( Cb - 128 ) - 255/224 Wr/Wg 2Vmax ( Cr - 128 )
B8 = Y8 + 255/224 2Umax ( Cb - 128 )

We can see that these final equations are quite similar, only the rational constants vary. That is 8-bit RGB can be computed with 4 (PbPr) or 5 (CbCr) non-integer multiplications.

Excel sheet

yuv.xls

I've made an Excel sheet to investigate YUV-RGB conversions.

It starts with RGB for black, white, a grey, the 3 primary and 3 complementary colors, a fix and one random color.

Then it computes float YUV values according to the equations using the given Wb/Wr constants.

It computes Y'PbPr (full range digital YUV) values in 3 steps:

Computing limited Y'CbCr from the same RGB colors is similar, but clipping is not nesessary.

In the second half of the sheet we try the recover the the RGB values from the 8-bit digital YUV values. For Y'PbPr we go as the standard using 128 zero-chroma. Note that from 8-bit digital YUV recovering is mathematically already impossible due to the lost fractions of the stored 8-bit integer values. In 3 stages:

Then compare the RGB values with the original ones: marked with red when different.

As a conclusion: using 8-bit integer values to represent Y'PbPr/Y'CbCr will give +/-1 errors and incorrect fully saturated RGB colors, with even the highest precision computations.

Conversion functions in C

Some example C-code using CPU floating point:

YPbPr_to_RGB(int Y, int Pb, int Pr)
{
  double y = Y + .5;  // add rounding bias
  int u = Pb - 128;
  int v = Pr - 128;
  int R, G, B;
  
  R = Clip ( y + 1.402 * v );
  G = Clip ( y - 0.3441 * u - 0.7141 * v );
  B = Clip ( y + 1.772 * u );
}


YCbCr_to_RGB(int Y, int Cb, int Cr)
{
  double y = 255.0/219.0 * (Y - 16) + .5;  // add rounding bias
  int u = Cb - 128;
  int v = Cr - 128;
  int R, G, B;
  
  R = Clip ( y + 1.596 * v );
  G = Clip ( y - 0.3918 * u - 0.813 * v );
  B = Clip ( y + 2.0172 * u );
}

Clipping is required, when restoring RGB values from 8-bit digital YUV. The fraction is lost.

Conversion using CPU integer arithmetics

The equations above all used real numbers to compute integer RGB from integer YUV values. Not using floating point instructions may speed up code execution on some CPUs. Fix-point integer arithmetics may nicely approximate results. The CPU just have to multiply, add and shift - all fast integer instructions. Or use pre-computed tables.

Examples using ITU BT601 constants (Wb and Wr) and full YUV range (JPEG).

The new integer constants are computed using for example 5-fix point, as 45 = round( 1.402 * (1<<5) ) = round( 44.864) etc.

YPbPr_to_RGB(int Y, int Pb, int Pr)  // fix=5
{
  int y = ( Y << 5 ) + 16;  // add rounding bias
  int u = Pb - 128;
  int v = Pr - 128;
  int R, G, B;
  
  R = Clip ( (y + 45 * v)>>5 );
  G = Clip ( (y - 11 * u - 23 * v)>>5 );
  B = Clip ( (y + 57 * u)>>5 );
}


YCbCr_to_RGB(int Y, int Cb, int Cr)  // fix=8
{
  int y = 298 * ( Y - 16 ) + 128;  // add rounding bias
  int u = Cb - 128;
  int v = Cr - 128;
  int R, G, B;
  
  R = Clip ( (y + 409 * v) >> 8 );
  G = Clip ( (y - 100 * u - 208 * v) >> 8 );
  B = Clip ( (y + 516 * u) >> 8 );
}

Conversion using pre-computed data tables

When a few KB of memory is not an issue, constants can be stored in 5 arrays of 256 integer values, and perform the conversion without multiplication. This is also a little more accurate than the one with integer multiplications.

static int  Ky[256]=  {16,48,80,112,144,176,208,240,272,304,336,368,400,432,464,496,528,560, ...    
static int  Ku[256]=  {-7258,-7201,-7145,-7088,-7031,-6975,-6918,-6861,-6804,-6748,-6691,-6634, ...
static int  Kv[256]=  {-5743,-5698,-5653,-5608,-5563,-5518,-5473,-5429,-5384,-5339,-5294,-5249, ...
static int Kug[256]= {-1410,-1399,-1388,-1377,-1366,-1355,-1344,-1332,-1321,-1310,-1299,-1288, ...
static int Kvg[256]= {-2925,-2902,-2879,-2857,-2834,-2811,-2788,-2765,-2742,-2719,-2697,-2674, ...

YPbPr_to_RGB(int Y, int Pb, int Pr)
{
  int R, G, B;
  
  R = ( Ky[Y] + Kv[Pr] ) >> 5;
  G = ( Ky[Y] - Kug[Pb] - Kvg[Pr] ) >> 5;
  B = ( Ky[Y] + Ku[Pb] ) >> 5;
}

What's great with this implementation is that the same function can be used for Y'CbCr (ranged) digital YUV, or for other Wb/Wr weights, like BT709. Only the static arrays will be different.


Fri Jun 21 11:17:58 UTC+0200 2019 © A. Tarpai