Analysis of ITU H.263 Deblocking Filter

(C) 2012 Attila Tarpai (tarpai76 gmail)

What is it?

In block-based video motion compensation blocking artifacts may appear on block edges. This is quite ugly and there are numerous methods and algorithms to reduce and make them disappear from the reconstructed video frame. H.263 defines it's own deblocking filter when Annex J is ON. This is Annex J Deblocking Filter in ITU-T Recommendation H.263.


I was interested in writing video decoders for my small, homebrew OS in C. Images has been generated by this decoder, a pure 32-bit ANSI C integer implementation. Deblocking is quite a heavy load within the decoding process, so the main goal of this implementation is to mimimize the number of operations.

Source: MTS4EA example stream Person track.

Analysis - The core

ITU-T Recommendation H.263 Annex J Deblocking Filter operates on a set of 4 pixel values on 8x8 block boundaries (A, B in Block 1 and C, D in Block 2), possibly changes the values based on a series of equations.

These images below show all the locations where pixel values had to be analised and possibly changed, on the edges of each 8x8 block. First in the luma plane. I simply set all values to zero (became black on the decoded frame) in the deblocking routine to see affected pixels and produced the following image:

On the chroma planes (U- and V) due to subsampling, the picture is different:


There are 3 parameters computed in the deblocking routine: d, d1 and d2. If any of them are zero, no further processing is required.

Parameter d

d= (A-4B+4C-D) / 8

Based on the above, this image shows the remaining luma locations where d is non-zero and deblocking should proceed at all, decreasing how many of the possible luma pixel quadrlets had to be filtered:

The exact number of all cases when d computed is non-zero for this particular 169-frame video sample was 62% - so we’ll save a lot of processing by examining d for zero.

Parameter d1

d1= MAX(0, d - MAX(0, 2*(d - strength))) 

These are the remaining locations where d1 is non-zero (luma only), which simply removes more cases:

Parameter d2

d2= clipd1((A-D)/4, d1/2) 

Further, when d2 is zero, A and D will not change either. This is the final test image then, with all the pixel locations where pixel values has been changed due to the filtering in the luma-plane:

and in the 2 chroma-planes (on either U- or V-plane, appearing as different shades):

This is the original frame and zooming on to top-left corner (from MTS4EA sample H.263+ stream, the first I-frame) before and after applying the deblocking filter:



It really works!

An example

Lets see an example of these 4 pixels along a vertical block boundary:

 A   B  |  C   D  
197 200 | 210 203

Parameters of d, d1 and d2 are non-zero, so values will be adjusted by the d1 and d2 calculated by the filter and added as:

 A   B  |  C   D
197 200 | 210 203  
-d2 +d1 | -d1 +d2
 A'  B' |  C'  D'

On this particular edge the quantization parameter was 13, which gives strength=6:

d = (A-4B+4C-D)/8 = 34/8 = 4
d1= MAX(0, d - MAX(0, 2*(d - strength))) = MAX(0, 4 - MAX(0, 2*(4 - 6))) = MAX(0,4) = 4
d2= clipd1((A-D)/4, d1/2) = clipd1(-6/4, 2) = -1

The final pixel values, where d1=4 and d2=-1 will be:

 A   B  |  C   D  
197 200 | 210 203
-d2 +d1 | -d1 +d2
198 204 | 206 202

Parameter d1 is a product of the H.263 Ramp-function, while d2 is simply a range-clip.