Benchmarking Video Codecs for Blockiness Compression Artefact

A. Punchihewa, J. Armstrong, ‘Benchmarking Video Codecs for Blockiness CompressionArtefact’, Proceedings of Image and Vision Computing New Zealand 2007, pp. 223–226, Hamilton, New Zealand, December 2007.Benchmarking Video Codecs for Blockiness CompressionArtefactA. Punchihewa, J ArmstrongInstitute of Information Sciences and Technology, Massey University, Palmerston North, 4410, New Zealand.Email: g.a.punchihewa@massey.ac.nzAbstractThis paper presents a method to benchmark video codecs objectively for the blockiness artefact. A static testpattern previously designed to highlight blockiness artefact of image codecs was applied to video codecs undertest to evaluate the performance when intra-frame coded mode is used. The performance of the video codec ininter-frame coded mode is evaluated using an image sequence generated using a similar function used togenerate the still test pattern. A software codec having two implementations, namely MPEG-2 and MPEG-1 wasconfigured at different bit rates. At low bit rates, the codec tested as MPEG-1 performs better than the MPEG-2with respect to the blockiness distortion. The test sequences and the blockiness distortion measure algorithm areuseful tools to evaluate codecs for blockiness artefact. Different types of video codecs or differentimplementations of similar type of video codec can be benchmarked for performance against blockinessdistortion using the tools developed.Keywords: Video codecs, blockiness artefact, benchmarking, test patterns, compression artefacts, MPEG-1,MPEG-21 IntroductionAdvances in computing and communicationtechnologies have led to common use of digital mediacontent such as pictures and video. However, digitalimages and video are still demanding in terms ofstorage space and transmission bandwidth. Lossycompression is essential to manage the raw image andvideo data before they are stored or transmitted. Theimage and video codecs use lossy compressiontechniques such as quantisation which will introducedistortions known as artefacts. Blockiness is acommon and dominant form of compression artefactin decoded images and video. References [1 to 6]address image artefacts and objective evaluation ofimage coding artefacts. Therefore, this paperconcentrates on video artefacts only.Video codecs such as MPEG uses JPEG like codingto compress key frames or I frames. I frames are theintra-frame coded frames that are generated at regularframe interval known as group of pictures (GOP).There are two other types of coded frames known asforward predicted frames and backward predictedframes. They are based on frame differences insuccessive frames and either forward predictedmotion vectors or bi-directionally predicted motionvectors are used. The longer the GOP, the greater thereduction in bit rate as the process takes the temporalredundancies into account. This research intends toinvestigate the impact of GOP on the blockinessdistortion over individual frames.2 Methodology2.1 MotivationMany researchers have developed blockiness artefactmitigation techniques which can be applied as preprocessingor post processing to a standard codec.Codecs such as MPEG-4 has optional features to preprocessvideo to mitigate blockiness artefacts.Usually, such pre or post-processing technique tradesoffone form of artefact to another. Gao et al.proposed a de-blocking algorithm [7] as a postprocessingalgorithm. The technique proposed iscalled semaphore extraction which attempts to reducethe blur due to post-processing of image boundariesas block boundaries. It is difficult to evaluate theeffectiveness of such techniques proposed as thesubjective methods are expensive and inconsistentthough they are the ultimate measure. An objectivemeasure based on a known reference would providean accurate and repeatable measure for blockinessartefact.The video codecs such as MPEG defines andstandardise the syntax for the decoder only. Hencemany implementations are possible for an encoder.Objective distortion measure tools for artefact willprovide an accurate and convenient method toevaluate codecs for comparison. Tan et al. proposedan algorithm [8] to detect and measure blockinessbased on the amplitude and phase information of theharmonics generated by the blockiness artefact in223

MPEG-2 coded video. The phase information of theharmonics was used to eliminate image boundariesbeing detected and measured as block boundaries.2.2 Design of Test Patterns andMetricBlockiness is the false block structure that wouldappear in reconstructed video. If a test pattern isgenerated with no perceivable edges in uncompressedsequence, then each frame would enables theevaluation of blockiness accurately in the codedsequence.2.2.2 Design of blockiness algorithmThis research is an extension of work done previouslyfor still image codecs. Some of the findings arepublished in references [2 to 6]. This work re-uses thesame blockiness algorithm presented in reference [4]to evaluate the level of blockiness in the reconstructedtest patterns/sequences. The figure 2 shows areconstructed frame from the test sequence. The blockboundaries are clearly visible and original frame didnot have any of those boundaries.2.2.1 Design of test PatternDesign of a test pattern as a static pattern with noboundaries is described in detail in the reference [4].The same test pattern was animated over a virtualimage space and then captured at the size of the framerequired for the codec. The test sequence generatoralgorithm reads test image data from the capturewindow having image size equal to the frame size)and then is stored as an uncompressed test sequence.When codec under test to be stressed, test sequence isread from the storage to the codec input. A framefrom the test pattern/sequence is shown in figure 1.Figure 2: A decoded frame from the testpattern/sequence used to evaluate blockiness artefactafter compression.The figure 3 shows distribution of blockiness measureover the complete frame. The modified algorithmcalculates the vertical and horizontal intensitygradients in the reconstructed test pattern, thenaverage of blockiness for each pixel. When thegradient values are greater than pre-defined thresholdof 2% of the maximum representable original value,which is based on an informal subjective test, thatblockiness value is classified and counted separatelytowards the calculation of global average for thecomplete frame considered.Figure 1: A frame prior to compression from the testpattern/sequence used to evaluate blockiness artefactdue the codec under test.It can be observed that in the above frame there are novisible boundaries. The spatial distribution of theintensity of each of the frame in the test sequence hasbeen defined as a raised cosine. The intensity of eachof the pixel is defined so that, in the direction of x-axis or in the direction of y-axis, the intensitygradients are less than 2% of the maximum value ofthe intensity; in this case 8 bits were used hence on ascale of 255.Number of pixels35003000250020001500100050001 2 3 4 5 6 7 8 9 10 11 12 13Blockiness of each pixelFigure 3: The histogram of blockiness of each ofpixels for the MPEG-1 codec.224

3 Results and DiscussionExperiments were carried out to evaluate theblockiness introduced by each codec MPEG-1 andMPEG-2 when intra-frame coded is used. Figure 4and figure 5 show the variation of average blockinessover the complete frame when intra-framed codecunder different quality factors. In figure 3 it can beobserved that the individual blockiness or gradientsvary between 1 and 7. Gradient of 4 and above doesnot make any perceivable distortion as they are smallgradient though large in numbers in the totalpopulation.Blockiness4.543.532.520 20 40 60 80 100Quality factorFigure 4: Average blockiness vs. Quality factor forMPEG-1 codec.4.543.1 The Impact of GOP on averageblockinessThe test pattern/sequence was compressed usingmany GOP structures such as I, I, …, or I, B, P, B orI, P I, P. so on. Figure 6 and figure 7 arerepresentative plot of average blockiness for GOP ofI, B, P, B. This GOP structure was applied to bothMPEG-1 and MPEG-2 codecs. The averageblockiness was measured on each frame for thesequence. The three types of frames, namely, Intraframecoded frame I, forward predicted frames P andthe bi-directional predicted frames B showed a similarpattern over the complete test sequence. The highestaverage blockiness resulted from I frames, then Pframes and B frames had the lowest value. This is dueto the fact that P frames are generated using I framesand B frames are generated using P and I framesresulting averaging or low pass filtering.Blockiness43.532.521.510.501 2 3 4 5 6 7 8 9 10Frame numberBlockiness3.532.5Figure 6: Average blockiness for MPEG-1 codecwith GOP of IBPBIn general MPEG-1 resulted in more averageblockiness compare to MPEG-2. I frames always hadthe highest and fairly constant level of blockiness.20 20 40 60 80 100Quality factorFigure 5: Average blockiness vs. Quality factor forMPEG-2 codec.Quality factor is the control parameter given to theuser to select the level of quality required. The codecwill set a certain quantisation level based on thequality factor set by the user and end up havingdifferent bit rate. As quality factor is less meaningfulfor comparisons subsequent experiments used the bitrate. Two objectives of the research were toinvestigate on the variation of blockiness with varyingframes types such as I, P and B or the GOP structuresand to benchmark or compare level of blockinessdistortion introduces by the two codecs MPEG-1 andMPEG-2.Blockiness43.532.521.510.501 2 3 4 5 6 7 8 9 10Frame numberFigure 7: Average blockiness for MPEG-2 codecwith GOP of IBPB225

3.2 MPEG-1 vs. MPEG-2When video codecs are to be benchmarked, they needto be compared at a given bit rate. Experiments werecarried out to check the level of average blockinessintroduced by each MPEG-1 and the MPEG-2 codecsunder different bit rates but for the same testsequence. Figure 8 shows the results of theexperiment. Around 1750 kbps, performance curvescross each other.Blockiness9876543MPEG-1MPEG-220 500 1000 1500 2000 2500 3000bitrate (kb/s)Figure 8: A comparison between MPEG-1 andMPEG-2 codecs for average blockiness on thesame test sequence.For bit rates below 1750kbps, the MPEG-1 performsbetter. For high bit rate applications MPEG-2introduce less average blockiness. In this case withthe given test sequence, if the video bit rate is higherthan 1750kbps, MPEG-2 codec outperforms MPEG-1against the blockiness distortion. MPEG-1 wasdesigned for CD-ROM applications. Hence it has notoptimised to handle bit rates above 2Mbps where asMPEG-2 being used for broadcast application, it canhandle well the bit rates above 2Mbps.4 ConclusionsThe test patterns developed for still image codecs canbe easily extended for video codecs evaluations. Forhigh bit rates, MPEG-2 outperforms MPEG-1 codec.The blockiness algorithm and test sequences areuseful tools for codec evaluations against blockinessartefact distortion. Further improvements can be doneto the test sequence by identifying the limitations ofthe animated test sequence. The blockiness algorithmand the test sequence can be used to benchmark thedifferent implementations of the same type of codecssuch as MPEG-2 as well.5 Future workFuture work involves comparing similar type ofcodecs. Recently acquired state of the art codec forbroadcast applications, namely a MPEG-4 codec, willbe benchmarked against the available MPEG-2 codec.References[1] F. Ebrahimi, M. Chamik, and S. Winkler,“JEPG vs. JPEG2000: An objectivecomparison of image encoding quality” Theproceeding of the International Society foroptical Engineering, Applications of DigitalImage Processing, XXVII, Vol. 5558, pp.300– 308, 2004.[2] A. Punchihewa and D. G. Bailey, Artefactsin Image and Video Systems; Classificationand Mitigation, Proceedings of theConference of Image and Vision ComputingNew Zealand, Auckland, New Zealand,2002, pp. 197-202.[3] A. Punchihewa, D. G. Bailey and R. M.Hodgson, A Survey of Coded Image andVideo Quality Assessment, Proceedings ofImage and Vision Computing New Zealand,Palmerston North, New Zealand, 2003, pp.326-331.[4] A. Punchihewa, D. G. Bailey and R. M.Hodgson, Objective Quality Assessment ofCoded Images: The Development of NewQuality Metrics, Proceedings of Internet,Telecommunication Conference, Adelaide,Australia, 2004, pp. 1-6.[5] G. A. D. Punchihewa, D. G. Bailey and R.M. Hodgson, The Development of aSynthetic Colour Test Image for Subjectiveand Objective Quality Assessment of DigitalCodecs, Proceedings of Asia-PacificCommunication Conference, 2005, pp. 881-885[6] A. Punchihewa, D. G. Bailey and R. M.Hodgson, The Development of a NovelImage Quality Metric and a Synthetic ColourTest Image for Objective QualityAssessment of Digital Codecs, Proceedingsof the Telecommunication Conference,Melbourne, Australia, 2005, pp 1-6.[7] W. Gao, C. Mermer, and Y. Kim, “A deblockingalgorithm and a blockiness metricfor highly compressed images” Theproceeding of the IEEE transaction oncircuits and systems for video technology,vol. 12, No 12, pp. 1150–1159, 2002.[8] K. T Tan, and M. Ghanbari, “Blockinessdetection for MPEG-2 coded video” IEEEsignal processing letters, vol. 7, No 8 pp.213–215, 2000.226