UNDERSTANDING COLOUR IMAGING Chakravarthy Bhagvati

WORKSHOP ON RECENT ADVANCES IN COMMUNICATIONS AND SIGNAL PROCESSING (21,22 AUG ’09)UNDERSTANDING COLOUR IMAGINGTHE PHYSICAL, THE VISUAL AND THE DIGITAL DOMAINSChakravarthy BhagvatiDept. of Computer & Information SciencesUniversity of Hyderabad22 August 2009ELECTRONICS AND COMMUNICATIONS ENGINEERING, OSMANIA UNIVERSITY

• Colour Science◦ The Physics Perspective◦ Human Vision System◦ Digital Colour• The Physical Aspects of the Human Vision System◦ Colour Matching Experiments◦ XYZ Tristimulus Space◦ x-y Chromaticity Diagram• Digital Colour◦ Trichromatic Colour Models and Colour Spaces• Why is Colour Image Processing only in its Infancy?• ConclusionsOVERVIEW

• Colour is electromagnetic radiation within a specific range of wavelengths(380nm < λ < 780nm)• Colour is a Spectral Power Distribution (SPD)THE PHYSICS OF COLOUR

Trichromatic Theory• Colour is the response to the three stimuli corresponding to the threetypes of cones: R for L, G for M and B for S∫+∞C i = R i (λ)P(λ)dλ−∞R i = S, M and LOpponent Colour Theory• Every colour shade is a linear combination of a colour and its opponent/complementarycolour• Hering (c. 1886) proposed red–green and blue–yellow pairs as primaryopponent colours◦ there are no shades like bluish yellow or greenish redCOLOUR THEORIES

• Users asked to match monochromaticcolour (reference) by combininglight from red, green andblue lamps◦ primaries with known SPDs◦ adjustable knobs to control intensities• Reference colours varied in intervalsof 2 nm and 5 nm from 400nm to 700 nm• The knob settings were plottedagainst wavelength to obtaincolour matching functions x, yand zCOLOUR MATCHING EXPERIMENT

• Committee Internationale De L’Eclairage (CIE) proposed the XYZ TristimulusSpace based on the colour matching experiments as a model ofhuman colour perception• XYZ values are obtained from the colour matching functions (x, y, z)asX =∫780nmR(λ)x(λ) dλ Y =∫780nmR(λ)y(λ) dλ380nmZ =∫780nm380nmR(λ)z(λ) dλ380nmwhere R(λ) is the SPD of the object that is imagedXYZ TRISTIMULUS SPACE

• XYZ Tristimulus values are normalizedto give the xyz trichromaticityspace• A plot of x vs. y gives the famousx-y chromaticity diagram(or tongue diagram)x =y =z =XX + Y + ZYX + Y + ZZX + Y + Z• Only two coordinates are necessaryfor describing colourTHE (IN)FAMOUS TOUNGUE!

• The entire set of colours that canbe perceived or handled by a sensor/deviceis called its gamut• The tongue is the gamut of thehuman vision system• Although we use a trichromaticsystem, our human vision systemis non-linear and generatesa large gamut• Any linear trichromatic space resultsin a triangular gamut• How many colours can we see?COLOUR GAMUTS

• Define three standard primaries— R, G and B• Cartesian Coordinate system:Dark Biscuit is (192,128,0)• Most popular because of hardwaresupport• Not a very good candidate forcolour image processing◦ perceptually non-uniform◦ non-descriptive◦ does not decouple achromaticand chromatic aspectsRGB COLOUR SPACE (at last!)

• Hue and Saturation describecolour• Value is the achromatic component• Hue is the dominant colour (circularaxis): red is 0 ◦ , green is120 ◦ , blue is 240 ◦• Saturation is purity of colour (radialaxis): red (0 ◦ , 1.0) vs. Pink(0 ◦ , 0.60)• HSV colour space is either cylindricalor a double cone• Description-oriented• Non-Euclidean space• Perceptually non-uniform (butbetter than RGB)HSV COLOUR SPACE

Equal distance between coloursdoes not mean equally differentcolours(e.g. red-green vs. blue-green)MacAdam Ellipses (1942)NON-UNIFORM PERCEPTION

• Perceptually uniform space createdby CIE• Correct name is L ∗ a ∗ b ∗• Lightness (L ∗ ) is achromatic• a ∗ and b ∗ are opponent colourpairsL ∗ = 116a ∗ = 500b ∗ = 200( YY 0)13− 16[ ( )1 ( ])1X3 Y3−X 0 Y 0[ (Y )1 ( ])13 Z3−Y 0 Z 0CIELAB SPACE

• Major tasks associated with colour are specification, description andmeasurement• Colour specification models require completeness, uniqueness and resolution◦ XYZ and RGB allow specification but do not satisfy completeness• Colour description models need to be human-oriented and should separatechromatic and achromatic aspects◦ HSV and variants• Colour measurement requires linearity and existence of a simple distancemeasure◦ CIELAB is the best exampleLet’s turn to colour image processing now . . .COLOUR MODELS

• Basic operations◦ Threshold◦ Ranging◦ Negative• Enhancement◦ Contrast◦ Gamma◦ Histogram-based• Spatial and Frequency Domain◦ Noise-removal◦ Edge detection◦ Texture analysis• Feature Extraction• Basic operations◦ ??◦ Colour filters◦ Colour negative• Enhancement◦ Colour space dependent◦ Gamma (3 times?)◦ No histograms• Spatial and Frequency Domain◦ Scalar or Vector◦ No frequency domain ∗• Quantization and half-toningIMAGE PROCESSING

• What do you mean by thresholding?◦ same as for grayscale but 3 times?• Or, is it to separate coloured and non-coloured?Original RGB-Threshold Colour Threshold• How did we do the colour threshold?◦ In HSV space - threshold is on saturation (25%)COLOUR THRESHOLDING

• Pick up red coloured regions from the image• It is surprisingly difficult in RGB space◦ How do you specify red?• It is almost trivial in HSV space◦ Specify it as H = 0 ◦ ± ǫOriginal RGB-Ranging HSV RangingCOLOUR RANGING

YET ANOTHER EXAMPLE!

• Standard approach: invert each component in RGB space• Two alternatives◦ In HSV, replace hue with its complementary hue◦ In HSV, replace hue and saturation with their inversesOriginal RGB Neg Hue Neg Hue-Sat NegIn this case, the RGB approach appears betterCOLOUR NEGATIVES

• Histogram equalization is often used to enhance contrast• Modify the histogram of the image such that it becomes as uniform aspossible• For a colour image, apply it three times• A better alternative: apply only to V component in HSV spaceOriginal RGB Equalization V EqualizationCONTRAST ENHANCEMENT

• For image enhancement, it is better to use a colour space that separateschromatic and achromatic (intensity) components• We saw examples from HSV space but similar results are obtained fromCIELAB space• There are many unanswered questions as yet◦ enhancing faded colours (saturation in HSV?)◦ creating cartoon type images from photos◦ eliminating effects of illuminants◦ colour filtersSUMMARY

• Using masks and convolution —normally done on individual components• Loss of correlation betweencolour planesVector BasedApproachesNewer algorithms use Vector-based approaches• colour is a three-dimensional vector• image is a vector valued function of two variablesI(x, y) = C• image processing → meaningful mathematical operations on the vectorvaluedfunction I(x, y)SPATIAL PROCESSING

Vector MedianFilterTake a 3 × 3 nbrhood, N, of p.Let c i = (r i , g i , b i ), i ∈ N.Define c = ∑ i∈N c i/|N|Compute d i = ||(c i − c)|| 2 , i ∈ NSort c i according to d i in ascendingorderReplace c center with the first c i inthe sorted listOriginalVMF OutputNOISE REMOVAL

• New developments are vector-based methods• Edge detection is a derivative operation• Compute colour derivatives using partial and total derivatives concepts• Vector versions of Canny have been developed• Cumani edge detector is a new vector based method that improves uponCannyOriginal Vector Edges Grayscale EdgesEDGE DETECTION

• Horizontal and Vertical directional gradient operators are• Let us define the followingu = ∂R∂xv = ∂R∂yg xx = u · u =g yy = v · v =g xy =∂G ∂B⃗r + ⃗g +∂x ∂x ⃗ b∂G ∂B⃗r + ⃗g +∂y ∂y ⃗ b∂R2∣∂x∣+∂G∣∂x∣∂R2∣ ∂y ∣ +∂G∣ ∂y ∣∂R ∂R∂x ∂y + ∂G∂x2+∂B∣ ∂x ∣2+∂B∣ ∂y ∣∂G∂y + ∂B∂x22∂B∂yVECTOR DERIVATIVES

From the derivatives defined in the previous slide, the maximum rate ofchange of f and the direction of the maximum contrast are2g xyθ = 1 2 arctan g xx − g yyF(θ) = 1 2 {(g xx + g xy ) + cos 2θ(g xx − g yy ) + 2g xy sinθ}The edges are obtained by thresholding F(θ)CANNY EDGE DETECTOR(Vector Version)&CUMANI EDGEDETECTOR• have two thresholds on F(θ)• do non-max suppression and edgelinkingOR• calculate eigenvectors of F(θ) forextreme values• find zeros of derivativeVECTOR EDGE DETECTION

Original Cumani Edges Canny Edges• Canny misses some edges but the overall image is clean• Cumani is more sensitive but also picks up some noiseEXAMPLE

CAN’T DO WITHOUT LENA!

• Frequency domain operations did not really exist until recently for colourimages◦ FFT may be computed on each component but how to combine• Recently, Sangwine and Ell proposed a novel approach based on representingcolour as a quaternionq = k + r⃗i + g⃗j + b ⃗ k• These are known as hypercomplex numbers and FT can be computedfor quaternions analogous to the complex FTF(u, v) = SM−1∑m=0N−1∑n=0e −j2πmu M f(m, n)e−k2π nvNFREQUENCY DOMAIN OPERATIONS

OriginalOriginalLow-Pass w 0 = 35%High-Pass w 0 = 10%• Images are analogous to thoseobtained for grayscale FTLow-Pass w 0 = 10%QFT OPERATIONS

There are certain things that are valid only on colour images• polarization• colour filters• spectral estimationOriginal 3 Colours 6 ColoursSPECIAL COLOUR OPERATIONS

• We compute a wavelength corresponding to an (R,G,B) colour tripletthrough (X,Y,Z) tristimulus space and use it for filtering• Example shows an image with pixels having colours within the band of620 ± 40 nmSPECTRAL OPERATION

• Colour image processing is very popular today, but with many openquestions and issues◦ analysis of colour spaces and models◦ scalar vs. vector processing, quaternions and other representations◦ image processing operations and applications• Not yet addressed — a physical basis◦ colour is a very physical phenomenon but processing is done entirely on the digitaldomain◦ links between digital operations and physical processes tenuous at best◦ a great need to link digital operations to physical processes that manipulate colourin nature◦ applications will be revolutionary• At University of Hyderabad, we have now a good team working on colourimage processing and producing results (whose value only time will tell)CONCLUSION