Slides

I know what you have done 

in your last 

HOTO HOP Session 

An Introduction to Digital Image Forensics 

Thomas Gloe 

Matthias Kirchner * 

Faculty of 

Computer Science 

TU Dresden 

ICSI Berkeley 

21|01|2010

This is actually not from a horror movie . . . 

◮ We have some clues that an alien species has begun to infiltrate (wo)mankind 

watch out for: 

⊲ two left feet 

Gloe & Kirchner Digital Image Forensics slide 1





⊲ missing belly button 







⊲ overly long legs 







⊲ overly long legs 

⊲ “ghost” shadows 

more examples can be found at 

http://photoshopdisasters.blogspot.com 


. . . yet it is as scary, at least. 

◮ more serious harm of image 

manipulations in areas like: 

⊲ political debates 

⊲ science 

⊲ evidence in court 

⊲ . . . 

former German Foreign Minister Steinmeier 

submission to the 2009 DOCMA award (Matthias Kleemann)





⊲ science 


⊲ . . . 

Sciene 27 July 2007, Vol. 317, no. 5837, p. 450





⊲ science 


⊲ . . .





⊲ science 


⊲ . . . 

lots of interesting examples can 

be found at Hany Farid’s website 

www.cs.dartmouth.edu/farid/ 

research/digitaltampering 

submission to the 2009 DOCMA award (Christine Gerhardt)

“Edit and share from anywhere”

Authenticity of images in the digital age? 

◮ Where is the picture coming from? 

Approaches 

◮ (How) Has the image been processed? 

◮ cryptography or digital watermarks 

⊲ marks needs to be added / embedded “actively” (high-end digital cameras) 


Authenticity of images in the digital age? 

◮ Where is the picture coming from? 

Approaches 

◮ (How) Has the image been processed? 

◮ cryptography or digital watermarks 

⊲ marks needs to be added / embedded “actively” (high-end digital cameras) 

◮ digital image forensics 

⊲ does not assume any knowledge about the original image 

⊲ is based on a statistical analysis of the image content 

N (µ, σ) 

Σ → min 

statistical analysis 

source? 

original? 


Digital image forensics: a first glimpse 


assess the authenticity 

of digital images by exploiting 

their inherent statistical 

characteristics 

source identification 

manipulation detection 








class 

‘legitimate’ 

model 

What means source? 



device 

What means manipulation? 

‘malicious’ 



images represent reality 

⊲ empirical science 






class 


model 




device 






⊲ empirical science 






requires a large body of 

reference data samples 

class 


model 




device 






1 ⊲ empirical science 2 

3 






requires a large body of 

reference data samples 

class 


model 




device 




Interesting. Tell me more! 

Image forensics may exploit 

◮ artifacts of processing operations 

⊲ resampling · copy & paste · inconsistent lightning · double compression 

◮ characteristics of the source device 

scene 

⊲ e. g. digital camera: 

lens 

lens 

distortion 

filter 

R 

G 

G 

B 

CFA layout 

sensor 

hot pixels, 

sensor noise 

◮ metadata (has to be handled with care though) 

⊲ EXIF header · thumbnails 

black box model, camera response function 

color 

interpolation 

interpolation 

scheme 

post 

processing 

quantization 

table 

digital image 


Two introductory examples 

◮ digital camera identification 

based on sensor noise 





? 





? 





≈ 





≈ 

◮ copy & paste detection 





≈ 

◮ copy & paste detection 

presumed original 


1 

After all, it’s all about bits and 

bytes, like in computer forensics. 

Is digital image forensics 

really a science on its own?

1 

After all, it’s all about bits and 

bytes, like in computer forensics. 

Is digital image forensics 

really a science on its own? 

digital image forensics = computer forensics

By the way, what is computer forensics?

By the way, what is computer forensics? 

1 

1 

1100 0 0 

0 

1 

0 

1


52 51 51 51 49 

49 40 36 34 33 

55 48 40 33 23 

62 58 45 33 22 

66 62 53 34 22 

0 

0 

0 

1000 1 1 

0 

0 

0


52 51 51 51 49 

49 40 36 34 33 

55 48 40 33 23 

62 58 45 33 22 

66 62 53 34 22 

1 

1 

1 

0010 1 1 

0 

0 

0

Digital forensics: proposed ontology 

forensics 

1 1 1 1 0 1 0 0 0 0 0 

0 

1 

0 

0 

0 

0 

1 0 0 0 0 

digital 

0 0 

forensics 

1 0 0 

1 

1 

0 

0 

1 

0 

analog forensics 

1 1 0 1 0 0 0 0 0 1 1 

1 

1 

1 

1 

0 0 1 

computer 1 1 1 

forensics 1 1 0 

0 1 1 

1 

0 

0 

1 

1 

0 

0 

1 

1 multimedia 0 1 1 1 

0 1(image) 0 1 0 

0 forensics 0 1 1 1 

1 1 1 1 1 

1 1 1 digital 0 0 evidence 1 1 1 0 0 0 physical evidence 

1 1 0 1 0 1 1 1 1 1 1 



forensics 

0 0 0 0 1 0 1 0 0 0 0 

0 

0 

0 

1 

1 

0 

0 1 1 1 0 

digital 

0 1 

forensics 

0 0 1 

0 

1 

0 

0 

1 

1 


0 0 1 1 1 1 0 1 0 1 0 

0 

0 

1 

1 

1 1 0 



0 0 1 

1 

0 

0 

1 

0 

1 

0 

1 


1 1(image) 0 1 1 


0 1 0 1 1 


1 1 0 0 1 1 1 0 0 1 1 

finite sequence of discrete and 

perfectly observable symbols 


WARNING! 

The following slides 

intentionally draw a very 

black-and-white 

picture

Computer forensics = Image forensics 

computer forensics image forensics 

physical evidence 

digital evidence 

1 0 0 1 

1 1 0 1 

WWW 



1 0 0 1 

1 1 0 1 






1 0 0 1 

1 1 0 1 

WWW 

WWW 



1 0 0 1 

1 1 0 1 






1 0 0 1 

1 1 0 1 

WWW 

◮ digital evidence is not linked 

to the outside world 

WWW 



1 0 0 1 

1 1 0 1 






1 0 0 1 

1 1 0 1 

WWW 



WWW 



1 0 0 1 

1 1 0 1 






1 0 0 1 

1 1 0 1 

WWW 



WWW 



1 0 0 1 

1 1 0 1 

◮ digital evidence is linked 



Computer forensics: a closer look 

processing 

digital 

data 


suspicious 

traces? 


processing 

digital 

data 


suspicious 

traces? 


processing 

digital 

data 

reality 

◮ digital evidence is stored in the 

finite automaton each computer 

represents 

◮ number of states in a closed 

system is finite 


suspicious 

traces? 


processing 

digital 

data 

reality 



represents 




suspicious 

traces? 


processing 

digital 

data 

reality 



represents 



◮ non-negligible chance that a 

computer is left in a state which 

perfectly erases all traces 


Image forensics: a closer look 

processing 

digital 

image 



original? 

source 

(device) ? 

processing 

digital 

image 



original? 

source 

(device) ? 

processing 

digital 

image 

sensor 

◮ sensors capture parts of the reality and 

transform them into digital representations 



original? 

source 

(device) ? 

processing 

digital 

image 

sensor 



◮ reality is incognizable: ultimate knowledge 

whether a digital image reflects reality or 

not cannot exist 



original? 

source 

(device) ? 

processing 

digital 

image 

sensor 



◮ reality is incognizable: ultimate knowledge 

whether a digital image reflects reality or 

not cannot exist 

◮ image forensics = empirical science 


Sensors: a source of uncertainty 

◮ projection of reality to discrete symbols means a dimensionality reduction 




◮ image forensics has to cope with an additional source of uncertainty 

degrees of freedom 




◮ image forensics has to cope with an additional source of uncertainty 

◮ what kind of common 

post-processing is 

legitimate / tolerable? 

Gloe & Kirchner Digital Image Forensics slide 15 

?

Models: yet another dimensionality reduction 

◮ models make projection of reality to 

discrete symbols tractable with formal 

methods 

◮ typical models in image forensics: 

⊲ sensor noise follows a Gaussian distribution 

⊲ connected regions of identical pixel values are 

unlikely to occur in original images 





methods 






p




methods 





projection to a 

1-dimensional 

variable 

◮ models of reality are yet another dimensionality reduction 

◮ quality of forensic methods depends on the quality of the employed model! 


p




methods 





projection to a 

1-dimensional 

variable 

◮ models of reality are yet another dimensionality reduction 

◮ quality of forensic methods depends on the quality of the employed model! 


p

2 

Now, after all the theory, what 

are you really doing all day long? 

A practical view on digital image forensics

Fundamentals of digital image forensics 

real scene 

analog 2D 

document 



real scene 

analog 2D 

document 

digitalization 



real scene 

analog 2D 

document 


digital image 



real scene 

analog 2D 

document 


digital image 

manipulation 



real scene 

analog 2D 

document 


digital image 

manipulation 

output 



real scene 

analog 2D 

document 


digital image 

manipulation 

output 

analog image 



real scene 

analog 2D 

document 


digital image 

manipulation 

output 

re-digitalization 

analog image 



real scene 

lightning, shadows 

analog 2D 

document 


digital image 

manipulation 

output 


analog image 



real scene 


analog 2D 

document 

surface characteristics 


digital image 

manipulation 

output 


analog image 



real scene 


analog 2D 

document 



device 


digital image 

manipulation 

output 


analog image 



real scene 


analog 2D 

document 



device 


digital image 

file format 

manipulation 

output 


analog image 



real scene 


analog 2D 

document 



device 


digital image 

file format 

manipulation 

processing 

artifacts 

output 


analog image 



real scene 


analog 2D 

document 



device 


digital image 

file format 

manipulation 

processing 

artifacts 

output 

device 



analog image 



synthetic 3D scene 

real scene 


analog 2D 

document 


texture 


device 


digital image 

file format 

manipulation 

processing 

artifacts 

output 

device 



analog image 




real scene 


analog 2D 

document 


texture 


device 


rendering 

digital image 

file format 

manipulation 

processing 

artifacts 

output 

device 



analog image 




real scene 


analog 2D 

document 


texture 


device 


rendering 

missing device 


digital image 

file format 

manipulation 

processing 

artifacts 

output 

device 



analog image 




real scene 


analog 2D 

document 


texture 


device 


rendering 

missing device 


digital image 

file format 

manipulation 

processing 

artifacts 

output 

device 



⊲ too complex to be 

studied as a whole 

⊲ focus on small, 

tractable problems 

analog image 


Image source identification 

goal: determine either the correct class, the correct model / manufacturer, 

or even the actual device that took a particular image 


?





?





?

Intra- and inter-model similarities 

◮ device identification – examined characteristic is unique for each device 

⊲ CCD / CMOS sensor noise [Lukáˇs et al., 2005] 

inter-model 

intra-model similarity intra-model similarity 

similarity 

low for device identification 

high for model identificatio 

low similarity 


high for model identfication 


Intra- and inter-model similarities 

◮ device identification – examined characteristic is unique for each device 

⊲ CCD / CMOS sensor noise [Lukáˇs et al., 2005] 

◮ model identification – examined characteristic is similar for all devices of one 

particular model and differs between different models 

inter-model 

intra-model similarity intra-model similarity 

similarity 


high for model identification 

low similarity 


high for model identification 


Camera model identification 

◮ most probably, no single feature can distinguish between different camera models 

⊲ find a set of features with desirable properties 

Black box model [Kharrazi et al., 2004; Gloe et al., 2009] 

◮ overall 34 features to characterize images from the same camera model 

⊲ 12 color features to capture particularities of for instance white balancing 

⊲ 9 wavelet domain noise features 

⊲ 13 additional ‘image quality metrics’ for a general purpose description 

scene 

lens 

image quality 

metrics 

filter 

R 

G 

G 

B 

sensor 

image quality metrics, 

wavelet statistics 

color 

interpolation 

post 

processing 

color features color features 

digital image 


Camera model identification 

◮ most probably, no single feature can distinguish between different camera models 

⊲ find a set of features with desirable properties 

Black box model [Kharrazi et al., 2004; Gloe et al., 2009] 

◮ overall 34 features to characterize images from the same camera model 

⊲ 12 color features to capture particularities of for instance white balancing 

⊲ 9 wavelet domain noise features 

⊲ 13 additional ‘image quality metrics’ for a general purpose description 

scene 

lens 

image quality 

metrics 

filter 

R 

G 

G 

B 

sensor 

image quality metrics, 

wavelet statistics 

color 

interpolation 

◮ machine learning algorithms (SVM) for classification 

post 

processing 

color features color features 

digital image 


Some results on camera model identification 

◮ large-scale test with ≈ 15 000 images, stemming from 25 different models and 

overall 73 devices [Gloe et al., 2009]

Camera model identification: overall results 

◮ test set contains at least two devices per model 

◮ 60 % of the images for training the classifier, 40 % for testing 

◮ classification accuracy over all models: 96.42 % 

identified as 

camera model S DT DS DSs DC FZ H T W E F G I L M N O R µ 

Coolpix S710 S 100,00 - - - - - - - - - - - - - - - - - - 

D200 DT - 93,73 - - - 0,52 0,10 - 0,05 2,77 - - 2,61 - - - 0,05 - 0,16 

D70 DS - - 84,57 15,43 - - - - - - - - - - - - - - - 

D70s DSs - 0,11 67,51 30,90 - - - - - 0,85 - - 0,63 - - - - - - 

DCZ 5.9 DC - - 0,72 0,64 98,42 - - - - 0,12 - - - - - - - - 0,10 

DMC-FZ50 FZ - 1,14 - - - 97,31 0,50 0,58 0,32 - 0,06 - - - - 0,09 - - - 

DSC-H50 H - - - - - 0,20 98,23 0,20 1,28 - - 0,10 - - - - - - - 

DSC-T77 T - 0,74 - - - 3,43 2,04 91,90 1,81 - - 0,09 - - - - - - - 

DSC-W170 W - 0,65 - - - - 0,52 0,26 98,06 - - 0,52 - - - - - - - 

EX-Z150 E - - 0,19 0,08 - - - - - 99,71 - - - 0,03 - - - - - 

FinePixJ50 F - - - - - 0,06 - - - - 99,94 - - - - - - - - 

GX100 G - - - - - - - - - - - 100,00 - - - - - - - 

Ixus70 I - - 0,79 0,47 - - - - - - - - 98,26 0,42 - - - - 0,05 

L74wide L - - 0,04 0,04 - - - - - - - - 1,25 98,66 - - - - - 

M1063 M 0,07 0,67 - 0,01 - 0,01 - - - 0,04 - - 0,01 - 99,19 - - - - 

NV15 N 0,47 0,47 - - - 4,07 - - 0,05 - - - 0,05 - - 92,85 0,09 - 1,96 

OptioA40 O 3,97 0,13 0,09 - - 0,17 - 0,13 - 0,52 - 0,22 0,17 - 0,09 2,28 90,86 - 1,38 

RCP-7325XS R - - - - 0,21 - - - - - - - - - - - - 99,79 - 

µ1050SW µ - 0,29 - - - 0,12 - - - - - - 0,14 0,10 - 0,38 0,02 - 98,95 






◮ mis-classifications between almost equal models of the same manufacturer 

identified as 


Coolpix S710 S 100,00 - - - - - - - - - - - - - - - - - - 

D200 DT - 93,73 - - - 0,52 0,10 - 0,05 2,77 - - 2,61 - - - 0,05 - 0,16 

D70 DS - - 84,57 15,43 - - - - - - - - - - - - - - - 

D70s DSs - 0,11 67,51 30,90 - - - - - 0,85 - - 0,63 - - - - - - 

DCZ 5.9 DC - - 0,72 0,64 98,42 - - - - 0,12 - - - - - - - - 0,10 

DMC-FZ50 FZ - 1,14 - - - 97,31 0,50 0,58 0,32 - 0,06 - - - - 0,09 - - - 

DSC-H50 H - - - - - 0,20 98,23 0,20 1,28 - - 0,10 - - - - - - - 

DSC-T77 T - 0,74 - - - 3,43 2,04 91,90 1,81 - - 0,09 - - - - - - - 

DSC-W170 W - 0,65 - - - - 0,52 0,26 98,06 - - 0,52 - - - - - - - 

EX-Z150 E - - 0,19 0,08 - - - - - 99,71 - - - 0,03 - - - - - 

FinePixJ50 F - - - - - 0,06 - - - - 99,94 - - - - - - - - 

GX100 G - - - - - - - - - - - 100,00 - - - - - - - 

Ixus70 I - - 0,79 0,47 - - - - - - - - 98,26 0,42 - - - - 0,05 

L74wide L - - 0,04 0,04 - - - - - - - - 1,25 98,66 - - - - - 

M1063 M 0,07 0,67 - 0,01 - 0,01 - - - 0,04 - - 0,01 - 99,19 - - - - 

NV15 N 0,47 0,47 - - - 4,07 - - 0,05 - - - 0,05 - - 92,85 0,09 - 1,96 

OptioA40 O 3,97 0,13 0,09 - - 0,17 - 0,13 - 0,52 - 0,22 0,17 - 0,09 2,28 90,86 - 1,38 

RCP-7325XS R - - - - 0,21 - - - - - - - - - - - - 99,79 - 

µ1050SW µ - 0,29 - - - 0,12 - - - - - - 0,14 0,10 - 0,38 0,02 - 98,95 






◮ mis-classifications between almost equal models of the same manufacturer 

identified as 


Coolpix S710 S 100,00 - - - - - - - - - - - - - - - - - - 

D200 DT - 93,73 - - - 0,52 0,10 - 0,05 2,77 - - 2,61 - - - 0,05 - 0,16 

D70 DS - - 84,57 15,43 - - - - - - - - - - - - - - - 

D70s DSs - 0,11 67,51 30,90 - - - - - 0,85 - - 0,63 - - - - - - 

DCZ 5.9 DC - - 0,72 0,64 98,42 - - - - 0,12 - - - - - - - - 0,10 

DMC-FZ50 FZ - 1,14 - - - 97,31 0,50 0,58 0,32 - 0,06 - - - - 0,09 - - - 

DSC-H50 H - - - - - 0,20 98,23 0,20 1,28 - - 0,10 - - - - - - - 

DSC-T77 T - 0,74 - - - 3,43 2,04 91,90 1,81 - - 0,09 - - - - - - - 

DSC-W170 W - 0,65 - - - - 0,52 0,26 98,06 - - 0,52 - - - - - - - 

EX-Z150 E - - 0,19 0,08 - - - - - 99,71 - - - 0,03 - - - - - 

FinePixJ50 F - - - - - 0,06 - - - - 99,94 - - - - - - - - 

GX100 G - - - - - - - - - - - 100,00 - - - - - - - 

Ixus70 I - - 0,79 0,47 - - - - - - - - 98,26 0,42 - - - - 0,05 

L74wide L - - 0,04 0,04 - - - - - - - - 1,25 98,66 - - - - - 

M1063 M 0,07 0,67 - 0,01 - 0,01 - - - 0,04 - - 0,01 - 99,19 - - - - 

NV15 N 0,47 0,47 - - - 4,07 - - 0,05 - - - 0,05 - - 92,85 0,09 - 1,96 

OptioA40 O 3,97 0,13 0,09 - - 0,17 - 0,13 - 0,52 - 0,22 0,17 - 0,09 2,28 90,86 - 1,38 

RCP-7325XS R - - - - 0,21 - - - - - - - - - - - - 99,79 - 

µ1050SW µ - 0,29 - - - 0,12 - - - - - - 0,14 0,10 - 0,38 0,02 - 98,95 

⊲ What happens if we have to consider substantially more camera models or 

post-processing of the images? 


Camera device identification 

◮ most probably, one single feature can distinguish between specific camera devices 

Camera ID from sensor noise [Lukáˇs et al., 2005, 2006] 

◮ CCD / CMOS sensor elements convert light into a digital signal, a process subject to 

various noise sources 

⊲ temporal noise: varies throughout all images from a camera 

⊲ spatial noise: similar for all images of the very same camera, 

but different between distinct devices 










fixed pattern 

noise (FPN) 

⊲ additive noise, mainly due 

to dark noise 

⊲ flattened inside camera 

photo-response 

non-uniformity (PRNU) 

⊲ multiplicative noise, due to varying 

light-sensitivity of sensor elements 

⊲ hard to correct in cameras 










fixed pattern 

noise (FPN) 

⊲ additive noise, mainly due 

to dark noise 

⊲ flattened inside camera 

photo-response 

non-uniformity (PRNU) 

⊲ multiplicative noise, due to varying 

light-sensitivity of sensor elements 

⊲ hard to correct in cameras 

camera 

fingerprint 


How to obtain a digital camera fingerprint 

◮ camera-specific PRNU fingerprint is approximated by a reference noise pattern 

⊲ take at least 50 images from the same camera (ideally: perfectly lit, bright scenes) 

⊲ apply a de-noising filter to estimate the noise term in in each of these images 

⊲ take the pixel-wise average of the noise residuals 

digital camera stack of images noise residuals averaged noise residuals 

(reference noise pattern) 

◮ for practical applications, it is recommended to use a more sophisticated maximum 

likelihood estimation procedure [Chen et al., 2007, 2008] 


A typical noise residual Wavelet de-noising filter [Miçhak et al., 1999] 

◮ de-noising filters are not perfect ⊲ residual retains some image content 

Faculty of Computer Science, TU Dresden noise residual 


Reference noise pattern of a Canon S70 

noise pattern averaged over 300 images 

◮ reference noise pattern is free of 

image content 

◮ also contains information about the 

camera model (CFA artifacts, . . . ) 

[Filler et al., 2008] 

enlarged detail with pixel defects 


correlation coefficient 

Was it your camera that took this image? 

◮ cameras are identified by measuring the similarity of the noise residual of a 

questionable image and all known reference noise pattern 

⊲ correlation coefficient or (more preferably) peak to correlation energy 

0.3 

0.2 

0.1 

0 

reference pattern: S70 

Canon S70 

Canon S45 

Epson 1240U 

Ixus IIs 

1 image index 

350 

Closed camera set problem: 

◮ choose the camera with the highest 

similarity 

Open camera set problem: 

◮ no decision if maximum similarity is 

below a certain threshold 


correlation coefficient 

Was it your camera that took this image? 

◮ cameras are identified by measuring the similarity of the noise residual of a 

questionable image and all known reference noise pattern 

⊲ correlation coefficient or (more preferably) peak to correlation energy 

0.3 

0.2 

0.1 

0 

reference pattern: S70 

Canon S70 

Canon S45 

Epson 1240U 

Ixus IIs 

1 image index 

350 

Closed camera set problem: 

◮ choose the camera with the highest 

similarity 

Open camera set problem: 

◮ no decision if maximum similarity is 

below a certain threshold 

◮ large-scale test with more than 10 6 images from ≈ 7000 individual cameras: 

97.6 % correct identification at a false acceptance rate of < 3 × 10 −5 [Goljan et al., 2009] 


Image source identfication: Summary 

◮ discussed methods are only the tip of the iceberg 

◮ further approaches include: 

⊲ sensor dust [Dirik et al., 2007] 

⊲ JPEG quantization tables [Farid, 2008] 

⊲ chromatic abberation [Van et al., 2007] 

⊲ demosaicing artifacts [Swaminathan et al, 2007] 

⊲ and many more . . . 

◮ current literature concludes that sensor noise is the method of choice whenever we 

have access to the questionable device (or a number of reference images) 

⊲ extensions to flatbed scanners [Gloe et al., 2007] 

⊲ robust against JPEG compression; may even survive printing and re-scanning 

⊲ but: computational issues with de-synchronization (cropping, rotation) 

◮ device model identification beneficial whenever we don’t have direct access to the 

questionable device 

⊲ but: does it scale with the number of examined models? 


Need a break? 

submission to the 2009 DOCMA award (Bernd Busche)

Need a break? Ooops. . . 

submission to the 2009 DOCMA award (Bernd Busche)

Image manipulation detection 

Digital image forensics checks for 

◮ missing or inconsistent device characteristics 

◮ presence of processing artifacts 

But what actually constitutes a manipulation?





But what actually constitutes a manipulation? 

‘malicious’ post-processing ‘legitimate’ post-processing






‘malicious’ post-processing ‘legitimate’ post-processing 

⊲ (mostly) local changes 

⊲ splicing 

⊲ copy & paste 

⊲ . . . 

⊲ content-preserving global changes 

⊲ denoising 

⊲ compression 

⊲ contrast enhancement 

definitions depend on established habits and conventions






‘malicious’ post-processing ‘legitimate’ post-processing







pictures: Andrea Sommer, Doc Baumann






research focuses on the 

detection of basic image 

processing primitives 


pictures: Andrea Sommer, Doc Baumann

Resampling detection 

◮ image manipulations often rely on geometric transformations 

(scaling, rotation, . . . ) of images or parts thereof 


Andrea Sommer, Doc Baumann

Resampling detection 

◮ image manipulations often rely on geometric transformations 

(scaling, rotation, . . . ) of images or parts thereof 

◮ resampling to a new image grid; involves an interpolation step 

◮ interpolation introduces periodic linear correlations between neighboring pixels 

s1,1 s1,2 

s2,1 s2,2 

bilinear upsampling 

(×2) 

0.5 

0.5 

0.5 0.5 

s1,1 s1,2 s1,3 

s2,1 s2,2 s2,3 

s3,1 s3,2 s3,3 

0.5 0.5 

◮ resampling artifacts can be detected by the analysis of linear predictor residue 

[Popescu & Farid, 2005], [Kirchner, 2008] 


0.5 

0.5

Resampling detection scheme 

t − 2 t − 1 t t + 1 t + 2 



example: linear upsampling (×2) 

s(ωt ′ ) = P 

t h(ωt′ − t)s(t) 

t − 2 t − 1 t t + 1 t + 2 



ωt ′ − ω ωt ′ 


ωt ′ + ω 

t − 2 t − 1 t t + 1 t + 2 

◮ predictor residue: e(ωt ′ ) = s(ωt ′ ) − 

KX 

αks(ωt ′ + ωk) (α0 := 0) 

k=−K 

◮ large absolute prediction errors indicate minor degree of linear dependence 



ωt ′ − ω ωt ′ 


ωt ′ + ω 

t − 2 t − 1 t t + 1 t + 2 


KX 

αks(ωt ′ + ωk) (α0 := 0) 

k=−K 


◮ optimal weights in the example: α = (0.5, 0, 0.5) 

⊲ every second sample defaults to 0 



ωt ′ − ω ωt ′ 


ωt ′ + ω 

t − 2 t − 1 t t + 1 t + 2 


KX 

αks(ωt ′ + ωk) (α0 := 0) 

k=−K 


◮ optimal weights α can be determined with an EM algorithm [Popescu & Farid, 2005] 

or set to a fixed linear filter mask [Kirchner, 2008] 



ωt ′ − ω ωt ′ 


ωt ′ + ω 

t − 2 t − 1 t t + 1 t + 2 


KX 

αks(ωt ′ + ωk) (α0 := 0) 

k=−K 


◮ optimal weights α can be determined with an EM algorithm [Popescu & Farid, 2005] 

or set to a fixed linear filter mask [Kirchner, 2008] 

◮ p-map: p(ωt ′ ) ∝ exp (−σ|e(ωt ′ )| τ ) 

◮ measure for the strength of linear dependence 


Original 

105 % 

120 % 

Typical detection results 

p-map DFT (p-map) 

* each spectrum graph has been individually normalized and processed with a maximum filter 

◮ resampling causes periodic 

pattern in the p-map and 

distinct peaks in the p-map’s 

DFT 

◮ peak position is characteristic for 

resampling parameters 

◮ typical resampling detectors 

employ a frequency domain 

peak detector 


It’s not only the biggest potato that counts 

www.bountyfishing.com 

◮ website awards a nice price to the largest fish caught by registered users 

◮ problem: ph fishing attacks with Photoshop 


It’s not only the biggest potato that counts 

www.bountyfishing.com 

◮ website awards a nice price to the largest fish caught by registered users 

◮ problem: ph fishing attacks with Photoshop 

◮ resampling detector unveils cheating 


Color filter array interpolation 

◮ typical digital cameras use only one CCD / CMOS sensor 

and a color filter array (CFA) to capture full-color images 

◮ missing color information is estimated from surrounding 

genuine elements in the raw image 

GR 

G 

GR 

G 

GR 

RAW image full-color image 

G 

GB 

G 

GB 

G 

GR 

G 

GR 

G 

GR 

G 

GB 

G 

GB 

G 

GR 

G 

GR 

G 

GR 

CFA interpolation / demosaicing 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 


R 

R 

R 

R 

R 

R 

R 

R 

R 

R

Color filter array interpolation 

◮ typical digital cameras use only one CCD / CMOS sensor 

and a color filter array (CFA) to capture full-color images 

◮ missing color information is estimated from surrounding 

genuine elements in the raw image 

GR 

G 

GR 

G 

GR 

RAW image full-color image 

G 

GB 

G 

GB 

G 

GR 

G 

GR 

G 

GR 

G 

GB 

G 

GB 

G 

GR 

G 

GR 

G 

GR 

CFA interpolation / demosaicing 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

R 

demosaiced images 

exhibit specific inter-pixel 

correlation artifacts 


Example application of CFA artifacts [Popescu & Farid, 2005] 

Dresden Palace


Dresden Palace


◮ periodic artifacts in the linear predictor residue (p-map) 

Dresden Palace 

CFA peak 

DFT(p-map)

Double JPEG compression 

Main idea / background 

◮ digital images are often already JPEGs before being further processed 

◮ digital images are often re-saved as JPEGs after processing 

◮ usually, we will see different quantization tables 


Double JPEG compression 

Main idea / background 

◮ digital images are often already JPEGs before being further processed 

◮ digital images are often re-saved as JPEGs after processing 

◮ usually, we will see different quantization tables 

quantization of 

DCT coefficients 

QFpre 

pre-compression post-compression 

quantization 

factor q0 

» – 

c 

cq0 = 

q0 

= 

QFpost 

quantization 

factor q1 

" » – # 

c q0 

c (q0,q = 

1) 

q0 q1 


Double JPEG compression artifacts 

Quantization of DCT coefficients: a toy example 

q 0 

c ∈ R 

q 0cq 0 ∈ Z 

cq 0 = 

" # 

c 


q 0



q 0 

q 1 

c ∈ R 

q 0cq 0 ∈ Z 

q1c (q0 ,q1 ) ∈ Z 

cq 0 = 

" # 

c 

q 0 

c (q0 ,q 1 ) = 

" # 

cq0 


q 1



q 0 

q 1 

c ∈ R 

q 0cq 0 ∈ Z 

q1c (q0 ,q1 ) ∈ Z 

◮ double quantization leads to empty DCT coefficient histogram bins 

◮ periodicity depends on q0 / q1 ab [Popescu & Farid, 2004; Lukáˇs et al., 2003] 

no artifacts for q1 = n q0 

cq 0 = 

" # 

c 

q 0 

c (q0 ,q 1 ) = 

" # 

cq0 


q 1

Church of our Ladies, Dresden 

A typical example . . . 

(1, 1) DCT coefficient histograms 

Q = 90 

-200 0 

200 

Q = (80, 90) 

-200 0 

200 

Q = (95, 90) 

-200 0 

200 


. . . and a practical application 

submission to the 2009 DOCMA award (Julia Enkelmann)

. . . and a practical application 

JPEG ghosts [Farid, 2009] 

submission to the 2009 DOCMA award (Julia Enkelmann)

Image manipulation detection: summary 

◮ discussed methods are only the tip of the iceberg 

◮ further approaches include: 

⊲ copy & paste 

[Lukáˇs et al., 2003; Popescu & Farid, 2004; . . . ] 

⊲ lightning analysis [Jonson & Farid, 2007] 

⊲ blocking artifacts [Li et al., 2009] 

◮ there is no single catch-all detector 

⊲ sensor noise [Lukáˇs et al., 2007] 

⊲ chromatic aberrations [Johnson & Farid, 2006] 

⊲ and many more . . . 

◮ image forensics provides a whole toolbox of different methods that aim at particular 

aspects of image manipulation 

but: semantic analysis of the results remains to be a task for the human investigator 


3Digital image forensics 

requires images. 

A lot of them. 

Test datasets in digital image forensics

How does your algorithm know that this 

image is authentic? 


accompanying authentic submission to the 2009 DOCMA award (Xaver Klaußner)

How does your algorithm know that this 

image is authentic? 

Training, training, training, . . . 

◮ requires a vast amount of test data 

◮ scientifically sound results should rely on publicly available data 

Approaches to obtain test data 

◮ capture the images on your own 

⊲ time-consuming and redundant 

◮ use specific databases (mostly designed for other applications) 

⊲ images may have undesired properties 

◮ download 

⊲ images may have even more undesired properties 

⊲ no knowledge about the processing history available 

⊲ persistence and privacy issues 


Dresden Image Database [Gloe & Böhme, 2010] 

◮ image database, specifically designed for applications in digital image forensics 

◮ altogether more than 14 000 digital camera 

images, stemming from 

⊲ 73 devices 

⊲ 23 camera models 

⊲ 14 major camera manufacturers 

◮ several devices for most model 

◮ full range of consumer, semi-professional and 

professional digital cameras 

◮ main foucs: image source identification 

⊲ body of images allows to study characteristics specific to either manufacturer, 

model, or device 

⊲ images of each scene captured with each device 

◮ applications in manipulation detection [Kirchner & Gloe, 2009] 


How to capture 14 000 images 

. . . 

2 

2 

ICSI 2010 

2A 

3 image forensics 

3 

3A 

4 

4 

ICSI 2010 

4A 

set of cameras 


5 

5A 

6 

6 

ICSI 2010 

6A 


◮ each 2 tripods per indoor / outdoor location to capture 2 scenes 

◮ 3 different focal lengths per scene and device 

. . . and a lot of time 

7 

7A 

8 

8 

ICSI 2010 

8A 


9 

9A . 

. .

How to capture 14 000 images 

. . . 

2 

2 

ICSI 2010 

2A 


3 

3A 

4 

4 

ICSI 2010 

4A 

set of cameras 


5 

5A 

6 

6 

ICSI 2010 

6A 


◮ each 2 tripods per indoor / outdoor location to capture 2 scenes 

◮ 3 different focal lengths per scene and device 

. . . and a lot of time and hot beverages 

7 

7A 

8 

8 

ICSI 2010 

It looks cold? 

8A 


9 

9A . 

. .

min / max temperature [ ◦ C] 

8 

4 

0 

-4 

-8 

-12 

-16 

-20 

It was cold! 

1.1. 11.1. 21.1. 31.1. 10.2. 20.2. 

day 

◮ photo-shooting took place during the 

coldest period of the year 2009 


DOCMA Award 

Background 

◮ creating convincing forgeries is a time-consuming task 

◮ researchers are usually no Photoshop experts 

◮ typical test images do not reflect “real-life conditions” 

How can we learn to detect real forgeries from real 

Photoshop users if we don’t have them in our test 

database?

DOCMA Award 

Background 

◮ creating convincing forgeries is a time-consuming task 

◮ researchers are usually no Photoshop experts 

◮ typical test images do not reflect “real-life conditions” 

How can we learn to detect real forgeries from real 

Photoshop users if we don’t have them in our test 

database? 

DOCMA Award 2009 

◮ competition amongst (semi-)professional Photoshop 

users to create an arbitrary newspaper story, 

supported by a convincing image manipulation 

◮ initiator: Doc Baumann and his DOCMA magazine for 

professional Photoshop users 

◮ images (+ originals) are made available to researchers 

◮ around 100 interesting submissions 

Doc Baumann

DOCMA Award: some examples 

Fake 

Peter Hebler


Original 

Peter Hebler


Fake 

Bernd Busche


Original 

Bernd Busche


Fake 

Peter Wienerroither


Original 

Peter Wienerroither

4Now that we have 

the theory behind, 

the practical algorithms, 

and the test data, 

everything should be fine?

4Now that we have 

the theory behind, 

the practical algorithms, 

and the test data, 

everything should be fine? 

Attacks against digital image forensics

Introducing attacks 

✔ existing forensic schemes seem to work well under laboratory conditions 

✔ usage in practical applications is documented 

✘ What if there was a smart counterfeiter, who knew about our techniques? 

◮ attacks: methods to systematically mislead digital image forensics 

Goals of an attacker 

◮ hide traces of image 

manipulations 

Benefits of studying attacks 

◮ eventually better detectors 

◮ erase evidence about the source of an image 

◮ pretend another source device of an image 

◮ protect the source of an image (e .g. retain anonymity) 



forensics 

0 1 1 1 1 1 0 0 0 0 1 

1 

0 

1 

1 

0 

0 

1 1 0 0 0 

digital 

0 0 

forensics 

1 1 0 

0 

0 

1 

0 

1 

1 


1 0 1 0 0 1 1 0 0 1 1 

0 

1 

1 

0 

0 0 1 



0 1 1 

1 

1 

1 

0 

1 

0 

0 

0 


1 1(image) 1 0 1 


1 0 1 1 1 


0 0 1 0 1 1 0 0 0 1 1 

forgeability 

b= 

counter-forensics 



forensics 

0 1 0 1 0 1 0 1 0 0 1 

0 

1 

0 

1 

1 

1 

0 0 0 0 1 

digital 

0 1 

forensics 

0 1 1 

1 

1 

0 

1 

0 

0 


0 1 1 1 0 0 1 1 0 0 0 

”physical evidence cannot be wrong, 

0 

0 

1 

0 

1 1 1 



0 0 0 

1 

0 

1 

0 

0 

1 

1 

0 


0 1(image) 1 1 0 


1 1 0 0 1 

it cannot perjure itself, 

it cannot be wholly absent” 

[Kirk, 1953] 


1 0 0 1 1 1 1 1 1 1 0 

forgeability 

b= 



Counter-forensics: computer forensics 

leave 

traces 

valid state invalid state 



leave 

traces 

eliminate 

traces 

valid state invalid state valid state 



leave 

traces 

eliminate 

traces 


valid states are perfectly known 

or can be recorded before 



leave 

traces 

preemptively 

avoid traces 

eliminate 

traces 






leave 

traces 

virtualization 

preemptively 

avoid traces 

eliminate 

traces 





Counter-forensics: image forensics 

leave 

traces 

preemptively 

avoid traces 

eliminate 

traces 


invalidity depends on 

the model of reality 

virtualization 




Counter-forensics: image forensics 

leave 

traces 

preemptively 

avoid traces 

eliminate 

traces 


invalidity depends on 

the model of reality 

virtualization is not possible 

valid states are not perfectly known 


and cannot be recorded before 



forensics 

1 0 0 0 1 0 1 1 0 1 1 

1 

0 

0 

0 

0 

0 

1 1 0 0 0 

digital 

1 0 

forensics 

1 1 0 

0 

0 

1 

1 

1 

0 

1 1 1 1 0 1 0 1 1 1 1 

1 

1 

1 

0 

0 1 1 



0 0 0 

1 

1 

1 

0 

1 

1 

1 

1 


1 1(image) 0 0 0 


0 0 1 1 0 

0perfect 1 1 crime 0 1 

1 possible 0 1 0 1 

1 

1 

1 1 1 1 1 

compete for 

the 

1 0 

best 

0 

model 

0 0 

forgeability 

b= 



perfect crime 

impossible 


Attacks: practical considerations 

Who is that guy in the pixel next to you? 

◮ successful attacks against particular forensic algorithms are available 

[Gloe et al., 2007; Kirchner & Böhme, 2008, 2009] 

◮ defeating a whole set of forensic approaches is a much more challenging task 

⊲ attacks may introduce new detectable artifacts 

⊲ attacks may interfere with each other 

⊲ competition for the best model 


Attacks: practical considerations 

Who is that guy in the pixel next to you? 

◮ successful attacks against particular forensic algorithms are available 

[Gloe et al., 2007; Kirchner & Böhme, 2008, 2009] 

◮ defeating a whole set of forensic approaches is a much more challenging task 

⊲ attacks may introduce new detectable artifacts 

⊲ attacks may interfere with each other 

⊲ competition for the best model 

Attacks in terms of plausible 

post-processing 

◮ in practice, plausible image statistics 

might be enough 

⊲ downscaling and / or lossy compression 

are likely to smooth out and destroy most 

of the subtle traces we are looking for 


fin 

Concluding remarks

Summary 

◮ image forensics is a science to assess the authenticity of digital images 

image forensics is about statistical analysis of images 

⊲ main ingredients: device characteristics and manipulation artifacts 

⊲ there exists a variety of different approaches to determine the source of an image 

or to detect manipulations 

image forensics is an empirical science 

⊲ the better our model of reality, the more we can trust our tools 

image forensics requires rigor testing on large data sets 

⊲ since we cannot model reality entirely, we need as much observations as possible 


Multimedia forensics: a growing field 

◮ publications on multimedia forensics per year 

source: Hany Farid 

Digital Forensic Database 

8 

before 

4 

2003 

12 

2004 

21 

2005 

36 

2006 


65 

2007 

71 

2008 

87 

2009

Multimedia forensics: a growing field 

◮ publications on multimedia forensics per year 

source: Hany Farid 

Digital Forensic Database 

first publication 

from Dresden 

8 

before 

4 

2003 

12 

2004 

21 

2005 

36 

2006 


65 

2007 

71 

2008 

87 

2009

A broader view 

2 

2 

7 

7 

ICSI 2010 

2A 

ICSI 2010 

7A 

3 

3 

8 

8 

ICSI 2010 

3A 

ICSI 2010 

8A 

4 

4 

9 

9 

ICSI 2010 

4A 

ICSI 2010 

9A 

5 

5 

10 

10 

ICSI 2010 

5A 

ICSI 2010 

10A 

6 

6 

11 

11 

ICSI 2010 

6A 

ICSI 2010 


11A

I know what you have done 

in your last 

HOTO HOP Session 

An Introduction to Digital Image Forensics 

Thomas Gloe 

Matthias Kirchner * 

Faculty of 

Computer Science 

TU Dresden 

ICSI Berkeley 

21|01|2010

Computer forensics in a broader sense 

◮ computers interact with their environment 



1 0 0 1 

1 1 0 1 

WWW 

WWW 






1 0 0 1 

1 1 0 1 

WWW 

WWW 

WWW 

WWW 

◮ computers can be part of a network 


WWW 

WWW 

WWW 

WWW





1 0 0 1 

1 1 0 1 

WWW 

WWW 

WWW 

WWW 


◮ computers can be sensors itself 


WWW 

WWW 

WWW 

WWW





1 0 0 1 

1 1 0 1 

WWW 

WWW 

WWW 

WWW 


◮ computers can be sensors itself 

◮ computers leave physical evidence 


WWW 

WWW 

WWW 

WWW

Image sources 

⊲ Photoshop logo (title,64) http://commons.wikimedia.org/wiki/File:Photoshop_CS4.svg 

⊲ Porta advertisement (1) DOCMA 32, January 2010, http://www.docma.info 

⊲ models (1) http://photoshopdisasters.blogspot.com 

⊲ Iranian missile test (7) http://www.spiegel.de 

⊲ hard drive (9,61) http://commons.wikimedia.org/wiki/File:Open_hard-drive.jpg 

⊲ floppy disk (13,54) http://commons.wikimedia.org/wiki/GNOME_Desktop_icons 

⊲ core memory (13) http://commons.wikimedia.org/wiki/File:KL_CoreMemory.jpg 

⊲ digital image (14,55) http://commons.wikimedia.org/wiki/File:Crystal_Project_lphoto.png 

⊲ newspaper (18) http://commons.wikimedia.org/wiki/Nuvola/apps 

⊲ camera1, scanner, monitor, printer, 3D (18,19,20,25) http://commons.wikimedia.org/wiki/Crystal_Clear 

⊲ video camera, color palette (18,19) http://commons.wikimedia.org/wiki/Category:Crystal_Project 

⊲ beamer (18) http://www.oxygen-icons.org 

⊲ Blender logo (18) http://commons.wikimedia.org/wiki/Blender_(software) 

⊲ camera2 (19,20) http://commons.wikimedia.org/wiki/Camera 

⊲ George Bush (57) http://www.snopes.com/photos/politics/bushbook.asp 

⊲ fingerprints (61) http://www.lanl.gov/news/albums/chemistry/fingerprint.jpg 

⊲ handcuffs (61) http://commons.wikimedia.org/wiki/File:Handcuffs01_2003-06-02.jpg

Slides

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?