Combinational image watermarking in the spatial and frequency ...

Pattern Recognition 36 (2003) 969 – 975
www.elsevier.com/locate/patcog
Combinational image watermarking in the spatial
and frequency domains
FrankY. Shih ∗ , Scott Y.T. Wu
Computer Vision Laboratory, Department of Computer Science, New Jersey Institute of Technology, Newark, NJ 07102, USA
Abstract
Received 15 February 2002; received in revised form 24 April 2002; accepted 30 May 2002
In order to provide more watermarks and to minimize the distortion of the watermarked image, a novel technique using
the combinational spatial and frequency domains is presented in this paper. The splitting of the watermarkimage into two
parts, respectively, for spatial and frequency insertion relies on the user’s preference and data importance. Experimental results
provide the comparisons when di erent sized watermarks are embedded into a grayscale image. The proposed combinational
image watermarking possesses the following advantages. More watermark data can be inserted into the host image, so that
the capacity is increased. The splitting of the watermarkinto two parts makes the degree of protection double. The splitting
strategy can be designed even more complicated to be unable to compose. Furthermore, to enhance robustness, a random
permutation of the watermarkis used to defeat the attacks of signal processing, such as image crops.
? 2002 Published by Elsevier Science Ltd on behalf of Pattern Recognition Society.
Keywords: Image processing; Image watermarking; Compression; Security; Encryption; Signal processing
1. Introduction
Digital images, video, and audio have revolutionized in
the way of largely stored, manipulated, and transmitted. It
gives rise to a wide range of applications in electronics, entertainment
and medial industry [1]. Creative ways of storing,
accessing and distributing data have generated lots of
bene ts into the digital multimedia eld. However, these
bene ts brought with concomitant risks of data piracy. One
of the solutions to provide security in copyright protection is
watermarking which embeds special marks in a host image.
Digital watermarking has been proposed as a way to
identify the source, creator, owner, distributor, or authorized
consumer of a document or an image. It can also be
used for tracing images that have been illegally distributed.
Watermarking, when complimented by encryption,
∗ Corresponding author. Tel.: +1-973-596-5654; fax: +1-973-
596-5777.
E-mail address: shih@cis.njit.edu (F.Y. Shih).
can serve many purposes including copyright protection,
broadcast monitoring, and data authentication.
There are many aspects to be noticed in watermarking
design, for example, imperceptibility, security, capacity and
robustness. Many researchers have been focusing on security
and robustness [2–6], but rarely on the watermarking
capacity [7,8]. Indeed, both security and robustness are important
because the watermarkimages are expected to be
irremovable and unperceivable. Nevertheless, if we can embed
a large watermarkimage into a host image, the application
becomes more exible in many areas.
There are two methods of performing watermarking, one
in spatial domain, and the other in frequency domain. Each
technique has its own advantage and disadvantage. In the
spatial domain [3,9], we can simply insert watermarkinto
a host image by changing the gray levels of some pixels
in the host image, but the inserted information may be easily
detected using computer analysis. In the frequency domain
[4,5], we can insert watermarkinto the coe cients of
a transformed image, for example, using the discrete Fourier
0031-3203/02/$30.00 ? 2002 Published by Elsevier Science Ltd on behalf of Pattern Recognition Society.
PII: S0031-3203(02)00122-X

970 F.Y. Shih, S.Y.T. Wu / Pattern Recognition 36 (2003) 969 – 975
transform (DFT), discrete cosine transform (DCT) and discrete
wavelet transform (DWT). But we cannot embed too
much data in the frequency domain because the quality of
the host image will be distorted signi cantly. That is, the
size of watermarkshould be smaller than the host image.
Generally, the size of watermarkis 1=16 of the host image.
In order to provide more watermarkdata and to minimize
the distortion of the watermarked image, a novel technique
using the combinational spatial and frequency domains is
developed in this paper. This paper is organized as follows.
In Section 2, we present the overview of combinational image
watermarking. Sections 3 and 4 introduce the techniques
while embedding watermarkin the spatial and frequency
domains, respectively. Experimental results are shown in
Section 5. Section 6 discusses the further encryption of combinational
watermarking. Finally, the conclusions are made
in Section 7.
2. Overview of combinational image watermarking
In order to insert more data into a host image, the simple
way is to embed them in the spatial domain of the host
image. However, the disadvantage is that the inserted data
could be detectable by some simple extraction skills. How
can we insert more signals but unperceivable? We address
a new strategy of embedding large watermarks into a host
image by splitting the watermarkimage into two parts. One
is embedded in the spatial domain of a host image, and the
other in the frequency domain.
Let H be the original gray-level host image with size
N × N and W be the binary watermarkimage with size
M × M. W 1 and W 2 are the two separated watermarks from
W: H S is the image combined from H and W 1 in the spatial
domain. H DCT is the image where H s is the transformed
into the frequency domain by DCT. H F is the image where
H DCT and W 2 are combined in the frequency domain. Let ⊕
denote the operation that substitutes bits of watermarkfor
least signi cant bits (LSB) of the host image.
The algorithm of the proposed combinational image
watermarking is presented below. Its owchart is shown in
Fig. 1.
Fig. 1. The owchart in combinational spatial and frequency
domains.
Algorithm:
1. Separate watermarkinto two parts:
W = {w(i; j); 0 6 i; j ¡ M}, where w(i; j) ∈{0; 1},
W 1 = {w 1 (i; j); 0 6 i; j ¡ M1}, where w 1 (i; j) ∈{0; 1},
W 2 = {w 2 (i; j); 0 6 i; j ¡ M2}, where w 2 (i; j) ∈{0; 1},
M = M1 + M2
2. Insert W 1 into the spatial domain of H to obtain H S
asH s ={h s (i; j)=h(i; j)⊕w 1 (i; j); 0 6 i; j ¡ N}, where
h(i; j);h S (i; j) ∈{0; 1; 2;:::;2 L − 1} and L is the number
of bits used as in the gray level of pixels.
3. Transform H S by DCT to obtain H DCT .
4. Insert W 2 into the coe cients of H DCT to obtain H F as
H F = {h F (i; j) =h DCT (i; j) ⊕ w 2 (i; j); 0 6 i; j ¡ N},
where h F (i; j) ∈{0; 1; 2;:::;2 L − 1}.
5. Transform the embedded host image by Inverse DCT.
The criteria of splitting the watermarkimage into two
parts, which are individually inserted into the input image
in the spatial and frequency domains, depend on users and
applications. In principle, the most important data exist in
the center of the image. Therefore, a simply way of splitting
is to select the central window in the watermarkimage
to be inserted into the frequency domain. With the user’s
preference, we can crop the most private data to be inserted
into the frequency domain.
3. The watermarking in the spatial domain
There are many ways of embedding a watermarkinto the
spatial domain of a host image, for example, substituting the
less signi cant bits of some pixels [10], changing the paired
pixels [11], and coding by textured blocks [12]. As shown in
Fig. 2, the watermarking can be implemented by modifying
the bits of some pixels in the host image. Let H ∗ be the
watermarked host image. The algorithm is shown below.
Algorithm:
1. Obtain pixels from the host image.
H = {h(i; j); 06i; j¡N }; h(i; j)∈{0; 1; 2;:::;2 L − 1}:
2. Obtain pixels from the watermark.
W = {w(i; j); 0 6 i; j ¡ M}:
Fig. 2. The owchart in spatial domains.

3. Substitute the pixels of the watermarkinto the LSB pixels
of the host image.
H ∗ = {h ∗ (i; j)=h(i; j) ⊕ w(i; j); 0 6 i; j ¡ N };
h ∗ (i; j) ∈{0; 1; 2;:::;2 L − 1}:
4. The watermarking in the frequency domain
Several approaches can be used in the frequency domain,
for example, JPEG-based [13], spread spectrum [2,14], and
content-based approaches [6]. How can we embed data into
the frequency domain of a host image and it appears unperceivable?
The transformation functions often-used are DCT,
DWT, and DFT. Generally, we can insert data into the coe
cients of a transformed image.
As shown in Figs. 3 and 4, we embed watermarkinto the
coe cients of a transformed host image. The important consideration
is what locations are best to place for embedding
watermarkin the frequency domain to avoid distortion [15].
Let H m and W n be the subdivided images from H and W ,
respectively, H m DCT be the image transformed from H m by
DCT, and H m F be the image combined by H m DCT and W n
in the frequency domain.
The algorithm is described as follows.
Algorithm:
1. Divide the host image into sets of 8 × 8 blocks.
H = {h(i; j); 0 6 i; j ¡ N },
H m = {h m (i; j); 0 6 i; j ¡ 8}, where h m (i; j) ∈{0; 1;
2;:::;2 L − 1} and m is the total number of the 8 × 8
blocks.
2. Divide the watermarkimage into sets of 2 × 2 blocks.
W = {w(i; j); 0 6 i; j ¡ M},
W n ={w n (i; j); 0 6 i; j ¡ 2}, where w n (i; j) ∈{0; 1} and
n is the total number of the 2 × 2 blocks.
3. Transform H m to H m DCT by DCT.
4. Insert W m into the coe cients of H m DCT .
H m F ={h m F (i; j)=h m DCT (i; j)⊕w m (i; j); 06i; j¡8};
h m DCT (i; j) ∈{0; 1; 2;:::;2 L − 1}:
5. Transform the embedded host image, H m F , by Inverse
DCT.
The criterion for embedding the watermarkimage into the
frequency domain of a host image is that the total number
of 8×8 blocks must be larger than the total number of 2×2
blocks.
5. Experimental results
Fig. 5 illustrates that some parts of a watermarkare important
to be split for security purpose. For example, people
F.Y. Shih, S.Y.T. Wu / Pattern Recognition 36 (2003) 969 – 975 971
Fig. 3. The owchart in frequency domains.
Fig. 4. The embedding skill in frequency domain.
cannot view whom the writer is in the images. Therefore, it
is embedded into the frequency domain and the rest into the
spatial domain. In this way, we not only enlarge the capacity,
but also secure the information that we are concerned
with.
Fig. 6 shows the original Lena picture with size 256×256.
Fig. 7 is the traditional technique of embedding a 64 ×
64 watermarkinto the frequency domain of a host image.
Fig. 7(a) is the original 64 × 64 watermarkimage and
Fig. 7(b) is the watermarked Lena image by embedding Fig.
7(a) into the frequency domain of Fig. 6. Fig. 7(c) is the
extracted watermarkimage from Fig. 7(b).
Fig. 8 demonstrates the embedding of a large watermark,
a 128×128 image, into a host image. Fig. 8(a) is the original
128 × 128 watermarkimage. Fig. 8(b) and (c) are the two
divided images from the original watermark, respectively.
We obtain Fig. 8(e) by embedding Fig. 8(b) into the spatial
domain of Lena image and obtain Fig. 8(f) by embedding
Fig. 8(c) into the frequency domain of the watermarked
Lena image in Fig. 8(e). Fig. 8(d) is the extracted watermark
from Fig. 8(f).
Fig. 9(a) shows a larger watermark, a 256 × 256 image,
which is split into two parts as in Fig. 5. Fig. 9(c)
and (d) is the watermarked images after embedding watermarkin
the spatial and frequency domains of a host image,
respectively. Fig. 9(b) is the extracted watermarkfrom
Fig. 9(d).
The error measures used in this paper are normalized correlation
(NC) and power signal-to-noise ratio (PSNR). They
are de ned as follows, where the symbols are de ned in the

972 F.Y. Shih, S.Y.T. Wu / Pattern Recognition 36 (2003) 969 – 975
previous sections:
N N i=1
N i=1
Fig. 6. Lena image.
j=1
NC =
H(i; j)H F (i; j)
N ;
j=1 [H(i; j)]2
N N i=1 j=1
PSNR = 10 log10 [h∗ (i; j)] 2
N N i=1 j=1 [h(i; j) − h∗ (i; j)] 2
:
Table 1 presents the results of embedding di erent sizes
of watermarkinto a host image. PSNR in the rst step means
comparing both the original and the embedded Lena images
in the spatial domain. PSNR in the second step means comparing
both images in the frequency domain. NC means the
normalized correlation by comparing both the original and
the extracted watermarks.
Fig. 5. A 64 × 64 image is cut from a 256 × 256 image.
Fig. 7. The traditional technique embedding a 64 × 64 watermark
into Lena image.
6. Further encryption of combinational watermarking
For the purpose of enhancing robustness, a random permutation
of the watermarkis used to defeat the attacks of
signal processing, such as image crops. The procedures are
shown in Fig. 10.
A random sequence generator is used in order to relocate
the order of sequential numbers [5]. For example, the 12-bit
random sequence generator is used to relocate the order of
a watermarkwith size 64 × 64, as shown in Fig. 11. We
rearrange the bits 9 and 6 to the rear of the whole sequence
and the result is shown in Fig. 11(b).
Fig. 12 shows the result when a half of the Lena image
is cropped. Fig. 12(a) is the original 128 × 128 watermark,
Fig. 12(b) is the cropped Lena image, and Fig. 12(c) is the
extracted watermark.

F.Y. Shih, S.Y.T. Wu / Pattern Recognition 36 (2003) 969 – 975 973
Fig. 8. The results when embedding a 128 × 128 watermarkinto Lena image.
Fig. 9. The results when embedding a 256 × 256 watermarkinto Lena image.

974 F.Y. Shih, S.Y.T. Wu / Pattern Recognition 36 (2003) 969 – 975
Table 1
The comparisons when embedding di erent sized watermarks into
a Lena image with size 256 × 256
Fig. 10. The procedures of randomly permuting the watermark.
64 × 64 128 × 128 256 × 256
PSNR in rst step None 56.58 51.14
PSNR in second step 64.57 55.93 50.98
NC 1 0.9813 0.9644
Fig. 11. A 12-bit random sequence generator.
Table 2 shows the results when parts of an embedded
host image with a watermarkof 128 × 128 are cropped. For
example, if the size of 64 × 64 is cropped from a 256 ×
256 embedded host image, the NC is 0.92. If a half of the
embedded host image is cropped (i.e., eight of 64 × 64) as
shown in Fig. 12(b), the NC is 0.52.
7. Conclusions
In this paper, we have presented the progression of embedding
a large watermarkinto a host image. The technique
is based on both spatial and frequency domains. In the
Fig. 12. The results when cropping half of a Lena image.
Table 2
The comparisons when cutting di erent sizes of a Lena image
Times of 1 2 3 4 5 6 7 8
64 × 64
NC 0.92 0.88 0.81 0.73 0.68 0.61 0.58 0.52
spatial domain, we mainly substitute the LSB bits in a host
image with the bits in the watermarkimage. In the frequency
domain, we insert data into the low frequency of a host
image. Simulation results demonstrate the embedding a large
watermarkinto a host image.

Mostly, the important part of an image is not enormous.
We can cut the important part and embed it into the frequency
domain and the rest into the spatial domain of a host
image. Therefore, we cannot only enlarge the size of watermark,
but also remain the property of security and imperceptibility.
The proposed combinational image watermarking
possesses the following advantages. More watermarkdata
can be inserted into the host image, so that the capacity is increased.
The splitting of the watermarkinto two parts makes
the degree of protection double. The splitting strategy can be
designed even more complicated to be unable to compose.
References
[1] H. Berghel, L. O’Gorman, Protecting ownership rights through
digital watermarking, IEEE Comput. Mag. 29 (7) (1996)
101–103.
[2] I. Cox, J. Kilian, T. Leighton, T. Shamoon, Secure
spread spectrum watermarking for images audio and video,
Proceedings of the 1996 IEEE International Conference on
Image Processing (ICIP’96), Lausanne, Switzerland, Vol. III,
pp. 243–246.
[3] N. Nikolaidis, I. Pitas, Robust image watermarking in the
spatial domain, Signal Process. 66 (3) (1998) 385–403.
[4] Jiwu Huang, Yun Q. Shi, Yi Shi, Embedding image
watermarks in DC components, IEEE Trans. CSVT 10 (6)
(2000) 974–979.
[5] Shinfeng D. Lin, Chin-Feng Chen, A robust DCT-based
watermarking for copyright protection, IEEE Trans. Consumer
Electron. 46 (3) (2000) 415–421.
F.Y. Shih, S.Y.T. Wu / Pattern Recognition 36 (2003) 969 – 975 975
[6] P. Bas, J.-M. Chassery, B. Macq, Image watermarking: an
evolution to content based approaches Pattern Recognition 35
(2002) 545–561.
[7] M. Barni, F. Bartolini, A. De Rosa, A. Piva, Capacity of the
watermarkchannel: how many bits can be hidden within a
digital image? Proc. SPIE 3657 (1999) 437–448.
[8] P. Moulin, M.K. Mihcak, The data-hiding capacity of image
sources, IEEE Trans. Image Process. (2002), in press.
[9] O. Bruyndonckx, J.-J. Quisquater, B. Macq, Spatial method
for copyright labeling of digital images, Proceeding of IEEE
Workshop on Nonlinear Signal and Image processing, Neos
Marmaras, Greece, 20–22 June 1995, pp. 456–459.
[10] R. Wolfgang, E. Delp, A watermarking technique for digital
imagery: further studies, International Conference on Imaging
Science, Systems and Technology, Los Vegas, Nevada, Juillet
1997.
[11] I. Pitas, T. Kaskalis, Applying signatures on digital images,
Workshop on Nonlinear Signal and Image Processing, IEEE,
Neos Marmaras, June 1995, pp. 460–463.
[12] G. Caronni, Assuring ownership rights for digital images,
Proceedings of Reliable IT Systems, VIS’95, Viewveg
Publishing Company, Germany, 1995.
[13] K.E. Zhao J., Embedding robust labels into images for
copyright protection, Technical Report, Fraunhofer Institute
for Computer Graphics, Darmatadt, Germany, 1994.
[14] I. Cox, J. Killian, T. Leighton, T. Shamoon, Secure spread
spectrum watermarking for multimedia, IEEE Trans. Image
Process. 6 (12) (1997) 1673–1687.
[15] C.-T. Hsu, J.-L. Wu, Hidden signature in images, IEEE Trans.
Image Process. 8 (1999) 58–68.
About the Author—FRANK Y. SHIH received the B.S. degree from National Cheng-Kung University, Taiwan, in 1980, the M.S. degree
from the State University of New Yorkat Stony Brook, in 1984, and the Ph.D. degree from Purdue University, West Lafayette, Indiana,
in 1987, all in Electrical Engineering. He is presently a professor jointly appointed in the Department of Computer Science (CS) nad the
Department of Electrical and Computer Engineering (ECE) at New Jersey Institute of Technology, Newark, NJ. He currently serves as the
Associate Chair of the CS Department and the Director of Computer Vision Laboratory.
Dr. Shin is on the Editorial Board of the International Journal of Systems Integration. He is also an associate editor of the International
Journal of Information Sciences, and of the International Journal of Pattern Recognition. He has served as a member of several organizing
committees for technical conferences and workshops. He was the recipient of the Research Initiation Award from the National Science
Foundation in 1991. He was the recipient of the Winner of the International Pattern Recognition Society Award for Outstanding Paper
Contribution. He has received several awards for distinguished research at New Jersey Institute of Technology.He has served several times
in the Proposal Review Panel of the National Science Foundation on Computer Vision and Machine Intelligence.
He holds the IEEE senior membership. Dr. Shih has published over 110 technical papers in well-known prestigious journals and conferences.
His current research interests include image processing, computer vision, computer graphics, arti cial intelligence, expert systems, robotics,
computer architecture, fuzzy logic, and neural networks.
About the Author—YI-TA WU was born in Taipei, Taiwan, on March 22 1971. He received the B.S. degree in Physics from Tamkang
University, Taipei, Taiwan, in 1995, and the M.S. degree in Computer Science from National Dong-Hwa University, Hualien, Taiwan, in
1997. He is current a Ph.D. student in the Computer Science Department, New Jersey Institute of Technology, Newark, NJ. His research
interest include image processing, current vision, pattern recognition, and database.