Problem - Kevin Tafuro

See Also
Recipe 11.11
11.16 Compressing Data with Entropy into a
Fixed-Size Seed
Problem
You are collecting data that may contain entropy, and you will need to output a
fixed-size seed that is smaller than the input. That is, you have a lot of data that has a
little bit of entropy, yet you need to produce a fixed-size seed for a pseudo-random
number generator. At the same time, you would like to remove any statistical biases
(patterns) that may be lingering in the data, to the extent possible.
Alternatively, you have data that you believe contains one bit of entropy per bit of
data (which is generally a bad assumption to make, even if it comes from a hardware
generator; see Recipe 11.19), but you’d like to remove any patterns in the data that
could facilitate analysis if you’re wrong about how much entropy is there. The process
of removing patterns is called whitening.
Solution
You can use a cryptographic hash function such as SHA1 to process data into a
fixed-size seed. It is generally a good idea to process data incrementally, so that you
do not need to buffer potentially arbitrary amounts of data with entropy.
Discussion
Be sure to estimate entropy conservatively. ( See Recipe 11.19.)
It is a good idea to use a cryptographic algorithm to compress the data from the
entropy source into a seed of the right size. This helps preserve entropy in the data,
up to the output size of the message digest function. If you need fewer bytes for a
seed than the digest function produces, you can always truncate the output. In addition,
cryptographic processing effectively removes any patterns in the data (assuming
that the hash function is a pseudo-random function). Patterns in the data can
help facilitate breaking an entropy source (in part or in full), particularly when that
source does not actually produce as much entropy as was believed.
Compressing Data with Entropy into a Fixed-Size Seed | 613
This is the Title of the Book, eMatter Edition
Copyright © 2007 O’Reilly & Associates, Inc. All rights reserved.