Production Technology Seminar 2009 - EBU Technical

37
servers managers, the more fingers in the pie!) - unless somehow complexity can be used to reduce risk
- and to the reliability of individual devices. Besides the loss of storage devices there are many more
risks: format obsolescence, IT infrastructure obsolescence, file corruption, system corruption, human
errors and other human actions. They all increase in significance (impact) in proportion to the amount of
storage in use.
First conclusion: as storage gets really cheap… it gets really risky.
The control of loss includes:
� The prevention of loss. This is where most of the attention (and research) is directed: reducing
MTBF for devices, making copies (!), using storage management layer(s), introducing virtual storage
layer(s), using Digital Library technology (OAIS „packages‟ and preservation metadata).
� The mitigation of loss. When it‟s gone, it‟s gone? [30]. But this doesn‟t have to be the case. For
example, if you have a BMP raster scan image file, with 160 errors in 40k [31-right], the bit errors are
completely local and only affect the byte they occur on. If you have a GIF compressed file with 3 errors
in 10k [30-center], the bit errors affect the equations which re-create the image from an encoded form
and that propagate the errors. That is another reason to have uncompressed files in the archives.
Fortunately, there are files that can be read despite errors - anything with a sequence (lines, pages,
images with raster scan (or any sequential ordering of the data), audio, video. The structure „unit of
loss‟ needs then to be identifiable: particular pixels, samples, lines (text or video!), pages/frames. A
structure of independent units is also contributing to the mitigation of loss: files with independent units
(pages, lines, bytes) - so that the loss of one element does not affect any others. Unfortunately, most
files have lost this property because compression removed redundancy - using the similarities between
units ties them together - and whole conglomerations of data are affected by a single byte error.
In summary, a survival strategy:
� Understand costs and risks. But is very difficult to get from the storage industry substantial information
(besides the MTBF of their hardware devices); e.g. how much it will cost to have a 1% error rate versus
a 0.1 error rate.
� Keep (uncompressed) master material off-line (for now).
� Use only expensive managed on-line servers where usage (production/public access) justifies the
cost.
� The cost-benefits equation of robots needs re-analysis against very cheap hard drives, for low-volume
access.
� Uncompressed files have lowest risk.
� Migrate every five years.
… If delegating storage: maintain control!
2.10 Living in a Digital World - PrestoSpace & PrestoPRIME
Daniel Teruggi, INA Recherche, France
1) Getting your old analogue assets into the Digital World
The European PrestoSpace 25 project (2004 - 2008) involved 35 Partners collaborating in the project,
180 Archives users in 52 countries, 144 service providers in 26 countries.
The project [4] was about from making new machines for audio, video, and film handling, acquisition and
digitisation [5], restoration [6], storage and archive management, metadata extraction [7] to a Turnkey
System for hosting Digital Audiovisual Archives. The main objective was to make preservation faster,
better, and cheaper! A main concept was the Preservation Factory 26 , to take the industrial model versus
the mainly applied artisanal approach.
25 http://prestospace.org/
26 This concept had not been protected and was „taken over‟ and trademarked by Sony! Since the new name of „PrestoSpace
Factory‟
© EBU 2009 / Production Technology seminar / January 27 - 29, 2009
Reproduction prohibited without written permission of EBU TECHNICAL & EBU TRAINING