19.07.2016 Views

Migrating digital records

1URFZdw

1URFZdw

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong><br />

A guideline for Queensland public authorities<br />

Queensland State Archives<br />

June 2012<br />

Security classification: Public


Department of Science, Information Technology and Innovation<br />

Document details<br />

Security Classification<br />

PUBLIC<br />

Date of review of security classification June 2012<br />

Authority<br />

Queensland State Archives<br />

Author<br />

Queensland State Archives<br />

Document Status<br />

Final<br />

Version Version 1.0<br />

Contact for enquiries<br />

All enquiries regarding this document should be directed in the first instance to:<br />

Digital Archives Unit<br />

Queensland State Archives<br />

07 3131 7777<br />

info@archives.qld.gov.au<br />

Copyright<br />

<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: a guideline for Queensland public authorities<br />

Copyright © The State of Queensland (Department of Science, Information Technology, Innovation<br />

and the Arts) 2012<br />

Licence<br />

<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: a guideline for Queensland public authorities by Queensland State<br />

Archives is licensed under a Creative Commons Attribution 3.0 Australia Licence. To view a copy<br />

of this licence, please visit http://creativecommons.org/licenses/by/3.0/au/.<br />

Information security<br />

This document has been security classified using the Queensland Government Information<br />

Security Classification Framework (QGISCF) as PUBLIC and will be managed according to the<br />

requirements of the QGISCF.<br />

Page 2 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Executive Summary<br />

This guideline provides advice to public authorities on the key recordkeeping issues to be<br />

considered when planning for and carrying out data migration projects where the business system<br />

contains public <strong>records</strong>. Moving public <strong>records</strong> from one hardware/software configuration to<br />

another, or from one generation of computer technology to a subsequent generation, requires<br />

explicit planning to ensure that the public <strong>records</strong> maintain their authenticity and integrity in<br />

accordance with the Public Records Act 2002. Digital source <strong>records</strong> that have been migrated<br />

cannot be disposed of unless they have met certain criteria as outlined under the General<br />

Retention and Disposal Schedule for Digital Source Records (QDAN678), and this guideline<br />

provides guidance to Queensland public authorities about the migration of <strong>digital</strong> <strong>records</strong> and the<br />

lawful disposal of the source <strong>records</strong>.<br />

The guideline assumes:<br />

<br />

<br />

familiarity with the requirements of the Public Records Act 2002 and a knowledge of the<br />

various supporting standards and guidelines, and<br />

that <strong>records</strong> management expertise is incorporated into the overall migration project<br />

planning and implementation.<br />

The guideline is organised into three parts:<br />

1. Introduction – outlines the context of migrating <strong>digital</strong> <strong>records</strong> as it is applied in the<br />

guideline.<br />

2. Fundamental issues when migrating public <strong>records</strong> – examines the challenge of evolving<br />

technology, drivers for migration, risks, obligations and the treatment of source <strong>records</strong><br />

after migration.<br />

3. Recordkeeping considerations for migration projects – outlines key recordkeeping<br />

considerations to be incorporated into the overall migration project, including various<br />

checklist tools included as appendices.<br />

This document forms part of a wider framework of <strong>digital</strong> continuity advice under development by<br />

Queensland State Archives to support public authorities in meeting their obligations in managing<br />

their <strong>digital</strong> public <strong>records</strong>.<br />

Page 3 of 57


Department of Science, Information Technology and Innovation<br />

Table of contents<br />

Executive Summary ...................................................................................................................... 3<br />

1. Introduction ............................................................................................................................ 5<br />

1.1 Purpose ............................................................................................................................ 5<br />

1.2 Audience .......................................................................................................................... 5<br />

1.3 Scope ............................................................................................................................... 5<br />

1.4 Authority .......................................................................................................................... 6<br />

1.5 Relationship to other Queensland State Archives’ publications ................................. 6<br />

1.6 Definitions ....................................................................................................................... 7<br />

1.7 Acknowledgements ......................................................................................................... 9<br />

2. Important considerations for migrating public <strong>records</strong> .................................................... 10<br />

2.1 Drivers for migration ..................................................................................................... 10<br />

2.2 Risks .............................................................................................................................. 11<br />

2.3 Obligations .................................................................................................................... 11<br />

2.4 Treatment of source <strong>records</strong> after migration .............................................................. 12<br />

3. Recordkeeping considerations for migration projects ...................................................... 14<br />

3.1 Migration project initiation ........................................................................................... 15<br />

3.2 Determining what is to be migrated ............................................................................. 17<br />

3.3 Determining where and how to migrate ....................................................................... 23<br />

3.4 Ensuring quality ............................................................................................................ 27<br />

3.5 Performing migration .................................................................................................... 30<br />

3.6 Post migration ............................................................................................................... 32<br />

Appendices ................................................................................................................................. 37<br />

Appendix A – IS40 Principle 7 and the migration of <strong>digital</strong> <strong>records</strong> ................................... 38<br />

Appendix B – Records manager’s project stages flowchart ............................................... 40<br />

Appendix C – Records manager’s migration project checklist ........................................... 42<br />

Appendix D – Recordkeeping controls in the target system ............................................... 53<br />

Appendix E – Testing and validation checklist ................................................................... 55<br />

Appendix F – Lessons learnt – hints and tips ...................................................................... 56<br />

Page 4 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

1. Introduction<br />

1.1 Purpose<br />

The purpose of this guideline is to assist public authorities to meet their recordkeeping obligations<br />

under the Public Records Act 2002 when they are planning and implementing data migration<br />

projects involving business systems that contain <strong>digital</strong> public <strong>records</strong>. The document also forms<br />

part of a wider framework of <strong>digital</strong> continuity advice developed by Queensland State Archives, to<br />

support public authorities in <strong>digital</strong> recordkeeping obligations. Specifically the guideline provides<br />

advice on:<br />

<br />

<br />

key recordkeeping considerations where the migrated version of the record will act as the<br />

official public record, and<br />

the correct disposal of the <strong>digital</strong> source <strong>records</strong> in accordance with the General Retention<br />

and Disposal Schedule for Digital Source Records (QDAN678).<br />

1.2 Audience<br />

The intended target audience for this guideline is information and <strong>records</strong> management staff<br />

responsible for advising on a public authority’s recordkeeping compliance and who will be working<br />

with:<br />

<br />

<br />

project team members, business analysts, business system owners and ICT colleagues,<br />

and<br />

other key stakeholders responsible for data and business system migration projects or<br />

activities.<br />

1.3 Scope<br />

This guideline applies to all Queensland public authorities, as defined in Schedule 2 of the Public<br />

Records Act 2002.<br />

The guideline provides support for the migration of <strong>digital</strong> public <strong>records</strong>, where the migration of the<br />

<strong>digital</strong> public <strong>records</strong> is from one hardware/software configuration to another or from one<br />

generation of information technology to a subsequent generation, for ongoing business and<br />

legislative purposes.<br />

This guideline:<br />

<br />

<br />

<br />

does not seek to provide all the technical advice on carrying out a data migration project<br />

does not replace the existing project management methodology adopted by the public<br />

authority, and<br />

does not cover the digitisation of <strong>records</strong> from a physical hardcopy format to a <strong>digital</strong> format<br />

such as when paper <strong>records</strong> are digitised. 1<br />

1<br />

For guidance about digitisation or microfilming of <strong>records</strong>, please see the advice on Digitisation and disposal<br />

Page 5 of 57


Department of Science, Information Technology and Innovation<br />

1.4 Authority<br />

The State Archivist has issued this guideline in accordance with section 25(1)(f) of the Public<br />

Records Act 2002, which enables the State Archivist to make policy, standards, and guidelines<br />

about the making, keeping, preserving, managing and disposing of public <strong>records</strong>.<br />

1.5 Relationship to other Queensland State Archives’ publications<br />

This guideline is specifically supported by:<br />

<br />

<br />

the QSA publication Metadata for <strong>digital</strong> continuity: a companion guideline to the<br />

Queensland recordkeeping metadata standard and the<br />

General Retention and Disposal Schedule for Digital Source Records (QDAN678).<br />

Metadata for <strong>digital</strong><br />

continuity<br />

supports<br />

<strong>Migrating</strong> <strong>digital</strong><br />

<strong>records</strong><br />

expands on<br />

Queensland<br />

Recordkeeping<br />

Metadata Standard<br />

(QRKMS)<br />

Guidelines and<br />

functional<br />

requirements for<br />

<strong>records</strong> in<br />

business systems<br />

supports<br />

supports<br />

General Retention<br />

and Disposal<br />

Schedule for Digital<br />

Source Records<br />

Figure 1 – Migration, metadata and retention and disposal connections<br />

It is also recommended that recordkeeping and information management staff have an<br />

understanding of the following documentation:<br />

<br />

<br />

<br />

Information standard 40: recordkeeping (IS40)<br />

Information standard 31: retention and disposal of public <strong>records</strong> (IS31)<br />

Related public <strong>records</strong> briefs:<br />

o<br />

o<br />

ICA: Guidelines and functional requirements for <strong>records</strong> in business systems<br />

Advice on the Destruction of Public Records<br />

o<br />

Decommissioning Business Systems<br />

It is assumed that the recordkeeping considerations detailed in this guideline will be incorporated<br />

into the public authority’s own project management methodology, for example the Queensland<br />

Government Project Management Methodology 2 .<br />

2<br />

Accessible at Queensland Government Project Management Methodologies<br />

Page 6 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

1.6 Definitions<br />

Records and information management specific terms are defined in Queensland State Archives’<br />

Glossary of Archival and Recordkeeping Terms available on Queensland State Archives’ website.<br />

Some specific terms as they are used in the context of migration processes in this guideline are<br />

described below.<br />

Migration<br />

There are many different definitions and understandings of the term ‘migration’ across disciplines.<br />

Within the context of this guideline, the term migration is used to encompass the process of<br />

preserving the authenticity of <strong>records</strong> when relocating <strong>digital</strong> <strong>records</strong> from one hardware/software<br />

configuration to another, or from one generation of computer technology to a subsequent<br />

generation. 3<br />

Source <strong>records</strong> and migrated <strong>records</strong><br />

‘Source record’ and ‘migrated record’ are terms used in this document. For clarity, ‘source record’<br />

refers to the record being migrated, and the ‘migrated record’ is the record resulting from the<br />

migration, as illustrated in Figure 2 below.<br />

Old<br />

home<br />

Relocation<br />

to<br />

New home<br />

Source<br />

system<br />

Target<br />

system<br />

Source<br />

<strong>records</strong><br />

Migration<br />

Migrated<br />

<strong>records</strong><br />

Figure 2 – Migration of source <strong>records</strong> to a new target location<br />

3<br />

Referred to by R. Harvey (2005), Preserving Digital Materials, K. G. Saur, Munich, page 148. Originally used in<br />

Preserving Digital Information: Report of the Task Force on Archiving of Digital Information, 1996. Available at:<br />

http://www.clir.org/pubs/reports/pub63watersgarrett.pdf<br />

Page 7 of 57


Department of Science, Information Technology and Innovation<br />

Data cleansing<br />

Data cleansing involves the removal of out-of-date, duplicated, inaccurate or redundant information<br />

and is a recommended practice prior to undertaking migration activities in order to minimise the<br />

relocation of unnecessary or inaccurate information. In the context of this document, data<br />

cleansing processes are applied to the review of the data/<strong>records</strong>/metadata in the source system in<br />

order to make decisions about what data/<strong>records</strong>/metadata will be migrated into the target system.<br />

It should be noted that data cleansing activities are likely to trigger the need for an authorised<br />

disposal of time-expired <strong>records</strong>. This means that any public <strong>records</strong> identified for deletion during<br />

data cleansing in the source system prior to the migration activity will need to be disposed of in<br />

accordance with an approved Retention and Disposal Schedule.<br />

Data mapping<br />

Data mapping refers to the process of establishing connections or relationships between the data<br />

elements of the source and target systems that will enable the migration of the <strong>records</strong> and their<br />

associated metadata, and contribute to the maintenance of the authenticity and integrity of the<br />

<strong>records</strong>.<br />

Deletion vs destruction vs disposal<br />

In this guideline, these three terms have distinct meanings and are described below:<br />

<br />

<br />

<br />

Deletion in this guideline is a generalised term referring to the erasing of <strong>digital</strong> information<br />

contained in a business system.<br />

Destruction refers to the process of eliminating or deleting <strong>records</strong> that do not have continuing<br />

value, beyond the point of any possible reconstruction.<br />

Disposal of <strong>records</strong> as described in the Public Records Act (2002) includes the actions of<br />

destroying, damaging, abandoning, transferring, selling, donating or giving away a record, or<br />

part of a record. In the context of this guideline, the term disposal has been used to refer to:<br />

o<br />

o<br />

the deletion of the source record, following migration, or<br />

the deletion of eligible time-expired <strong>records</strong> prior to migration, undertaken in<br />

accordance with the terms stated in an approved Retention and Disposal Schedule.<br />

Digital continuity<br />

Digital continuity used in the context of this guideline is the ability to ensure that despite<br />

organisational, business and technological change, the access, readability and the authenticity of<br />

<strong>digital</strong> <strong>records</strong> is maintained for as long as they are required to be kept for business, legal and/or<br />

archival requirements. Digital continuity involves more than preserving <strong>digital</strong> <strong>records</strong>, and requires<br />

a range of robust <strong>records</strong> and information management policies and practices.<br />

Queensland State Archives is responsible for developing a <strong>digital</strong> continuity strategy and tools for<br />

state government and this guideline is one of the tools developed to support the continuity of <strong>digital</strong><br />

public <strong>records</strong>.<br />

Page 8 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Encryption and decryption<br />

Encryption refers to a process where security coding is applied to data that converts it into a<br />

secured format for protected transmission across networks. To read the data the receiver must<br />

have the decryption key coding to restore the data to its original form.<br />

1.7 Acknowledgements<br />

A range of advice was referenced in the development of this guideline. Queensland State Archives<br />

in particular acknowledges State Records New South Wales’ migration advice in their Managing<br />

Digital Records Guideline.<br />

Queensland State Archives would like to thank the many public authorities that contributed to the<br />

development of the guideline and their representatives who also commented on the exposure draft.<br />

Page 9 of 57


Department of Science, Information Technology and Innovation<br />

2. Important considerations for migrating public <strong>records</strong><br />

Two important recordkeeping values need to be upheld when undertaking migration projects. They<br />

are:<br />

<br />

<br />

maintaining the authenticity, useability, integrity and reliability of public <strong>records</strong> for as long<br />

as they are required to be kept for business, legal and / or archival purposes, and<br />

ensuring the disposal of <strong>records</strong> is authorised and in accordance with an approved<br />

Retention and Disposal Schedule.<br />

These values have broad reaching implications for <strong>records</strong> migration projects, and the<br />

recordkeeping and information professional not only needs to have a grounding knowledge of<br />

these values but also needs to be able to translate them and have input into applying these values<br />

to the overall migration project.<br />

Before dealing with specific recordkeeping considerations in Section 3, this section will specify the:<br />

<br />

<br />

<br />

<br />

<br />

drivers for migration<br />

potential risks to <strong>records</strong><br />

recordkeeping obligations, and the<br />

treatment of source <strong>records</strong> after migration<br />

and will assist in setting the scene for a recordkeeping and information management<br />

professional’s input into a <strong>records</strong> migration project.<br />

2.1 Drivers for migration<br />

Digital <strong>records</strong> are dependent on technology for their access and use. Over a period of time, the<br />

migration of the <strong>records</strong> from one hardware/software configuration to another may be required for<br />

the continuing access, useability and ongoing management of the public <strong>records</strong> in accordance<br />

with business, legal and archival obligations.<br />

The need to migrate public <strong>records</strong> may be triggered by a range of drivers, for example:<br />

<br />

<br />

<br />

<br />

obsolescence and technological changes that require the upgrading or decommissioning of<br />

business systems<br />

changing business needs that lead to the adoption of a new business system<br />

system and storage rationalisation, e.g. as a result of the implementation of the Application<br />

Rationalisation Methodology (ARM), and / or<br />

machinery of government changes that require the transfer of functions or activities<br />

recorded in business systems from one public authority to another entity.<br />

Page 10 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Upgrading or decommissioning<br />

business systems<br />

Changes to business needs<br />

Machinery of<br />

government changes<br />

Bulk <strong>records</strong><br />

migration<br />

Figure 3 – Examples of drivers for migration<br />

System and storage<br />

rationalisation<br />

2.2 Risks<br />

The processes for the relocation of public <strong>records</strong> from one hardware/software configuration to<br />

new technological environment/s may threaten the legal authenticity and integrity of <strong>digital</strong> public<br />

<strong>records</strong>. To mitigate risks to the public <strong>records</strong>, migrations must be controlled by authorised and<br />

planned policies and procedures that incorporate recordkeeping considerations within the public<br />

authority’s overall project management methodology.<br />

Specifically the threats and risks to <strong>records</strong> during migration include:<br />

<br />

<br />

<br />

failure to maintain the authenticity and integrity of the <strong>records</strong> (refer to Information Standard<br />

40 for details)<br />

loss of readability and accessibility, particularly for <strong>records</strong> that are required to be kept for<br />

the long-term or in perpetuity, and<br />

non-compliance with legal obligations for the maintenance of the public <strong>records</strong>, for<br />

example through inadvertent loss or unauthorised deletion through failure to identify and<br />

manage the existence of <strong>digital</strong> public <strong>records</strong> when decommissioning a system.<br />

It is important that risk mitigation assessments and strategies relevant to recordkeeping be<br />

incorporated into planning and that ownership of risk is delegated, managed and documented. Risk<br />

mitigation strategies may include:<br />

<br />

<br />

<br />

identifying and appraising the nature of the <strong>digital</strong> <strong>records</strong> to be migrated in accordance<br />

with approved Retention and Disposal Schedules, to assist in determining the potential risk<br />

value<br />

determining and documenting the capacity of the target system to capture the required<br />

recordkeeping metadata to sustain the authenticity and integrity of the migrated <strong>records</strong><br />

adopting robust quality assurance planning and processes.<br />

Section 3 of this guideline will highlight the key recordkeeping considerations that need to be<br />

integrated into risk mitigation planning.<br />

2.3 Obligations<br />

The Queensland Government Recordkeeping Policy Framework sets out the recordkeeping<br />

obligations of public authorities. Public authorities must meet all legislative and regulatory<br />

requirements relating to the management of <strong>records</strong> during migration projects and activities.<br />

This includes compliance with:<br />

Page 11 of 57


Department of Science, Information Technology and Innovation<br />

<br />

<br />

Public Records Act 2002 – this Act is concerned with ensuring the public <strong>records</strong> of<br />

Queensland are made, managed, kept and, if appropriate, preserved in a useable form for<br />

the benefit of present and future generations. It mandates a number of obligations for<br />

Queensland public authorities which include making and keeping full and accurate <strong>records</strong><br />

of its activities, ensuring the safe custody and preservation of <strong>records</strong> in its possession, and<br />

who must authorise disposal.<br />

Information Standard 40: Recordkeeping – this standard outlines seven mandatory<br />

principles for compliance by public authorities. These principles are summarised in Table 1.<br />

Appendix A provides further guidance on the application of Principle 7 in the <strong>digital</strong> <strong>records</strong><br />

migration context.<br />

Principle<br />

Principle 1<br />

Principle 2<br />

Principle 3<br />

Principle 4<br />

Principle 5<br />

Principle 6<br />

Principle 7<br />

Requirement<br />

Public authority recordkeeping must be compliant and accountable<br />

Recordkeeping must be monitored and audited for compliance<br />

Recordkeeping activity must be assigned and implemented<br />

Recordkeeping must be managed<br />

Recordkeeping systems must be reliable and secure<br />

Recordkeeping must be systematic and comprehensive<br />

Full and accurate <strong>records</strong> must be made and kept for as long as they are<br />

required for business, legislative, accountability and cultural purposes.<br />

Table 1 – Mandatory principles of Information Standard 40: Recordkeeping<br />

<br />

Information Standard 31: Retention and Disposal of Public Records - the Public Records<br />

Act 2002 prohibits the disposal of public <strong>records</strong> without the permission of the State<br />

Archivist. The primary purpose of this Information Standard is to assist public authorities to<br />

lawfully retain and dispose of public <strong>records</strong>.<br />

This standard, among other obligations, requires that public authorities must:<br />

o<br />

o<br />

o<br />

o<br />

dispose of public <strong>records</strong> in accordance with a Retention and Disposal Schedule<br />

approved by the State Archivist that is current at the time of disposal<br />

ensure all disposal is endorsed by the Chief Executive or an authorised delegate<br />

ensure the method of destruction of public <strong>records</strong> is appropriate to the sensitivity of<br />

the <strong>records</strong> and conforms with local environment regulations<br />

document the disposal of public <strong>records</strong>.<br />

2.4 Treatment of source <strong>records</strong> after migration<br />

Decisions about the treatment of the source <strong>records</strong> after their successful migration need to be<br />

incorporated into the migration project plan. Deletion of source <strong>records</strong> needs to be in accordance<br />

with the General Retention and Disposal Schedule for Digital Source Records (QDAN678), which<br />

addresses the treatment of the remaining source <strong>records</strong> after they have been migrated and<br />

enables lawful disposal.<br />

Section 3.6.1 and 3.6.2 of this guideline provide further advice about the application of retention<br />

and disposal obligations within the <strong>records</strong> migration project stages.<br />

Page 12 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Key points:<br />

The processes of moving <strong>records</strong> from one system to a new technological environment can<br />

threaten the authenticity, integrity, reliability and useability of <strong>digital</strong> <strong>records</strong>.<br />

To mitigate risks, migrations must be controlled by authorised and planned policies, processes<br />

and procedures that are designed to incorporate key recordkeeping standards.<br />

Source <strong>records</strong> are not automatically eligible for deletion. Certain obligations outlined in this<br />

guideline and the General Retention and Disposal Schedule for Digital Source Records<br />

(QDAN678) need to have been met before deletion can be authorised to be undertaken.<br />

Page 13 of 57


Department of Science, Information Technology and Innovation<br />

3. Recordkeeping considerations for migration projects<br />

This section outlines the recordkeeping considerations that need to be included in planning<br />

migration projects. The structure of this section follows a generic migration project methodology,<br />

however, it is acknowledged that public authorities will not necessarily all use the same<br />

methodology for migration projects. A public authority’s approach can be based on this document<br />

but will need to be adapted to the specific project management methodology adopted by the public<br />

authority.<br />

Table 2 below summarises the considerations that are detailed further in this section. A more<br />

readily useable version of the information in this table has been prepared as a flowchart, which is<br />

available at Appendix B. Additionally, the flowchart stages have been expanded into a checklist for<br />

<strong>records</strong> managers which provides, in essence, a summary of the material in the rest of this section.<br />

The checklist is available at Appendix C.<br />

Records<br />

migration<br />

stages<br />

1<br />

Migration<br />

project<br />

initiation<br />

2<br />

Determining<br />

what is to be<br />

migrated<br />

3<br />

Determining<br />

where &<br />

how to<br />

migrate<br />

4<br />

Ensuring<br />

quality<br />

5<br />

Performing<br />

migration<br />

6<br />

Post migration<br />

Description<br />

Recordkeeping considerations<br />

The initial stage<br />

of high-level<br />

scheduling and<br />

resource<br />

planning for the<br />

project.<br />

Project<br />

planning<br />

Roles and<br />

responsibilitie<br />

s<br />

Stakeholder<br />

communicatio<br />

ns<br />

The stage of the<br />

project when the<br />

scope of the<br />

migration is<br />

being<br />

established.<br />

Identifying the<br />

<strong>records</strong><br />

Establishing<br />

retention<br />

obligations<br />

Determining<br />

recordkeeping<br />

metadata<br />

Understanding<br />

characteristics<br />

of the <strong>records</strong><br />

The stage<br />

when<br />

decisions are<br />

being explored<br />

and made<br />

about the<br />

approach,<br />

method and<br />

target<br />

system(s) for<br />

the migration.<br />

Verifying<br />

target<br />

system/s<br />

recordkeepin<br />

g capabilities<br />

Making file /<br />

record format<br />

decisions<br />

Establishing<br />

migration<br />

approach<br />

The stage<br />

when quality<br />

assurance and<br />

testing and<br />

validation<br />

processes are<br />

identified,<br />

planned and<br />

undertaken.<br />

Preparing<br />

<strong>records</strong> for<br />

migration<br />

Testing and<br />

validation<br />

The stage<br />

concerned with<br />

the actual<br />

undertaking of<br />

the migration to<br />

extract/export,<br />

transfer and<br />

load/import the<br />

<strong>records</strong> into the<br />

target system(s).<br />

Testing roll<br />

back strategy<br />

Recordkeeping<br />

during cut over<br />

period<br />

Migration of<br />

<strong>records</strong><br />

Quality<br />

assuring the<br />

results of the<br />

migration<br />

The final<br />

recordkeeping<br />

considerations<br />

prior to project<br />

closure.<br />

Verifying<br />

conditions for<br />

disposal of<br />

source <strong>records</strong><br />

Disposal of<br />

<strong>digital</strong> ‘source<br />

<strong>records</strong>’<br />

Capture<br />

mandatory<br />

recordkeeping<br />

documentation<br />

Stakeholder<br />

relations and<br />

continuous<br />

improvement<br />

Table 2 – Project stages for recordkeeping considerations<br />

Page 14 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

3.1 Migration project initiation<br />

Migration<br />

project<br />

initiation<br />

Determining<br />

what is to be<br />

migrated<br />

Determining<br />

where & how<br />

to migrate<br />

Ensuring<br />

quality<br />

Performing<br />

migration<br />

Post<br />

migration<br />

3.1.1 Project planning<br />

A migration project that is systematically planned, initiated and carried out to well-defined<br />

procedures will help protect <strong>records</strong> and is required to ensure their evidential integrity.<br />

This guideline assumes that appropriate processes and documentation will be established to<br />

initiate a migration project by the broader organisational project team. This would include project<br />

planning documentation which clearly describes the drivers, approaches, legislative and policy<br />

requirements and expected outcomes for the project. It also assumes that appropriate change<br />

management activities are planned. It therefore does not address these planning issues in detail<br />

and assumes that the <strong>records</strong> manager will engage with the broader project.<br />

During the planning process it is important to ensure that:<br />

<br />

<br />

<br />

there is recorded acknowledgement within plans of the need to meet a public authority’s<br />

recordkeeping obligations throughout the project<br />

risk mitigation planning is incorporated that is commensurate with the level of value<br />

associated with the <strong>records</strong>, and<br />

there is appropriate scheduling of time and resources for the recordkeeping activities<br />

associated with the migration of <strong>digital</strong> public <strong>records</strong>.<br />

Key point:<br />

Ensure recordkeeping obligations, standards and activities are well understood by the project<br />

team and are sufficiently incorporated into the planning and scheduling for the project.<br />

3.1.2 Roles and responsibilities<br />

As outlined in section 2.2 migrations can be a high risk activity for <strong>records</strong>. Those responsible for<br />

the management of <strong>records</strong> must be involved in all <strong>records</strong> migration projects. This includes<br />

projects to migrate <strong>records</strong> stored in any business system, not just for projects that involve<br />

dedicated recordkeeping systems such as electronic document and <strong>records</strong> management systems<br />

(eDRMS). 4<br />

When migration projects involving <strong>records</strong> are established within a public authority, and roles and<br />

responsibilities are planned, <strong>records</strong> and information managers are well placed to contribute<br />

expertise to the project through:<br />

4<br />

In this guideline business systems refer to automated systems that have a primary purpose of facilitating business<br />

transactions (e.g. finance, licensing, client-relationship systems, etc.) and that do not necessarily contain <strong>records</strong><br />

management functionality. This is distinct from electronic document and <strong>records</strong> management systems (eDRMS) that are<br />

designed with <strong>records</strong> management functionality to manage the <strong>records</strong> it captures throughout their lifecycle.<br />

Page 15 of 57


Department of Science, Information Technology and Innovation<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

assisting with the identification and appraisal of the <strong>records</strong> within business systems, how<br />

long the <strong>records</strong> must be legally retained and which <strong>records</strong> are eligible for lawful disposal<br />

determining how the <strong>records</strong> are being managed within the existing system and their<br />

current and ongoing recordkeeping compliance needs<br />

facilitating the process for the authorised disposal of <strong>records</strong><br />

supporting quality assurance of migrated <strong>records</strong> in collaboration with business area<br />

experts and others<br />

assisting in the data mapping process between the source and target systems to ensure<br />

<strong>records</strong>, including their metadata, remain full and accurate <strong>records</strong> of business and that<br />

their authenticity can be verified<br />

advising on the recordkeeping controls needed in the design and configuration of the target<br />

system<br />

developing and delivering training and awareness materials and sessions related to <strong>records</strong><br />

management, and<br />

advising on and promoting continuous improvement in recordkeeping functionality within<br />

new systems.<br />

3.1.3 Stakeholder communication planning<br />

As mentioned above, this guideline assumes that appropriate change management activities are<br />

planned as part of the overall project management plan. However it is important that those<br />

responsible for <strong>records</strong> and information management engage with the overall communications<br />

strategy and the proposed timings for the migration of the <strong>records</strong>. This is needed in order to<br />

understand and plan for the potential impact on recordkeeping requirements, and on the business<br />

and its stakeholders, during the time of the <strong>records</strong> changeover period between the source system<br />

and the target system. (Please note: Appendix F provides some real-life insights into<br />

recordkeeping considerations during this time. Refer to F.3.)<br />

There will be a need to ensure that appropriate training of staff is undertaken. This may require<br />

targeted promotion and the development of new procedures relating to the management of the<br />

<strong>records</strong> in the newly migrated environment.<br />

Key points:<br />

Those with responsibility for <strong>records</strong> management should be engaged and consulted in migration<br />

projects to ensure that recordkeeping considerations are integrated into the overall project<br />

management planning.<br />

The key objective of <strong>records</strong>/information managers being involved is to support the achievement<br />

of maintaining the authenticity and evidential integrity of the public <strong>records</strong> throughout the<br />

migration process and to assist with retention and disposal compliance.<br />

Page 16 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

3.2 Determining what is to be migrated<br />

Migration<br />

project<br />

initiation<br />

Determining<br />

what is to be<br />

migrated<br />

Determining<br />

where & how<br />

to migrate<br />

Ensuring<br />

quality<br />

Performing<br />

migration<br />

Post<br />

migration<br />

A key decision point in a migration project is determining what information will be migrated. To<br />

determine this, there are some fundamental details about the <strong>records</strong> themselves that must first be<br />

established. Understanding these details will help to inform decisions about what data to migrate<br />

and what data does not need to be migrated. The processes for determining this are described<br />

below.<br />

3.2.1 Identifying the <strong>records</strong><br />

Records are the consequence or product of a business activity. Digital <strong>records</strong> are not simply data<br />

but comprise a complex set of related information which includes the:<br />

<br />

<br />

<br />

content of the record<br />

contextual metadata, for example, information about who created the record, how it has<br />

been managed, relationships to other <strong>records</strong>, etc, and the<br />

structure that enables the record to be accessible and meaningful, e.g. database table<br />

headings.<br />

Knowing what the <strong>records</strong> are within a system is critical to protecting and maintaining their<br />

completeness, integrity and authenticity. It is important to clearly scope and document them prior to<br />

undertaking a migration.<br />

Depending on the nature of the migration project, the <strong>records</strong> to be migrated may be easily<br />

identified and can be clearly described. However, identifying <strong>records</strong> in business systems can often<br />

be challenging as they are generally complex and are not always packaged as neatly discrete<br />

objects.<br />

The process of determining what <strong>records</strong> are contained in business systems involves undertaking<br />

an analysis of the business processes associated with the system’s use, in order to identify the<br />

information that <strong>records</strong> the evidence of business activity and transactions. For further advice on<br />

this process see Part 2.3 of the Guidelines and Functional Requirements for Records in Business<br />

Systems. 5<br />

This process of analysis will also highlight ephemera within the system, including any duplicated<br />

copies of information. Ephemera are items of short-term value that are not required to be kept as<br />

<strong>records</strong> and therefore may be able to be disposed of, if not required for other business or legal<br />

purposes, prior to the migration.<br />

5<br />

The use of this guideline has been endorsed by the State Archivist and is accessible from Queensland State Archives’<br />

website. Also refer to the public <strong>records</strong> brief Managing <strong>records</strong> in <strong>digital</strong> environments using ISO 16175:<br />

Information and documentation - Principles and functional requirements for <strong>records</strong> in electronic office environments.<br />

Please note that in 2008 the International Council on Archives (ICA) published three modules of its Principles and<br />

Functional Requirements for Records in Electronic Environment (ICA-Req). These modules have since been adopted by<br />

the International Standards Organisation and published as ISO 16175.<br />

Page 17 of 57


Department of Science, Information Technology and Innovation<br />

Key point:<br />

Know what <strong>records</strong> there are and describe them to ensure the <strong>records</strong> can be successfully<br />

maintained throughout and following the migration process.<br />

3.2.2 Establishing retention obligations<br />

The retention periods for <strong>records</strong> are documented in authorised Retention and Disposal<br />

Schedules. Once the <strong>records</strong> have been identified, an authorised schedule can be consulted and<br />

applied to determine how long they must be kept. Some <strong>records</strong> may have passed their retention<br />

period and can be deleted, which will reduce the volume of <strong>records</strong> requiring migration.<br />

IMPORTANT: The deletion of <strong>records</strong> must be in accordance with the authorised disposal<br />

requirements. It is also important to consider other aspects that may apply to the <strong>records</strong> prior to<br />

deletion, such as whether a disposal freeze is in place or another business or legislative reason<br />

overrides the Retention and Disposal Schedule/s. For example, are the <strong>records</strong> in question<br />

potentially needed in evidence in an existing judicial proceeding, including any reasonably<br />

possible judicial proceeding?<br />

When deletion is to be undertaken in the source system prior to migration, all disposal actions must<br />

be documented in accordance with requirements under Principle 2 of Information standard 31:<br />

retention and disposal of public <strong>records</strong>. For further information about disposing of <strong>records</strong>, see<br />

section 4 of the Guideline for the implementation of retention and disposal schedules.<br />

The disposal of public <strong>records</strong> must be authorised by the State Archivist under section 26 of the<br />

Public Records Act 2002. Such authorisation is usually given in a Retention and Disposal<br />

Schedule. In the event the identified <strong>records</strong> are not covered by an authorised Retention and<br />

Disposal Schedule, <strong>records</strong> cannot be deleted prior to migration.<br />

Key points:<br />

Identify the lawful retention period for the <strong>records</strong> through consultation with authorised Retention<br />

and Disposal Schedule/s and consider any other legal obligations e.g. <strong>records</strong> subject to litigation,<br />

discovery process.<br />

After establishing the lawful retention period, if there are no longer any legal, business or<br />

legislative requirements to retain the <strong>records</strong>, undertake an authorised process to dispose of any<br />

<strong>records</strong> that have met their minimum retention period.<br />

If there is no authorised Retention and Disposal Schedule covering the migrated <strong>records</strong>,<br />

disposing of them is unlawful.<br />

3.2.3 Determining recordkeeping metadata<br />

Recordkeeping metadata is a fundamental tool for creating and maintaining ‘full and accurate’<br />

<strong>records</strong> and is a key means by which the integrity and authenticity of a record can be proven.<br />

A key feature that differentiates recordkeeping metadata from other types of metadata is that it is<br />

not a static profile of a document or other resource. Recordkeeping metadata initially defines a<br />

record at the point of capture, but is also dynamic and accrues through time, to provide<br />

information on how a record has been used or managed. This characteristic of recordkeeping<br />

metadata is essential for preserving the authenticity of <strong>records</strong>.<br />

Page 18 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Safeguarding metadata relationships<br />

Effective migration of <strong>records</strong> requires that the associated metadata including the connections<br />

between a record and its metadata are persistently maintained. This requires an understanding of<br />

the <strong>records</strong>’ existing metadata. Documenting this metadata will help to highlight all the necessary<br />

data that must be migrated, particularly metadata which <strong>records</strong> the information about relationships<br />

and connections between data.<br />

Relationships requiring safeguarding may be:<br />

<br />

<br />

<br />

<br />

<br />

structural – that is within the document, such as a document containing a linked<br />

spreadsheet or images<br />

between <strong>records</strong> – e.g. between <strong>records</strong> documenting related aspects of business<br />

between <strong>records</strong> and/or ‘containers’ – e.g. documents aggregated to files/folders<br />

between <strong>records</strong> and other entities – e.g. between <strong>records</strong> and creating agents<br />

between <strong>records</strong> and control tools such as Business Classification Schemes, Retention and<br />

Disposal Schedules, access and security controls and mandates.<br />

Some of this information may be outside the system in question, for example, the Business<br />

Classification Scheme, however processes will need to be established to ensure the relationships<br />

are successfully maintained with the migrated <strong>records</strong>.<br />

Once the metadata and <strong>records</strong> are understood and documented, the process of mapping the<br />

metadata from source to destination system is essential to ensure that all the necessary metadata<br />

is carried over to the target system and remains useable and linked to the <strong>records</strong> it describes.<br />

This will contribute significantly to the maintenance of the <strong>records</strong>’ context and authenticity.<br />

Metadata and mapping analysis is an opportunity to address any insufficiencies and/or apply<br />

improved metadata capture that ensures that the minimum recordkeeping metadata requirements<br />

are met.<br />

In some migrations the date of the migration can become the default date for all recordkeeping<br />

actions, including date of creation of <strong>records</strong>. This severely impacts recordkeeping processes such<br />

as disposal and undermines the authenticity, useability and integrity of the <strong>records</strong>. Checks should<br />

therefore be in place to ensure dates that trigger recordkeeping actions in the source system<br />

remain accurate and unchanged post migration.<br />

The Queensland recordkeeping metadata standard and guideline (QRKMS) can be used as a tool<br />

to help identify the metadata that requires migration, and the links that must be maintained.<br />

Page 19 of 57


Department of Science, Information Technology and Innovation<br />

For example<br />

The table below contains columns to identify the QRKMS required metadata, the relevant field in<br />

the current system and the target field in the new system. The comments column can be used to<br />

note any system limitations or issues that require resolution before migration.<br />

QRKMS element Current system Target system Comments<br />

Identifier<br />

Transaction<br />

reference<br />

Transaction ID<br />

Check if ID format is supported by<br />

new system, or whether new<br />

format is required<br />

Date created Transaction date Date Must ensure dates carry across<br />

as is from old system and are not<br />

updated<br />

A metadata implementation matrix is available that can be adapted for this purpose.<br />

Capture metadata about migration<br />

When migration of <strong>digital</strong> <strong>records</strong> occurs, it is a key event in the lifecycle of the <strong>records</strong>. Metadata<br />

must be captured to document the process of the migration of the record. The mandatory ‘Record<br />

Event History’ element within the Queensland Recordkeeping Metadata Standard requires<br />

documenting the preservation, retrieval, disposal, control, access or use related activities<br />

performed on a record. This metadata provides a visible and auditable trail of <strong>records</strong> management<br />

actions and decisions, and can be used to record the migration activities on the record.<br />

Other metadata considerations<br />

In addition to specific recordkeeping metadata mentioned above, the system may hold other<br />

metadata that is crucial to the meaning of the <strong>records</strong>. For example, a database containing<br />

demographic data will need information that explains that a “1” in the gender column means the<br />

individual is a man, while a “2” indicates a woman. It is important that all metadata dependencies<br />

are identified when preparing for migration.<br />

It is acknowledged that the export of metadata from systems can be technically challenging. It is<br />

important that the time and effort required to undertake this step is not underestimated. Where it is<br />

not possible to export particular metadata, the effects on the functionality and authenticity of the<br />

<strong>records</strong> must be carefully analysed and documented.<br />

For further advice about recording metadata to manage <strong>digital</strong> <strong>records</strong> over time, see the Metadata<br />

for <strong>digital</strong> continuity: a companion guideline to the Queensland recordkeeping metadata standard.<br />

Key points:<br />

Analyse and document the recordkeeping metadata that exists about the <strong>records</strong>.<br />

Plan for the mandatory capture of metadata about the migration activities to be undertaken on the<br />

record in the line with Queensland Recordkeeping Metadata Standard.<br />

Take the opportunity to improve the capture of metadata about the <strong>records</strong>, if appropriate, by<br />

influencing the design of the target system.<br />

Page 20 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

3.2.4 Understanding characteristics of the <strong>records</strong><br />

Continued access to a record requires preservation of its content and other information and<br />

characteristics if the record is to remain authentic, meaningful and useable.<br />

All migrations carry risk related to the loss of these characteristics, information and content. One of<br />

the aims in any migration project is to minimise the amount of loss. An ideal approach may be to<br />

preserve all the <strong>records</strong>’ characteristics where possible, in order to maximise future access and<br />

use opportunities. However it may not be technically or economically feasible to achieve this, nor<br />

may it be necessary. For example, a new format may not technically be able to support all features<br />

of the preceding format but retains sufficient characteristics for business purposes.<br />

Defining the characteristics of the <strong>records</strong> needs to be driven by business requirements in<br />

consideration of obligatory recordkeeping requirements. This process includes consultation with<br />

the business stakeholders and considering the access and use requirements that current and<br />

potential future users of the <strong>records</strong> may have. Characteristics that enable these requirements can<br />

then be identified and migration decisions made to ensure their maintenance.<br />

Examples<br />

<br />

<br />

If a report contains a map where colours are used to signify different agricultural areas, these<br />

colours have meaning and the report could not be interpreted accurately if these colours were<br />

lost through migration to another format. For this report, the colours are a critical characteristic<br />

and any migration performed on this report must ensure that they are maintained.<br />

An experiment in the United States sought to determine whether complex three-dimensional,<br />

geometric CAD (Computer Aided Design) <strong>records</strong> of high tolerance machined piece parts<br />

could be converted from their native CAD environment to an open source archival format to<br />

facilitate their long-term preservation. The experiment showed that, at the time of the study, it<br />

was not possible to successfully convert all aspects of the <strong>records</strong> to the non-proprietary<br />

format. The open format could not adequately represent the fine accuracy and measurement<br />

levels (down to a millionth of an inch) that were necessary to sustain the accuracy of the<br />

engineering drawing. Because the exacting engineering requirements that were a core<br />

characteristic of the <strong>records</strong> could not be reproduced, a different conversion option was<br />

required. 6<br />

Characteristics may encompass:<br />

<br />

<br />

<br />

Content – information content within the record, e.g. text, still and moving images, audio,<br />

and other intellectual productions.<br />

Context - information that describes the environment in which the content was created and<br />

managed, e.g. creator name, date of creation, file format.<br />

Structure - information that describes the extrinsic or intrinsic relationship between content,<br />

as required to reconstruct the performance of the record, e.g. e-mails and their<br />

attachments.<br />

6<br />

This case study is discussed in L Duranti and R Preston, International research on permanent authentic <strong>records</strong> in<br />

electronic systems (InterPARES) 2: Experiential, interactive and dynamic <strong>records</strong>, 2008, pp. 34-35, Accessible from<br />

http://www.interpares.org/display_file.cfm?doc=ip2_book_complete.pdf<br />

Page 21 of 57


Department of Science, Information Technology and Innovation<br />

<br />

<br />

Appearance/rendering - any information that contributes to the re-creation of the<br />

performance of the record, e.g. font type, colour and size, bit depth.<br />

Behaviour - properties that indicate the method in which content interacts with other stimuli,<br />

e.g. hyperlinks. 7<br />

For example, <strong>records</strong> in business systems may possess the following characteristics: 8<br />

Content the actual content of the database tables<br />

queries used so the required content can be retrieved and represented<br />

Context<br />

metadata identifying:<br />

the name of the creator and creating organisation<br />

the business process(es) that produced the record<br />

date of creation of the record<br />

specified relationships to other <strong>records</strong><br />

original and current file formats<br />

name and version of the query languages that are used<br />

history of the recordkeeping actions performed on the record<br />

Structure the physical structure of the database. This includes:<br />

<br />

<br />

- the tables<br />

- relationships between the tables, including the constraints<br />

- the views<br />

- field attributes<br />

the structural composition of the data as presented onscreen<br />

the logical structure of the database can be preserved in an entity<br />

relationship diagram or an XML schema<br />

Appearance/ rendering the onscreen representation<br />

Behaviour the behaviour of the user application. This may be preserved in the form<br />

of descriptions of system-supported functions, as well as screenshots of<br />

the displays used for entering and amending data, generating reports,<br />

etc. It may also be preserved in system documentation and user<br />

manuals.<br />

Table 3 – Characteristics of <strong>records</strong> in business systems<br />

Where it is not possible to maintain all of the characteristics, a risk analysis of the characteristics<br />

which will and will not be preserved should be undertaken and justified, and the decisions made,<br />

approved and documented. At a minimum, characteristics should be prioritised for preservation by<br />

their capacity to ensure the authenticity, accessibility, usability and meaningfulness of the record<br />

as evidence of business activity.<br />

7<br />

InSPECT Project (Dec 2009) Investigating the significant properties of electronic content over time: Final report.<br />

Accessible from http://www.significantproperties.org.uk/inspect-finalreport.pdf<br />

8<br />

This example has been taken from the Digital Preservation Testbed Project led by the National Archives of the<br />

Netherlands. It should be noted that these characteristics are generic and are based on a specific file format type. There<br />

will be other specific characteristics unique to an agency’s business that will need to be identified and managed for<br />

databases.<br />

Page 22 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Key points:<br />

Assess, document and prioritise the characteristics of <strong>records</strong>.<br />

Consider the characteristics from different business perspectives and usage requirements.<br />

Ensure decisions about acceptable loss and/or change of record characteristics between the<br />

source and target systems are documented, and the consequences for useability, reliability, and<br />

authenticity of the migrated <strong>records</strong> are understood and accepted.<br />

3.3 Determining where and how to migrate<br />

Migration<br />

project<br />

initiation<br />

Determining<br />

what is to be<br />

migrated<br />

Determining<br />

where & how<br />

to migrate<br />

Ensuring<br />

quality<br />

Performing<br />

migration<br />

Post<br />

migration<br />

Following the determination of what requires migration to the target system, the following may<br />

require consideration:<br />

<br />

<br />

<br />

the target system’s recordkeeping options and capabilities<br />

the format into which the <strong>records</strong> will be migrated, and<br />

the overall approach, method and tools required to achieve the migration.<br />

3.3.1 Verifying target system/s recordkeeping capabilities<br />

Options for migration of inactive <strong>records</strong><br />

A key consideration in determining where the <strong>records</strong> will go and how they will be treated is<br />

whether the <strong>records</strong> are required for ongoing business purposes (active <strong>records</strong>) or can be less<br />

immediately accessible (inactive <strong>records</strong>).<br />

This guideline assumes that all active <strong>records</strong> are migrated to an online target system and that<br />

different migration options can be applied to the inactive <strong>records</strong>. The table below contains some<br />

examples with the related issues and outcomes:<br />

Source system Migration options Issues or outcomes<br />

Active <strong>records</strong><br />

Migrate to the new<br />

online target system<br />

<br />

<br />

Maintain business continuity<br />

Preservation monitoring is less of an overhead as<br />

<strong>records</strong> are in the current online environment<br />

Inactive <strong>records</strong><br />

1. Convert and<br />

move to a data<br />

store, repository<br />

or removable<br />

media<br />

<br />

<br />

May be most efficient and sustainable way of<br />

managing inactive <strong>records</strong> that must be retained for<br />

longer period<br />

Requires a conversion process on the <strong>records</strong> into an<br />

accessible format that is not dependent on the<br />

originating software<br />

Page 23 of 57


Department of Science, Information Technology and Innovation<br />

Source system Migration options Issues or outcomes<br />

2. Retain in the<br />

existing and now<br />

off-line system<br />

<br />

<br />

<br />

<br />

<br />

<br />

Business system functionality will not remain<br />

Requires active monitoring of the hardware, media and<br />

file formats for potential obsolescence<br />

Requires active monitoring and management in<br />

compliance with recordkeeping requirements under<br />

IS40 and IS31<br />

Higher risk of loss of accessibility due to technology<br />

obsolescence<br />

Inability to decommission the existing system which<br />

carries associated ongoing costs<br />

Requires ongoing maintenance and monitoring of the<br />

obsolete business system from the hardware, software<br />

and file format points of view<br />

<br />

Requires active monitoring and management in<br />

compliance with recordkeeping requirements under<br />

IS40 and IS31<br />

Table 4 – Inactive <strong>records</strong> migration options<br />

The chosen approach will be influenced by a range of business, recordkeeping and technical<br />

issues. An important initial step to undertake is to understand how the <strong>records</strong> are controlled in the<br />

current environment, for example whether recordkeeping controls are built into the system itself, or<br />

whether the <strong>records</strong> are exported and managed through a separate recordkeeping system.<br />

Ability of target system to meet recordkeeping obligations<br />

Whichever target system is selected, it is important to ensure there are adequate recordkeeping<br />

controls to manage the <strong>records</strong> in line with Information Standards 31 and 40. Refer to Appendix D<br />

for a checklist on the key obligations. The Guidelines and functional requirements for <strong>records</strong> in<br />

business systems provides additional guidance.<br />

Retention periods influencing target system decisions<br />

The retention and disposal obligations attached to the <strong>records</strong> may influence decisions about the<br />

migration approach. For inactive <strong>records</strong> that only need to be kept for relatively short periods of<br />

time beyond the planned migration, transfer into a target online business system and subsequent<br />

disposal of them once their retention period is reached may be the most appropriate choice, as<br />

migrating or storing them elsewhere may not be as cost-effective.<br />

Retention in the old system may also be appropriate, provided there is a commitment to sentence<br />

and dispose of these <strong>records</strong> when they are lawfully able to be disposed and to keeping the legacy<br />

system alive so the <strong>records</strong> remain findable and actionable until they can be disposed of.<br />

However leaving <strong>records</strong> in legacy systems for extended periods without active management to<br />

ensure they remain reliable, useable and authentic will not meet a public authority’s obligations<br />

under the Public Records Act 2002, nor necessarily comply with other legislation such as the Right<br />

to Information Act 2009.<br />

Page 24 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

3.3.2 Making file / record format decisions<br />

Repeated migrations can compromise the integrity, reliability, useability and authenticity of <strong>records</strong>,<br />

particularly those that are required to be retained for long periods of time. To minimise the number<br />

of migrations, conversion of those <strong>records</strong> that are no longer needed for active business purposes<br />

and have lengthy retention periods into an open file format which is not dependent on the<br />

originating software is recommended. 9 Open formats are stable, longer-term formats that are<br />

widely used. They are usually non-proprietary, platform-independent formats with freely available<br />

specifications. 10 The QSA Public Records Brief Advice on choosing file formats for the<br />

management of <strong>digital</strong> <strong>records</strong> contains more detailed information about choosing file formats.<br />

Some examples of open standard formats include:<br />

PDF/A<br />

ODF<br />

HTML<br />

XHTML<br />

JPEG<br />

TIFF<br />

PNG<br />

Portable document format/ archive<br />

OpenDocument Format<br />

Hypertext Markup Language<br />

Extensible Hypertext Markup Language<br />

Joint Photographic Experts Group<br />

Tagged Image File Format<br />

Portable Network Graphics<br />

FLAC<br />

Free Lossless Audio Codec<br />

Table 5 – Examples of open standard formats<br />

This approach still requires an active and ongoing commitment to monitoring and managing these<br />

migrated <strong>records</strong> to ensure their ongoing access.<br />

Appendix F provides an example of a record format conversion decision made by a public authority<br />

when it was required to decommission a business system.<br />

Key points:<br />

Ensure that the target system/s have recordkeeping capabilities and/or will meet the<br />

responsibilities for retention management.<br />

Plan to minimise the need for record migrations by determining a target system/s and record<br />

format(s) that suit the <strong>records</strong>’ retention periods and record types.<br />

Understanding the retention requirements, conversion formats and their associated stability also<br />

involves an active and ongoing commitment to their monitoring and maintenance.<br />

9<br />

It is acknowledged conversion is not the only longer-term preservation approach. There are other techniques such as<br />

emulation that can be applied.<br />

10<br />

State Records New South Wales (2009) Managing Digital Records Guideline: Effectively Manage the Migration of your<br />

Digital Records.<br />

Page 25 of 57


Department of Science, Information Technology and Innovation<br />

3.3.3 Establishing a migration approach<br />

There may be a number of approaches required to successfully migrate <strong>records</strong>. 11 After<br />

determining what is to be migrated as described in section 3.2 and considering the points raised<br />

above, a documented and approved plan needs to be devised which comprehensively outlines the<br />

approaches to be taken for all of the information in the business system.<br />

The plan should describe:<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

both the source and target business system<br />

formats for the information/<strong>records</strong><br />

the tools that will be used to achieve the migration<br />

details of any manual intervention activities<br />

security requirements for the information<br />

business rules and quality assurance procedures, and<br />

roll back strategy for mitigating risks to the <strong>records</strong> if affected by an unforeseen migration<br />

event.<br />

The migration approaches should be appropriately selected based on defined requirements,<br />

informed by the analysis of the <strong>records</strong> and recordkeeping obligations.<br />

Technological tools - Migrations may be facilitated by technological tools for extracting the data<br />

from the existing system, transforming the data and loading the data into the target system. For<br />

example the conversion of <strong>records</strong> to stable open standard formats can be achieved through a<br />

range of freely available tools such as the National Archives of Australia’s Xena. 12<br />

Manual intervention - Migration processes are likely to involve some manual intervention<br />

supported by appropriate business rules and quality assurance procedures. For example, if the<br />

mapping of information from the existing system to the target system proved complex, manual<br />

data entry of some of the information into the new system may be an appropriate approach when<br />

it is supported by appropriate checking processes to ensure the accuracy and completeness of<br />

the entered data.<br />

Key points:<br />

Document and secure approval for the plans that describe what will happen to all of the<br />

information within the system including the corresponding security classifications.<br />

Select tools which can best facilitate the maintenance of full and accurate <strong>records</strong> throughout the<br />

migration process.<br />

11<br />

Refer to Appendix F for examples of approaches taken by some public authorities.<br />

12<br />

Xena is free and open source software developed by the National Archives of Australia to converts <strong>records</strong> to an XMLbased<br />

archival data format. Xena is an acronym meaning Xml Electronic Normalising for Archives. See<br />

http://xena.sourceforge.net/ for further information.<br />

Page 26 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Ensure migration approaches, including any transforming of the data, loading and any reliance<br />

on manual extraction, is supported by clear business rules and quality checking processes.<br />

Ensure the approach includes a tested roll back strategy, as part of risk mitigation planning.<br />

3.4 Ensuring quality<br />

Migration<br />

project<br />

initiation<br />

Determining<br />

what is to be<br />

migrated<br />

Determining<br />

where & how<br />

to migrate<br />

Ensuring<br />

quality<br />

Performing<br />

migration<br />

Post<br />

migration<br />

For a record to retain its evidentiary value, it must be possible to demonstrate that migration has<br />

resulted in a full and accurate reproduction of the source record which has not been altered in an<br />

accidental or unauthorised way since it was made. The General Records Disposal Schedule for<br />

Digital Source Records (QDAN 678) requires that <strong>digital</strong> source <strong>records</strong> are retained until quality<br />

assurance has been undertaken and the results accepted. Trusted systems and quality assurance<br />

processes are therefore a critical component of migration activities.<br />

3.4.1 Preparing <strong>records</strong> for migration<br />

Prior to migration, it is important to take the opportunity to ensure that all <strong>records</strong> and metadata are<br />

as accurate and up-to-date as possible. This will improve the efficiency and quality of the<br />

information migrated. A migration project provides the opportunity to “cleanse” and maximise the<br />

access to data, for example through:<br />

<br />

<br />

<br />

Reviewing the location metadata about <strong>records</strong> when migrating between recordkeeping<br />

systems, for example to ensure people who have left the organisation are not recorded as<br />

holding <strong>records</strong>.<br />

Ensuring all <strong>records</strong> to be migrated are in the source system. If there is a high likelihood<br />

that there are <strong>records</strong> which should be migrated stored on shared directories, email, or<br />

personal and any other systems outside the identified system to be migrated, awareness<br />

activities can be scheduled to encourage transfer of the <strong>records</strong> into the source system<br />

prior to the migration.<br />

Ensuring any <strong>records</strong> stored in encrypted forms are decrypted as appropriate, as<br />

maintaining unnecessary encryption poses significant risks to the continuing useability and<br />

readability of public <strong>records</strong> over time.<br />

As migration can be a resource-intensive activity, it may also be appropriate to minimise its impact<br />

where relevant, by deleting any <strong>records</strong> that have met their minimum lawful retention period prior to<br />

the performance of the migration activity, as outlined in section 3.2.2.<br />

Any data cleansing activity undertaken should be documented and decisions to dispose of <strong>records</strong><br />

prior to migration endorsed and documented as required by IS31, to help ensure the authenticity of<br />

the <strong>records</strong> and to provide justification for decisions to dispose of existing <strong>records</strong> prior to<br />

migration.<br />

Page 27 of 57


Department of Science, Information Technology and Innovation<br />

Key points:<br />

Strive to minimise the overheads associated with migration processes by ensuring <strong>records</strong> and<br />

metadata are accurate and up-to-date and undertaking sentencing projects prior to migration to<br />

dispose of <strong>records</strong> that have met their minimum retention periods.<br />

Document any disposal decisions arising from data cleansing activities prior to migration, and<br />

ensure the documentation is retained as required by the General Retention and Disposal<br />

Schedule (QDAN249).<br />

3.4.2 Testing and validation<br />

To have assurance that migrated <strong>records</strong> retain their full and accurate attributes, testing and<br />

validation that includes recordkeeping considerations must be undertaken, both pre-migration in a<br />

test environment and post-migration in the live system. This will help to minimise and mitigate any<br />

unanticipated and unacceptable results at the ‘go live’ stage of the new system.<br />

A thorough and documented testing and validation process that is undertaken successfully also<br />

provides valuable assurances to a third party as to the authenticity, reliability, integrity and<br />

useability of the <strong>records</strong>. It is also required before source <strong>records</strong> can be disposed of lawfully in<br />

accordance with the General Retention and Disposal Schedule for Digital Source Records<br />

(QDAN678).<br />

The recordkeeping and information professional’s role and responsibilities through the testing and<br />

validation stages will need to be defined and incorporated into the overall project management<br />

planning.<br />

Appropriate tests to be undertaken to ensure the quality of the <strong>records</strong> will need to be devised in<br />

collaboration with the wider project team. A knowledge and understanding of the source and target<br />

states of the <strong>records</strong>, as set out in sections 3.2 and 3.3, will assist in this process. In addition, a<br />

guide to the key recordkeeping related testing and validation checks that should be considered is<br />

provided in Appendix E.<br />

The process will involve working with the project team, to establish and document baseline<br />

information about the <strong>records</strong> themselves and establish recordkeeping benchmark standards and<br />

allowances. Testing procedures and processes should include steps to follow when validation or<br />

testing falls short of the standards set. For example, where more than an agreed percentage of the<br />

total <strong>records</strong> and the required recordkeeping metadata examined are found to be defective, the<br />

project migration phase pauses and a review of technical migration elements are undertaken and<br />

issues resolved before resuming the migration project.<br />

The level of risk to the business if the authenticity of a particular class or group were not effectively<br />

maintained influences the testing nature and scope. For example, testing approaches could<br />

include: scripted and manual testing of migrated information within a business process context;<br />

sampling a broad range, percentage or block of migrated <strong>records</strong> for validation; analysis and<br />

confirmation that migration mapping outcomes have been achieved; log and error-message<br />

checking, etc.<br />

Page 28 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Under the General Retention and Disposal Schedule (QDAN249) there are a number of record<br />

classes that require long term retention with conditional criteria that applies. This would influence<br />

the nature and scope of testing and validation required. The level of validation required should<br />

be considered in the context that includes the possibility of any future activities that could bring<br />

the authenticity of the migrated <strong>records</strong> into question. This is in addition to the ability to be able<br />

to recall and access them in a useable format.<br />

The following are two examples of types of <strong>records</strong> that would need to have a risk analysis<br />

applied to determine the scope and level of testing and validation required:<br />

<br />

<br />

a workers’ compensation incident report concerning an individual that did not involve an<br />

immediate claim that would need to be held on the individual’s personnel file and retained<br />

for 70 years from date of birth of the individual or seven years from date of separation, or<br />

resignation, whichever is later;<br />

<strong>records</strong> relating to the removal and disposal of hazardous materials (including asbestos<br />

and radioactive material) from buildings or structures which require retention for 70 years<br />

after removal of the hazardous materials.<br />

In both of the above cases, if there was a thorough and documented testing and validation<br />

process this would assist in providing evidence to justify the authenticity, reliability and integrity<br />

of the <strong>records</strong> to a third party if this were ever required into the future.<br />

The extent of testing and validation should therefore be commensurate with the risks associated<br />

with the <strong>records</strong> retention requirements and the level of on-going importance required to justify<br />

their authenticity for the life of the <strong>records</strong>.<br />

IMPORTANT: The time and resources needed to undertake testing and validation is often<br />

underestimated. It is important that the <strong>records</strong> manager factors some flexibility into planning for<br />

this time-consuming and resource intensive activity to ensure that it can be undertaken adequately.<br />

Pre-migration testing is strongly recommended to be undertaken within a test environment and<br />

allows for a revised strategy to be devised in the event that adverse effects are observed. For<br />

example, the test may result in an undesirable outcome in its connection with other integrated<br />

software systems and these unanticipated results may be unacceptable to the business and put a<br />

halt to the migration until a solution is devised as part of the overall migration planning. If this were<br />

to occur once the new system had gone live, it could have serious adverse effects on the<br />

stakeholders’ confidence and business operations.<br />

Best practice recommends, and project management methodology calls for, pre and post migration<br />

testing and validation results to be documented and signed off by the appropriate project delegate<br />

at both the pre and post migration phases. As mentioned previously, this documentation will<br />

provide some assurance that the right practices were undertaken and results anticipated were<br />

validated and/or remediation was implemented.<br />

Key points:<br />

A thorough and documented testing and validation process must be undertaken and signed-off so<br />

that assurances can be made to a third party that the authenticity, reliability, integrity and<br />

useability of the <strong>records</strong> have been maintained.<br />

In collaboration with the broader project team members, contribute to the inclusion of<br />

Page 29 of 57


Department of Science, Information Technology and Innovation<br />

recordkeeping baseline standards and the target results/outcomes.<br />

Ensure the extent, range and types of tests are commensurate with the risks and value<br />

associated with the <strong>records</strong>.<br />

Be detailed in your checking for any unanticipated results or consequences from the testing or<br />

migration including any unacceptable affects on integrated systems.<br />

Be aware that time and resources are often underestimated for testing and validation processes.<br />

Ensure that planning includes flexibility of time and resources for this important element.<br />

3.5 Performing migration<br />

Migration<br />

project<br />

initiation<br />

Determining<br />

what is to be<br />

migrated<br />

Determining<br />

where &how<br />

to migrate<br />

Ensuring<br />

quality<br />

Performing<br />

migration<br />

Post<br />

migration<br />

3.5.1 Testing roll back strategy<br />

Prior to migration, it is important that a roll back strategy has been established and thoroughly<br />

tested. Sometimes, despite comprehensive pre-migration efforts and particularly with complex<br />

migrations, unexpected outcomes can occur in migration exercises. To mitigate the risks to the<br />

<strong>records</strong>, due to unexpected and unacceptable outcomes, having a tested roll back option means<br />

that <strong>records</strong> remain protected and business processes can be resumed with the old system until<br />

another migration attempt is made.<br />

3.5.2 Recordkeeping during cut over period<br />

Business arrangements will need to be made to maintain recordkeeping continuity and compliance<br />

during the period when the migration activities are performed up to when the target system<br />

becomes live and available for use. For example certain business transactions may need to be<br />

suspended or recorded manually until the new system is available, with any updates made or<br />

repeated in the new system once it is live.<br />

3.5.3 <strong>Migrating</strong> <strong>records</strong><br />

The process of migration will involve some element of extraction/export, transformation and<br />

import/loading of data into the target system. As outlined in section 3.2.3, metadata should be<br />

captured to record the actions carried out on the data as it progresses through this extraction,<br />

transformation and loading process.<br />

The process of migration in collaboration with members of the broader project team should be<br />

undertaken with the minimum of intervention and time delay as possible. This helps to demonstrate<br />

an unbroken chain of custody and can help to prove the authenticity and reliability of the migrated<br />

<strong>records</strong>, particularly in the event of legal proceedings. The process should be designed to prevent<br />

irreversible activity so in the event that problems arise the migration can be rolled back as<br />

necessary.<br />

To further strengthen the evidential integrity which may be required to be demonstrated in the<br />

event of legal proceedings, documentation should include the actions taken to validate that the<br />

particular equipment and systems used in the migration are in good operating order.<br />

Page 30 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Once <strong>records</strong> have been successfully migrated and all testing/validation processes accepted it is<br />

important that stakeholder expectations are clearly communicated to the business. For example, it<br />

should be clear that the source record is no longer the official record and business rules and<br />

processes should be established to ensure that the source <strong>records</strong> cannot be altered inadvertently<br />

or continue to be used for documenting business activity. Further guidance about the management<br />

of source <strong>records</strong> post-migration is detailed in sections 3.6.1 and 3.6.2.<br />

Key points:<br />

Ensure that a roll back strategy is included in planning and has been tested prior to undertaking<br />

migration activities.<br />

Record information about the process of migration that includes recordkeeping continuity<br />

arrangements during the cut over period.<br />

In collaboration with the broader project team, keep time delays and interventions to a minimum<br />

when moving <strong>records</strong> from the existing system to the target system to strengthen evidential<br />

integrity of the <strong>records</strong> and reduce data updates or repeated actions needed in the new system.<br />

Ensure business rules and processes are put in place to protect the authenticity of the <strong>records</strong>.<br />

3.5.4 Quality assuring the results of the migration<br />

It is necessary to validate the results of the complete migration of the <strong>records</strong> into the live<br />

environment. This is similar in its process to preliminary testing and validation undertaken in the<br />

test environment, prior to go-live, as described in section 3.4.2.<br />

Once again the results should be assessed, documented and a report compiled and signed off by<br />

the assigned project team delegate prior to the completion of the project. This report provides<br />

essential context to <strong>records</strong> that have undergone migration and is a key event in the lifecycle of<br />

<strong>digital</strong> <strong>records</strong>. This documentation should indicate, at a minimum, that the migrated <strong>records</strong> were<br />

inspected, that metadata and characteristics were compared to the original <strong>records</strong> and were intact<br />

in their entirety, that the <strong>records</strong> were properly migrated, and that all necessary control procedures<br />

were applied during the process.<br />

Only after this report attesting to the successful migration of the <strong>records</strong> has been signed-off,<br />

should the disposal authorisation for the <strong>digital</strong> source <strong>records</strong> be activated. Public authorities are<br />

encouraged to maintain this documentation for the life of the record, in order to help demonstrate<br />

the authenticity of the migrated <strong>records</strong>.<br />

Key points:<br />

Testing and validation of recordkeeping functionality post-migration is similar to the process<br />

undertaken pre-migration and requires the same thoroughness.<br />

Only after the post-migration testing and validation report has been signed-off, should the<br />

disposal authorisation for the <strong>digital</strong> source <strong>records</strong> be activated.<br />

Page 31 of 57


Department of Science, Information Technology and Innovation<br />

3.6 Post migration<br />

Migration<br />

project<br />

initiation<br />

Determining<br />

what is to be<br />

migrated<br />

Determining<br />

where & how<br />

to migrate<br />

Ensuring<br />

quality<br />

Performing<br />

migration<br />

Post<br />

migration<br />

3.6.1 Verifying conditions for disposal of source <strong>records</strong><br />

Migration should not immediately cause the destruction of the source <strong>records</strong> that are the subject<br />

of, or input to, those processes. The source <strong>records</strong> generally continue to exist in tandem with the<br />

output of the migration process. An informed business decision therefore needs to be made about<br />

when it is appropriate to dispose of the source <strong>records</strong>.<br />

The General Retention and Disposal Schedule for Digital Source Records (QDAN678) sets out the<br />

minimum conditions for the retention of source <strong>records</strong>. However the period of time for which the<br />

source <strong>records</strong> might require further retention beyond the minimum, should be established by<br />

individual public authorities through a documented risk assessment as necessary.<br />

When conducting the risk assessment, public authorities must consider any other legal or business<br />

continuity issues that may influence the duration for the further retention of the source <strong>records</strong>.<br />

These issues could include, but are not necessarily restricted to, the following:<br />

Verifying conditions for disposal - risk assessment considerations<br />

<br />

<br />

<br />

<br />

<br />

business purpose, litigation potential and value of the <strong>records</strong> and the risks associated with<br />

this, including the potential business, financial and legal implications of any loss of<br />

trustworthiness or access<br />

level of assurance that ‘full and accurate’ <strong>records</strong> have been achieved through the<br />

migration, including the maintenance of the completeness and authenticity of <strong>records</strong><br />

level of assurance that the migrated record is being well managed in a robust recordkeeping<br />

environment<br />

thoroughness of the migration processes, including quality assurance processes and the<br />

size and complexity of the migration<br />

extent of usage and level of integration into the operational environment of the <strong>records</strong>.<br />

(Please note: Records that are likely to be immediately used after migration may require a<br />

shorter retention period as use will quickly highlight any problems. Those <strong>records</strong> that are<br />

unlikely to be used in the immediate period following migration, may need to be considered for a<br />

longer retention that fits into a quality evaluation and validation plan before disposal.)<br />

Table 6 – Risk assessment considerations prior to disposal<br />

During the established period of retention, source <strong>records</strong> must be stored and protected to ensure<br />

that they remain accountable, well-managed <strong>records</strong> of business that can be used again as<br />

appropriate source <strong>records</strong>, should there be a need to roll back and re-initiate migration if issues<br />

arise.<br />

Page 32 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Also during this retention period, isolation of the original source <strong>records</strong> from further access by<br />

general stakeholders is highly recommended to eliminate the possibility of their inadvertent use.<br />

It is recommended that the authorised deletion or disposal of source <strong>records</strong> be performed in a<br />

timely way as part of good business practice in the context of the closure of the migration project.<br />

Key points:<br />

Source <strong>records</strong> cannot be deleted until the conditions outlined in the General Retention and<br />

Disposal Schedule for Digital Source Records (QDAN678) are met.<br />

Undertake a risk assessment to validate the need to retain source <strong>records</strong> for a longer period than<br />

specified in the General Retention and Disposal Schedule for Digital Source Records (QDAN678).<br />

Make sure processes are in place to ensure retained source <strong>records</strong> cannot be confused with or<br />

used for working business transactions.<br />

Deletion of source <strong>records</strong> at the end of the authorised retention period should be scheduled as<br />

part of the project plan closure.<br />

3.6.2 Mandatory requirements before disposing of <strong>digital</strong> ‘source <strong>records</strong>’<br />

The disposal of public <strong>records</strong> including <strong>digital</strong> source <strong>records</strong> requires the authorisation of the<br />

State Archivist under section 26 of the Public Records Act 2002. Authorisation for the disposal of<br />

<strong>digital</strong> source <strong>records</strong> following a successful <strong>records</strong> migration activity is given through the General<br />

Retention and Disposal Schedule for Digital Source Records (QDAN678).<br />

Before <strong>digital</strong> source <strong>records</strong> are deleted under the General Retention and Disposal Schedule for<br />

Digital Source Records (QDAN678), public authorities must ensure that:<br />

<br />

<br />

<br />

agreed quality assurance procedures as described in sections 3.4 and 3.5 have been<br />

conducted and any remediation need has been satisfied,<br />

the verification of the conditions for disposal readiness as outlined in 3.6.1 have been met,<br />

and<br />

their eligibility for disposal signed off by the Chief Executive Officer or their authorised<br />

delegate in line with IS31.<br />

Once the above conditions have been satisfied, the source <strong>records</strong> can be deleted under the<br />

General Retention and Disposal Schedule for Digital Source Records (QDAN678). The disposal of<br />

source <strong>records</strong> must be documented. 13 For disposal of the source <strong>records</strong> in a migration project,<br />

document at system level information about the disposal of the <strong>digital</strong> source <strong>records</strong>.<br />

Approval/endorsement and sign-off of the disposal of the source <strong>records</strong> can, for example be<br />

retained as an administrative record in the public authority’s EDRMS. Note that the General<br />

Retention and Disposal Schedule (QDAN249) requires that master disposal documentation be<br />

retained permanently (reference 9.4.2).<br />

13<br />

See: http://qgcio.qld.gov.au/qgcio/architectureandstandards/informationstandards/current/Pages/IS31-<br />

Implementationadvice.aspx<br />

Page 33 of 57


Department of Science, Information Technology and Innovation<br />

Public authorities should ensure that the source <strong>records</strong> are securely deleted once the approved<br />

retention period has expired so that they cannot be retrieved or recreated. While some business<br />

systems have recordkeeping functionality built-in that includes workflow for an authorised<br />

destruction process, there are many business systems that do not have this specific utility and<br />

secure deletion involves a separate process and documentation. To cover a public authority’s<br />

specific circumstances, it is recommended that the secure and irretrievable deletion process be<br />

incorporated into the overall project planning and be devised in consultation with the broader team<br />

for the project.<br />

As detailed in section 3.6.1, performing a migration results in the existence of two versions of the<br />

same record – the ‘source’ record and the new ‘migrated’ record (Refer to Figure 3). Maintaining<br />

two <strong>records</strong> is not cost effective and may cause confusion as to which record is the official record<br />

of the business. Digital source <strong>records</strong> should therefore be deleted once quality assurance and risk<br />

assessment procedures have been completed in line with the General Retention and Disposal<br />

Schedule for Digital Source Records (QDAN678). At a minimum they should be isolated and made<br />

inaccessible for general use.<br />

Key points:<br />

Source <strong>records</strong> that have been subject to migration to a new target system can only be destroyed<br />

in accordance with the General Retention and Disposal Schedule for Digital Source Records<br />

(QDAN678).<br />

Quality assurance procedures and validation must be completed and signed off by the authorised<br />

delegate prior to deletion.<br />

Disposal must be endorsed by the Chief Executive Officer or their authorised delegate and<br />

information about the source <strong>records</strong> disposal retained as required by IS31 and the General<br />

Retention and Disposal Schedule (QDAN249).<br />

Source <strong>records</strong> must be securely destroyed so that they cannot be retrieved or recreated.<br />

Capturing mandatory recordkeeping documentation<br />

An essential part of all project management methodologies is the documentation and sign-offs on<br />

the finalisation of the process to confirm that the project was completed successfully. In the case of<br />

the migration of public <strong>records</strong> the sign-offs need to include sign-off on the recordkeeping<br />

considerations.<br />

This is important because if migrated <strong>records</strong> were ever called into question during legal<br />

proceedings, it may be necessary to certify that the migration processes were reliable and<br />

produced accurate and authentic <strong>records</strong>. Generally, the greater the risk associated with the need<br />

to rely on the <strong>records</strong> in court or in business transactions, the greater the need to make and keep<br />

adequate documentation that attests to the accuracy and authenticity of the <strong>records</strong>.<br />

A range of documentation will have been produced throughout the project in accordance with the<br />

public authority’s project management methodology. As a guide for <strong>records</strong> and information<br />

managers, documentation of the aspects of the migration project outlined in the Table below is<br />

important for recordkeeping purposes to provide assurance about the integrity of the <strong>records</strong>’<br />

treatment throughout the migration process and the results achieved.<br />

Page 34 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Guide to recordkeeping documentation:<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

descriptions of the identified <strong>records</strong> and their characteristics, and the metadata being<br />

migrated along with reports associated with decisions made<br />

any data cleansing performed, including <strong>records</strong> of decisions and the processes<br />

undertaken<br />

the formal migration plans and processes<br />

the dates/times of the migration activities and the people involved, including their roles and<br />

positions<br />

system configurations, including metadata definitions and mappings<br />

risk assessments and mitigation planning and activities<br />

testing and quality assurance plans and results, pre and post migration<br />

any reports that compare original system functionality to target system functionality<br />

all sign-offs/approvals and all decisions, including any decisions not to migrate certain data<br />

any variations to plans around the recordkeeping considerations<br />

any necessary variation in <strong>records</strong> structure, metadata, format or content that will or has<br />

resulted from the migration<br />

the disposal of the source <strong>records</strong> including approved assurances that the minimum<br />

conditions were met and the appropriate further period of retention had elapsed.<br />

Table 7 – Guide to documentation requirements<br />

Key points:<br />

Projects should not be closed until there are appropriate assurances that the <strong>records</strong> have been<br />

successfully migrated and the original source <strong>records</strong> have been attended to in accordance with<br />

the authorised plan.<br />

Documenting the recordkeeping considerations and results applicable to the <strong>records</strong> migration<br />

process is important as this helps to demonstrate the integrity and authenticity of the <strong>records</strong> in<br />

the future.<br />

3.6.4 Stakeholder relations and continuous improvement<br />

While the migration project may have been completed and signed off by the business, the work of<br />

those responsible for <strong>records</strong> and information management continues. The following pointers<br />

highlight some of those areas for consideration.<br />

Staff training<br />

Although this is not dealt with in detail in this guideline, an important aspect arising from migration<br />

projects for a <strong>records</strong> and information manager to consider is the need to ensure that appropriate<br />

Page 35 of 57


Department of Science, Information Technology and Innovation<br />

training of staff is undertaken. This may require targeted promotion and the development of new<br />

procedures relating to the management of the <strong>records</strong> in the new migrated environment.<br />

Ongoing monitoring<br />

To ensure that the migrated <strong>records</strong> continue to be maintained as expected, and as a mechanism<br />

for identifying opportunities for recordkeeping improvements, ongoing monitoring of system usage<br />

and the system’s treatment of the <strong>records</strong> is required.<br />

Lessons learnt and continuous improvement opportunities<br />

There will be many lessons learned from the migration project that can be applied to future projects<br />

to improve the process and outcomes of <strong>records</strong> migration projects.<br />

By documenting the learnings for potential future reference and having <strong>records</strong> management<br />

expertise involved in business system procurement and implementation projects, ongoing<br />

recordkeeping considerations can be included and ongoing business benefits can be achieved.<br />

Key points:<br />

Migration projects are an opportunity to improve recordkeeping practices.<br />

Recordkeeping training material may need updating.<br />

Monitor ongoing usage of the system to ensure <strong>records</strong> continue to be managed in accordance<br />

with requirements.<br />

Page 36 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Appendices<br />

Attached supporting materials for the <strong>records</strong> and information manager<br />

To further assist recordkeeping and information professionals in their engagement with<br />

recordkeeping considerations during a migration project, various supporting documents are<br />

appended here as quick reference materials. These are referred to in the text where appropriate<br />

and include:<br />

Appendix A<br />

Appendix B<br />

Appendix C<br />

Appendix D<br />

Appendix E<br />

Appendix F<br />

IS40 Principle 7 and the migration of <strong>digital</strong> <strong>records</strong> – a guide<br />

Records manager’s project stages flow chart<br />

Records manager’s migration project check list<br />

Recordkeeping controls in the target system – a summary<br />

Testing and validation check list<br />

Lessons learnt – migration hints and tips for recordkeeping professionals<br />

Page 37 of 57


Department of Science, Information Technology and Innovation<br />

Appendix A – IS40 Principle 7 and the migration of <strong>digital</strong> <strong>records</strong><br />

The following is a guide to the application of IS40 Principle 7 to the migration of <strong>digital</strong> <strong>records</strong>.<br />

To be a full and accurate record a public record must, amongst other requirements, be:<br />

Attribute Standard meaning Digital migration context<br />

Complete<br />

Meaningful<br />

Accurate<br />

Authentic<br />

Inviolate<br />

Complete <strong>records</strong> comprise contextual<br />

and structural data as well as content.<br />

Contextual data is information about the<br />

creation and use of the record, including<br />

the business processes that give rise to<br />

it. Structural data includes formal<br />

internal structures of the information<br />

object and the structural relations<br />

between <strong>records</strong>. Content is the<br />

information contained within the<br />

information object.<br />

Meaningful <strong>records</strong> may be understood<br />

in the context of the processes and<br />

business for which they are created and<br />

in which they are used.<br />

An accurate record reflects what was<br />

communicated, decided or done (or not<br />

done). That is, the record’s content,<br />

context and structure can be trusted as<br />

a true and accurate representation of<br />

the transactions, activities or facts that<br />

they document and can be depended<br />

upon in the course of subsequent use.<br />

An authentic record is a record that can<br />

be proven and trusted to be what it<br />

purports to be and to have been<br />

created, used, transmitted or held by the<br />

person to whom these actions have<br />

been attributed. The maintenance of<br />

authentic <strong>records</strong> is facilitated by wellplanned,<br />

comprehensive and successful<br />

migration processes and metadata<br />

capture.<br />

Inviolate <strong>records</strong> are time-bound (that is<br />

they are fixed at a point in time) and<br />

A complete migrated record is an<br />

accurate replica of the source record,<br />

including all the record’s content,<br />

context and structure i.e. metadata.<br />

For <strong>digital</strong> <strong>records</strong> to be meaningful,<br />

metadata that <strong>records</strong> the history of the<br />

recorded information’s use is required to<br />

ensure it is understood in a factual<br />

context.<br />

An accurate migrated record is one that<br />

remains a reliable representation of the<br />

business activity after the migration.<br />

An authentic migrated record is one<br />

where it can be demonstrated,<br />

particularly through the capture of<br />

metadata about the migration process,<br />

that it maintains its trustworthiness as<br />

being what it purports to be. The<br />

requirement for authenticity means that<br />

any migration must be managed and<br />

documented with care so that the<br />

reasons for the process are explicit, the<br />

method used to validate the quality of<br />

the record following the process is clear,<br />

and the metadata detailing who did what<br />

and when is recorded and maintained.<br />

An inviolate migrated record is one<br />

where the content data acknowledges<br />

Page 38 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Attribute Standard meaning Digital migration context<br />

complete. To be inviolate, a record must<br />

be securely maintained to prevent<br />

alteration and unauthorised removal.<br />

Where alteration or changes must<br />

occur, the inviolate nature of the <strong>records</strong><br />

can be protected through the capture of<br />

metadata about the alterations.<br />

its time-bound elements (fixed at the<br />

point in time and evidence of business<br />

activity it is recording) and where it can<br />

be demonstrated that no unauthorised<br />

changes have been made to it. If<br />

changes are authorised, these changes<br />

and the authorisation are recorded in<br />

metadata.<br />

Accessible<br />

Useable<br />

Accessible <strong>records</strong> are <strong>records</strong> that can<br />

be identified, located and viewed as<br />

required.<br />

Useable <strong>records</strong> are those that may be<br />

viewed and are functional and reuseable.<br />

An accessible migrated record is one<br />

that is captured in a format, media and<br />

system in which the current<br />

technologies will enable timely retrieval<br />

and readable access of that migrated<br />

record<br />

A useable migrated record is one that<br />

can be used with the appropriate level<br />

of functionality required to provide a<br />

factual and reliable interpretation of the<br />

source record. For example, if the<br />

hyperlinks do not successfully work after<br />

migration, and these links have been<br />

determined as being an essential<br />

characteristic of the record, then the<br />

record is not “useable”.<br />

Page 39 of 57


Department of Science, Information Technology and Innovation<br />

Appendix B – Records manager’s project stages flowchart<br />

Project planning<br />

1. Migration project<br />

initiation<br />

Roles and<br />

responsibilities<br />

Stakeholder<br />

communications<br />

Identifying the<br />

<strong>records</strong><br />

2. Determining what<br />

is to be migrated<br />

Establishing<br />

retention<br />

obligations<br />

Determining<br />

recordkeeping<br />

metadata<br />

Understanding the<br />

characteristics of<br />

the <strong>records</strong><br />

3. Determining<br />

where and how to<br />

migrate<br />

Verifying target<br />

system/s<br />

recordkeeping<br />

capabilities<br />

Making file/record<br />

format decisions<br />

Establishing<br />

migration<br />

approach<br />

Page 40 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Preparing <strong>records</strong><br />

for migration<br />

4. Ensuring quality<br />

Testing and<br />

validation<br />

Testing roll back<br />

strategy<br />

5. Performing<br />

migration<br />

Recordkeeping<br />

during cut over<br />

period<br />

Migration of<br />

<strong>records</strong><br />

Quality assuring<br />

the results of the<br />

migration<br />

6. Post migration<br />

Verifying<br />

conditions for<br />

disposal of source<br />

<strong>records</strong><br />

Disposal of <strong>digital</strong><br />

‘source <strong>records</strong>’<br />

Capture of<br />

mandatory<br />

recordkeeping<br />

documentation<br />

Stakeholder<br />

relations and<br />

continuous<br />

improvement<br />

Page 41 of 57


Department of Science, Information Technology and Innovation<br />

Appendix C – Records manager’s migration project checklist<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

1. Migration project<br />

initiation<br />

The initial stage of highlevel<br />

scheduling and<br />

resource planning for the<br />

project.<br />

Project planning ensure that recordkeeping obligations and<br />

activities are well understood by the project<br />

team and are incorporated into the project<br />

brief and other project initiation<br />

documentation e.g. project stage plans.<br />

<br />

recordkeeping matters for consideration<br />

include:<br />

3.1.1<br />

o<br />

o<br />

risk mitigation planning is commensurate<br />

with the level of value associated with<br />

the <strong>records</strong> e.g. vital <strong>records</strong>, permanent<br />

value <strong>records</strong>, etc<br />

appropriate scheduling of the associated<br />

<strong>records</strong> related activities is incorporated<br />

into the project stage plan (as listed in<br />

the rest of this check list).<br />

Roles and responsibilities Records management role may include:<br />

3.1.2<br />

<br />

<br />

<br />

the appraisal / identification of <strong>records</strong>, how<br />

long the <strong>records</strong> must be retained and which<br />

<strong>records</strong> are eligible for lawful disposal<br />

determining whether the system is currently<br />

managing the <strong>records</strong> or whether this is<br />

through another recordkeeping system or<br />

combination of systems<br />

contribute to the mapping process between<br />

the existing and target system to ensure<br />

Page 42 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

recordkeeping elements are included,<br />

including the relevant metadata to maintain<br />

the authenticity of the <strong>records</strong> and that they<br />

will be ‘full and accurate’ in their<br />

characteristics<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

<br />

contribute to the design or configuration of<br />

the target system to ensure that adequate<br />

recordkeeping controls are established<br />

<br />

facilitate the process for the authorised<br />

disposal of eligible <strong>records</strong><br />

<br />

participate in testing and validation<br />

processes<br />

<br />

support the process of the quality assurance<br />

of migrated <strong>records</strong><br />

<br />

provide training and awareness in <strong>records</strong><br />

management procedures<br />

<br />

advocate for continuous improvement of<br />

recordkeeping functionality within business<br />

system/s<br />

<br />

Stakeholder<br />

communications<br />

<br />

engage with the overall project’s<br />

communications strategy<br />

3.1.3<br />

<br />

ensure that recordkeeping considerations<br />

are integrated into planning e.g. in<br />

communications and system training<br />

Page 43 of 57


Department of Science, Information Technology and Innovation<br />

Stages for <strong>records</strong><br />

manager involvement<br />

2. Determining what is<br />

to be migrated<br />

The stage of the project<br />

when the scope of the<br />

migration is being<br />

established.<br />

Key recordkeeping considerations<br />

Identifying the <strong>records</strong> undertake an analysis of the business<br />

processes<br />

<br />

Establishing retention<br />

obligations<br />

<br />

<br />

<br />

identify information that <strong>records</strong> evidence of<br />

business transactions (the <strong>records</strong>)<br />

disposal of public <strong>records</strong> must be<br />

authorised and given under a Retention and<br />

Disposal Schedule that has been approved<br />

by the State Archivist for use by the public<br />

authority.<br />

determine and document the lawful retention<br />

obligations that apply to the <strong>records</strong> in<br />

accordance with the authorised Retention<br />

and Disposal Schedule/s applicable to the<br />

public authority.<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

3.2.1 Part 2.3 Guidelines and<br />

Functional Requirements for<br />

Records in Business Systems<br />

3.2.2 Guideline for the implementation<br />

of retention and disposal<br />

schedules<br />

Information standard 31:<br />

retention and disposal of public<br />

<strong>records</strong><br />

General Retention and Disposal<br />

Schedule (QDAN249)<br />

<br />

<br />

<br />

determine which <strong>records</strong> have passed their<br />

retention period and that are eligible for<br />

authorised disposal.<br />

after establishing the lawful retention period,<br />

and if time and resources allow and there<br />

are no other legal, business or legislative<br />

requirements to retain the <strong>records</strong>,<br />

undertake an authorised process to dispose<br />

of any <strong>records</strong> that have met their minimum<br />

retention period.<br />

if disposal of eligible <strong>records</strong> has occurred<br />

prior to migration, capture metadata about<br />

the <strong>records</strong> that have been disposed in<br />

accordance with IS31. The documentation<br />

about the disposed public <strong>records</strong> must be<br />

Page 44 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

retained permanently.<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

<br />

Determining recordkeeping<br />

metadata<br />

<br />

<br />

<br />

analyse and document the recordkeeping<br />

metadata that exists about the <strong>records</strong><br />

plan for the capture of metadata about the<br />

migration activities to be undertaken on the<br />

<strong>records</strong><br />

if possible and appropriate, improve the<br />

capture of metadata about the <strong>records</strong> by<br />

contributing to the design and configuration<br />

decisions about the target system.<br />

3.2.3 Queensland Recordkeeping<br />

Metadata Standard and<br />

Guideline (QRKMS)<br />

Metadata for <strong>digital</strong> continuity: a<br />

companion guideline to the<br />

Queensland recordkeeping<br />

metadata standard<br />

<br />

Understanding the<br />

characteristics of the<br />

<strong>records</strong><br />

<br />

analyse and document the characteristics of<br />

the <strong>records</strong> including: content, context,<br />

structure, appearance/rendering,<br />

behavioural properties (e.g. interactions with<br />

other software/information, hyperlinks, etc).<br />

3.2.4<br />

Appendix A<br />

Appendix<br />

F.2<br />

<br />

consider the characteristics from different<br />

business perspectives and usage<br />

requirements.<br />

<br />

advise on the priority of the characteristics<br />

that need to be carried over with the<br />

migration.<br />

<br />

at a minimum, those characteristics that<br />

contribute to the authenticity, access,<br />

usability and meaningfulness of the <strong>records</strong><br />

should be preserved.<br />

<br />

undertake a risk analysis of the<br />

characteristics which will or will not be<br />

Page 45 of 57


Department of Science, Information Technology and Innovation<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

carried over, and justify and document<br />

decisions about acceptable change and/or<br />

loss.<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

3. Determining where<br />

and how to migrate<br />

The stage when decisions<br />

are being explored and<br />

made about the approach,<br />

method for the migration<br />

to the target system(s).<br />

<br />

Verifying target system/s<br />

recordkeeping capabilities<br />

<br />

<br />

in collaboration with the relevant project<br />

team members, explore the migration<br />

options for active and inactive <strong>records</strong> and<br />

ascertain how recordkeeping compliance<br />

can be maintained for each.<br />

gain an understanding of the ability of target<br />

system/s to meet recordkeeping obligations.<br />

3.3.1 Information Standard 31:<br />

Retention and Disposal of Public<br />

Records<br />

Information Standard 40:<br />

Recordkeeping<br />

<br />

note that retention periods that apply to the<br />

<strong>records</strong> may have an influence on target<br />

system decisions.<br />

<br />

Making file / record format<br />

decisions<br />

<br />

minimise the need for record migrations by<br />

determining record format(s) that take into<br />

account the <strong>records</strong>’ retention periods and<br />

the record types<br />

3.3.2<br />

Appendix<br />

F.1<br />

Storage for <strong>digital</strong> <strong>records</strong><br />

<br />

Establishing migration<br />

approach<br />

Contribute to the:<br />

<br />

documentation and approval of the plans<br />

that describe what will happen to all the<br />

information within the system including the<br />

corresponding security classifications<br />

3.3.3<br />

Appendix<br />

F.3<br />

<br />

<br />

selection of tools which can best facilitate<br />

the maintenance of full and accurate <strong>records</strong><br />

throughout the migration process<br />

development of the business rules and<br />

quality checking processes to support the<br />

Page 46 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

chosen migration approach/es, including any<br />

transforming of the data, loading and any<br />

reliance on manual extraction.<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

4. Ensuring quality<br />

The stage when quality<br />

assurance, including<br />

testing and validation<br />

processes, are identified,<br />

planned and undertaken.<br />

<br />

Preparing <strong>records</strong> for<br />

migration<br />

Contribute to the minimisation of the overheads<br />

associated with migration processes by:<br />

<br />

<br />

ensuring <strong>records</strong> and metadata are accurate<br />

and up-to-date<br />

undertaking sentencing projects prior to<br />

migration to dispose of <strong>records</strong> that have<br />

met their retention periods<br />

3.4.1<br />

<br />

documenting any data cleansing and<br />

disposal activities in accordance with the<br />

requirements of IS31.<br />

Testing and validation Contribute to the:<br />

3.4.2<br />

<br />

<br />

<br />

<br />

development of testing that incorporates the<br />

maintenance of the authenticity of the full<br />

and accurate <strong>records</strong><br />

extent, range and types of testing that is<br />

commensurate with the risks and value<br />

associated with the <strong>records</strong><br />

documenting of the testing and validation<br />

plans, including baseline information and<br />

target results / outcomes<br />

engagement of appropriate recordkeeping<br />

skilled people within the migration stage<br />

Appendix E<br />

Page 47 of 57


Department of Science, Information Technology and Innovation<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

<br />

sign-off process by ensuring that where<br />

appropriate recordkeeping elements are<br />

signed-off by an authorised delegate in<br />

addition to any other authorised sign-offs<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

<br />

<br />

provision of advice on adequate time and<br />

resources to be allocated to the testing and<br />

validation of recordkeeping elements<br />

including the recommendation of conducting<br />

testing on copies of <strong>records</strong> within a test<br />

environment prior to migration<br />

documentation of the recordkeeping<br />

elements of the testing and validation<br />

processes and outcomes.<br />

5. Performing migration<br />

The stage concerned with<br />

the actual undertaking of<br />

the migration to extract /<br />

export, transfer and load /<br />

import the <strong>records</strong> into the<br />

target system(s).<br />

Testing roll back strategy check that roll back strategy has been tested<br />

prior to migration exercise<br />

<br />

Recordkeeping during cut<br />

over period<br />

<br />

prior to performing the migration, ensure that<br />

procedures are in place to protect<br />

recordkeeping elements during the period of<br />

cut over to the new system<br />

3.5.1<br />

3.5.2<br />

Appendix F<br />

<br />

as some system transactions may have<br />

been suspended or recorded manually<br />

during this period, factor in time and<br />

resources to perform the system updates<br />

Migration of <strong>records</strong> Contribute to:<br />

3.5.3<br />

<br />

<br />

documentation of the recordkeeping<br />

elements of the migration process<br />

engagement of appropriate recordkeeping<br />

Page 48 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

skilled people within the migration stage<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

<br />

<br />

development of recordkeeping business<br />

rules and the establishment of subsequent<br />

processes to protect the authenticity and<br />

integrity of the <strong>records</strong><br />

supporting the minimisation of time delays<br />

and interventions.<br />

<br />

Quality assuring the results<br />

of the migration<br />

Similar to the testing and validation stage above,<br />

contribute to the:<br />

3.5.4<br />

<br />

<br />

<br />

<br />

<br />

<br />

development of testing that incorporates the<br />

maintenance of the authenticity of the full<br />

and accurate <strong>records</strong><br />

extent, range and types of testing that is<br />

commensurate with the risks and value<br />

associated with the <strong>records</strong><br />

documenting of the testing and validation<br />

plans, including baseline information and<br />

target results / outcomes<br />

engagement of appropriate recordkeeping<br />

skilled people within the migration stage<br />

sign-off process by ensuring that where<br />

appropriate recordkeeping elements are<br />

signed-off by an authorised delegate in<br />

addition to any other authorised sign-offs<br />

provision of advice on adequate time and<br />

resources to be allocated to the testing and<br />

validation of recordkeeping elements<br />

Page 49 of 57


Department of Science, Information Technology and Innovation<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

including the recommendation of conducting<br />

testing on copies of <strong>records</strong> within a test<br />

environment prior to migration<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

<br />

documentation of the recordkeeping<br />

elements of the testing and validation<br />

processes and outcomes.<br />

6. Post migration<br />

The final recordkeeping<br />

considerations prior to<br />

project closure<br />

<br />

Verifying conditions for<br />

disposal of source <strong>records</strong><br />

Contribute to ensuring that recordkeeping<br />

considerations are incorporated into:<br />

<br />

processes that eliminate any business user<br />

confusion, access or potential inadvertent<br />

use of the now redundant source <strong>records</strong><br />

3.6.1 General Retention and Disposal<br />

Schedule for Digital Source<br />

Records (QDAN678)<br />

<br />

review and update the risk level attached to<br />

the <strong>records</strong> that have been subject to<br />

migration, based on the success of the<br />

validation process, and confirm the lawful<br />

period of retention for the original source<br />

<strong>records</strong> left behind in the old home location,<br />

and<br />

<br />

once conditions for retention have been met,<br />

source <strong>records</strong> can be deleted in<br />

accordance with the General Retention and<br />

Disposal Schedule for Digital Source<br />

Records<br />

<br />

Disposal of <strong>digital</strong> ‘source<br />

<strong>records</strong>’<br />

<br />

source <strong>records</strong> can only be destroyed in<br />

accordance with the General Retention and<br />

Disposal Schedule for Digital Source<br />

Records<br />

3.6.2 General Retention and Disposal<br />

Schedule for Digital Source<br />

Records (QDAN678)<br />

<br />

quality assurance procedures must be<br />

Page 50 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

completed and signed off by the appropriate<br />

delegate<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

<br />

disposal of the source <strong>records</strong> must first be<br />

endorsed by the Chief Executive Office or<br />

their authorised delegate<br />

<br />

disposal must be documented and<br />

documentation must capture the minimum<br />

metadata requirements as outlined under<br />

IS31<br />

<br />

once these conditions have been satisfied,<br />

source <strong>records</strong> must be securely destroyed<br />

so that they cannot be retrieved or recreated<br />

<br />

Capture mandatory<br />

recordkeeping<br />

documentation<br />

<br />

the greater the risk associated with the need<br />

to rely on the <strong>records</strong> in court or in business<br />

transactions, the greater the need to make<br />

and keep adequate documentation that<br />

attests to the accuracy and authenticity of<br />

the <strong>records</strong><br />

3.6.3 Information Standard 31:<br />

Retention and Disposal of Public<br />

Records<br />

<br />

the disposal of source <strong>records</strong> post migration<br />

must be documented in line with IS31<br />

<br />

projects should not be closed until there are<br />

appropriate assurances that the<br />

recordkeeping considerations have been<br />

satisfied<br />

<br />

documenting the migration projects<br />

recordkeeping considerations and the<br />

associated results will help to demonstrate<br />

the integrity and authenticity of the <strong>records</strong><br />

Page 51 of 57


Department of Science, Information Technology and Innovation<br />

Stages for <strong>records</strong><br />

manager involvement<br />

Key recordkeeping considerations<br />

Guideline<br />

section<br />

reference<br />

Supporting<br />

guidance available on QSA<br />

Website<br />

<br />

Stakeholder relations and<br />

continuous improvement<br />

<br />

follow-up with your stakeholders to confirm<br />

system results and gauge user adaptation<br />

3.6.4<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

ensure that training and awareness<br />

materials are up-to-date<br />

ensure that communications and awareness<br />

sessions are completed<br />

document the recordkeeping related lessons<br />

learned from the migration project<br />

monitor ongoing usage and management of<br />

the system and the treatment of the <strong>records</strong><br />

by the new system and its business users<br />

contribute to continuing education of staff in<br />

recordkeeping matters<br />

continue to promote opportunities for<br />

business process improvement in<br />

recordkeeping matters<br />

maintain relationship/s with system owner<br />

and business users and ensure that<br />

recordkeeping responsibilities continue to be<br />

assigned<br />

Page 52 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Appendix D – Recordkeeping controls in the target system<br />

Key recordkeeping considerations for the target system<br />

can protect the<br />

integrity,<br />

authenticity and<br />

context of the<br />

<strong>records</strong><br />

is reliable and<br />

secure<br />

This includes the maintenance of recordkeeping metadata connections.<br />

<strong>Migrating</strong> <strong>records</strong> without effective recordkeeping controls, for example<br />

migrating <strong>records</strong> onto a shared drive, may not capture adequately the<br />

contextual metadata (e.g. author, version history, true date of creation,<br />

etc) required to maintain the authenticity and integrity of the <strong>records</strong>.<br />

Under Information Standard 40, all systems used to create or maintain<br />

<strong>records</strong> must work reliably and be secure to ensure that <strong>records</strong> are<br />

credible and authoritative. Public authorities are therefore required to<br />

implement systems which are secure from unauthorised access,<br />

damage and misuse. This requires supporting processes and tools that<br />

prevent the corruption or loss of the <strong>digital</strong> <strong>records</strong>, e.g. backup/restore<br />

procedures, disaster recovery procedures, sampling of media and <strong>digital</strong><br />

<strong>records</strong> to detect deterioration and corruption, and the refreshing of<br />

media.<br />

An ability to ensure the reliability and security of the <strong>records</strong> will also<br />

influence choices made about the storage medium of <strong>records</strong>. For<br />

example, storage of <strong>records</strong> on RAID (Redundant Array of Independent<br />

Disks) servers may be considered more secure and reliable than<br />

storage of <strong>records</strong> on removable media such as CDs, or DVDs.<br />

can enable access<br />

to the <strong>records</strong><br />

Under section 14 of the Public Records Act 2002, public authorities<br />

controlling <strong>records</strong> that can only be produced or made available with the<br />

use of particular equipment or information technology must take all<br />

reasonable action to ensure the information remains able to be<br />

produced or made available.<br />

At a minimum, because of technology obsolescence, the selected<br />

system should have the ability to easily export <strong>records</strong> so that at a future<br />

point in time when the system’s lifespan has been reached and<br />

decommissioning occurs, the <strong>records</strong> are able to be transferred again to<br />

a successor for continued access if required.<br />

Some approaches will enable easier and wider access to <strong>records</strong> than<br />

others. For example, <strong>records</strong> in offline systems that become legacy<br />

systems are arguably less accessible than other online <strong>records</strong> or<br />

<strong>records</strong> stored in accessible file formats.<br />

To be able to access a record requires that first the <strong>records</strong> are<br />

structured and stored in a way that they can be searched for and are<br />

discoverable. The ability of the system to facilitate discoverability would<br />

factor in the selection of a target system, as a system that does not<br />

enable discoverability prevents access to the <strong>records</strong>. For example,<br />

converting <strong>records</strong> and storing them in a data store that does not<br />

provide adequate searching facilities will not enable access to the<br />

<strong>records</strong>.<br />

Page 53 of 57


Department of Science, Information Technology and Innovation<br />

Key recordkeeping considerations for the target system<br />

The lifespan of the system and associated technologies, software<br />

licensing arrangements or any other constraints which may limit access,<br />

should also be considered when selecting an appropriate target system.<br />

can preserve the<br />

characteristics of<br />

the <strong>records</strong><br />

The characteristics of the <strong>records</strong> must be maintained throughout a<br />

migration process, as mentioned in section 3.2.4. The ability to<br />

maximise the extent of the characteristics that can be preserved may<br />

influence the choice of file format or system. Should a particular file<br />

format not be able to maintain, for example, the appearance of the<br />

content of the record, and the appearance is critical to the meaning of<br />

the record, it may not be an appropriate format to choose.<br />

Page 54 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

Appendix E – Testing and validation checklist 14<br />

Details<br />

Guideline<br />

reference if<br />

applicable<br />

□ Check accuracy of the content and metadata of the <strong>records</strong>. For example:<br />

expected fields are populated with data, e.g. mandatory value 3.2.3<br />

<br />

<br />

field values are in keeping with expected lengths and valid date ranges<br />

values are populated with expected data types (e.g. numeric,<br />

alphanumeric, date, etc.)<br />

Appendix F.2<br />

<br />

metadata values remain accurate, particularly those such as the date of<br />

creation and dates which trigger disposal countdowns<br />

accurate metadata about the migration process itself is captured 3.2.4<br />

□ Check for completeness – that all <strong>records</strong> identified for migration, have been<br />

migrated and that the completeness of the <strong>records</strong> themselves is verified.<br />

For example:<br />

<br />

numbers of <strong>records</strong> remains the same<br />

Appendix A<br />

linkages/relationships between metadata and <strong>records</strong> are maintained Appendix F.2<br />

<br />

results of standard reports and searches are repeatable, consistent, and<br />

accurate<br />

□ Check for maintenance of the integrity and authenticity of the <strong>records</strong>. For<br />

example:<br />

<br />

all agreed characteristics of the <strong>records</strong> as previously identified for<br />

migration during the planning process are maintained<br />

3.2.4<br />

<br />

any errors or corruption during transmission are flagged, e.g. as identified<br />

through checksums – an IT computational process that can highlight<br />

errors that might have occurred during the migration process.<br />

□ Check for meaningfulness and the access and usability of the <strong>records</strong>. For<br />

example:<br />

Appendix A<br />

<br />

real time checks of a sample of the <strong>records</strong> to ensure they remain<br />

readable and understandable<br />

□ Check the reliability of the system/s including functionality, processes, and<br />

integrated system/s and peripherals. For example:<br />

<br />

after roll-out the system is stable and operating within acceptable error<br />

log margins that have been established in collaboration with IT<br />

colleagues<br />

14<br />

A guide only. The individual example items in this checklist should be adapted and expanded to suit the public<br />

authority’s specific migration project and developed in collaboration with IT colleagues.<br />

Page 55 of 57


Department of Science, Information Technology and Innovation<br />

Appendix F – Lessons learnt – hints and tips<br />

The following hints and tips have been provided by <strong>records</strong> and information managers from public<br />

authorities that have been involved with <strong>records</strong> migration projects.<br />

F.1 File format decisions<br />

When migrating documents from an old system that was marked for decommissioning, single page<br />

tiff files were identified that were bound together by the proprietary software. Once migrated, these<br />

documents would have separated into single pages causing many access issues and creating<br />

thousands more documents than were necessary. The public authority was able to copy them,<br />

optical character recognition (OCR) them, convert them to single (multi-page) PDF documents<br />

prior to placing them into the new eDRMS, along with the treatment history of these <strong>records</strong> to<br />

ensure their authenticity could be verified.<br />

Overall there were significant benefits to the business in undertaking this exercise. These included:<br />

considerable improvements to the searchability and accessibility of these <strong>records</strong>, and<br />

maintenance of the <strong>digital</strong> continuity of the <strong>records</strong>.<br />

F.2 Complexities of determining what is to be migrated and what is to be tested and<br />

validated<br />

Identifying the full characteristics of the <strong>records</strong> within the business system and their<br />

dependencies and relationships with data elements can be complex. Records managers need<br />

to work with their IT colleagues to ensure that the <strong>records</strong> to be migrated will retain their<br />

authenticity and remain full and accurate. Decisions made need to be justified and<br />

documented to ensure that the integrity of the <strong>records</strong> can be substantiated.<br />

For example:<br />

Data elements in a database record may be:<br />

mandatory or not<br />

constrained to specific data types (e.g. date, text or number) and character lengths<br />

limited to a set of allowed values, and/or<br />

default values.<br />

There may also be complex editing rules. For example, if element A has a value of X then<br />

element B may be restricted to a particular set of values, etc.<br />

These attributes should be documented and included in validation testing.<br />

Sets of allowed values are often rationalised or updated in migration processes. These changes<br />

and the justifications for these changes should be documented in the migration process.<br />

Relationships between <strong>records</strong> can be constrained. They may be one to one, one to many,<br />

many to many or reflexive. For example, an employee may be the manager or supervisor of<br />

another employee. These constraints may be important in the design of the migration<br />

processes and the validation testing of the new system.<br />

Page 56 of 57


<strong>Migrating</strong> <strong>digital</strong> <strong>records</strong>: A guideline for Queensland public authorities<br />

F.3 Approaches to migrating <strong>digital</strong> <strong>records</strong> and the recordkeeping implications<br />

Within the planning phases for a migration project, a roll back strategy is highly recommended<br />

in order to mitigate any risks that may occur if unacceptable deficiencies, despite pre-migration<br />

testing, come to light after the live/production migration.<br />

The roll back strategy should itself be tested and validated before final migration takes place,<br />

and should include recordkeeping functionality testing.<br />

In addition, the following migration approaches require careful consideration and action planning by<br />

the <strong>records</strong> manager in collaboration with their IT colleagues:<br />

<br />

<br />

<br />

<br />

<br />

A cut over strategy could be planned. Ideally use of the source system should cease and<br />

the source system be frozen before migration commences. Business transactions may be<br />

suspended or recorded manually until the data is migrated and the new system is up and<br />

running.<br />

If the system is critical to the functioning of the business, a snapshot may need to be taken.<br />

Data additions, changes or deletions or functions performed must be logged so the changes<br />

can be repeated in the new system.<br />

Certain non critical functions in the source system might be suspended during the cut<br />

over period to simplify this process. For example, only new <strong>records</strong> may be allowed to be<br />

entered.<br />

Changes to existing <strong>records</strong> may be manually recorded and performed in the new<br />

system once the initial migration is validated and signed off.<br />

New <strong>records</strong> could be migrated to the new system in a second step to synchronise the<br />

two systems once the new system is signed off as functional and the source system frozen.<br />

Page 57 of 57

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!