Digital preservation

Digital preservation is the active management of digital information over time to ensure its accessibility. Preservation of digital information is widely considered to require more constant and ongoing attention than preservation of other media. This constant input of effort, time, and money to handle rapid technological and organizational advance is considered a major stumbling block for preserving digital information. Indeed, while we are still able to read our written heritage from several thousand years ago, the digital information created merely a decade ago is in serious danger of being lost, creating a digital Dark Age.

Digital preservation is the set of processes and activities that ensure continued access to information and all kinds of records, scientific and cultural heritage existing in digital formats. This includes the preservation of materials resulting from digital reformatting, but particularly information that is born-digital and has no analog counterpart. In the language of digital imaging and electronic resources, preservation is no longer just the product of a program but an ongoing process. In this regard the way digital information is stored is important in ensuring its longevity. The long-term storage of digital information is assisted by the inclusion of preservation metadata.

Digital preservation is defined as: long-term, error-free storage of digital information, with means for retrieval and interpretation, for the entire time span the information is required for. Long-term is defined as "long enough to be concerned with the impacts of changing technologies, including support for new media and data formats, or with a changing user community. Long Term may extend indefinitely". "Retrieval" means obtaining needed digital files from the long-term, error-free digital storage, without possibility of corrupting the continued error-free storage of the digital files. "Interpretation" means that the retrieved digital files, files that, for example, are of texts, charts, images or sounds, are decoded and transformed into usable representations. This is often interpreted as "rendering", i.e. making it available for a human to access. However, in many cases it will mean able to be processed by computational means.

Digital Format Preservation Concerns

Society's heritage has been presented on many different materials, including stone, vellum, bamboo, silk, and paper. Now a large quantity of information exists in digital forms, including emails, blogs, social networking websites, national elections websites, web photo albums, and sites which change their content over time. According an article by Brewster Kahle, in 1996 founder of Internet Archive, "Preserving the Internet", Scientific American, the average life of a URL was, in 1997, 44 days.

The unique characteristic of digital forms makes it easy to create content and keep it up-to-date, but at the same time brings many difficulties in the preservation of this content. Margaret Hedstrom points out that "...digital preservation raises challenges of a fundamentally different nature which are added to the problems of preserving traditional format materials."

Physical deterioration

The media on which digital contents are stored are more vulnerable to deterioration and catastrophic loss than some analog media such as paper. While acid paper is prone to deterioration, becoming brittle and yellowing with age, the deterioration may not become apparent for some decades and progresses slowly. It remains possible to retrieve information without loss once deterioration is noticed. Digital data recording media may deteriorate more rapidly and once the deterioration starts, in most cases there may already be data loss. This characteristic of digital forms leaves a very short time frame for preservation decisions and actions.

Digital obsolescence

Main article: digital obsolescence

Another challenge is the issue of long-term access to data. Digital technology is developing quickly and retrieval and playback technologies can become obsolete in a matter of years. When faster, more capable and less expensive storage and processing devices are developed, older versions may be quickly replaced. When a software or decoding technology is abandoned, or a hardware device is no longer in production, records created with such technologies are at great risk of loss, simply because they are no longer accessible. This process is known as digital obsolescence.

This challenge is exacerbated by a lack of established standards, protocols and proven methods for preserving digital information. We used to save copies of data on tapes, but media standards for tapes have changed considerably over the last five to ten years, and there is no guarantee that tapes will be readable in the future. Recovering these materials may require special tools Hedstrom further explained that almost all digital library researches have been focused on "...architectures and systems for information organization and retrieval, presentation and visualization, and administration of intellectual property rights" and that "...digital preservation remains largely experimental and replete with the risks associated with untested methods".

Strategies

In 2006, the Online Computer Library Center developed a four-point strategy for the long-term preservation of digital objects that consisted of:

• Assessing the risks for loss of content posed by technology variables such as commonly used proprietary file formats and software applications.
• Evaluating the digital content objects to determine what type and degree of format conversion or other preservation actions should be applied.
• Determining the appropriate metadata needed for each object type and how it is associated with the objects.
• Providing access to the content.

There are several additional strategies that individuals and organizations may use to actively combat the loss of digital information.

Refreshing

Refreshing is the transfer of data between two types of the same storage medium so there are no bitrate changes or alteration of data. For example, transferring census data from an old preservation CD to a new one. This strategy may need to be combined with migration when the software or hardware required to read the data is no longer available or is unable to understand the format of the data. Refreshing will likely always be necessary due to the deterioration of physical media.

Migration

Migration is the transferring of data to newer system environments (Garrett et al., 1996). This may include conversion of resources from one file format to another (e.g., conversion of Microsoft Word to PDF or OpenDocument), from one operating system to another (e.g., Windows to Linux) or from one programming language to another (e.g., C to Java) so the resource remains fully accessible and functional. Resources that are migrated run the risk of losing some type of functionality since newer formats may be incapable of capturing all the functionality of the original format, or the converter itself may be unable to interpret all the nuances of the original format. The latter is often a concern with proprietary data formats.

The US National Archives Electronic Records Archives and Lockheed Martin are jointly developing a migration system that will preserve any type of document, created on any application or platform, and delivered to the archives on any type of digital media. In the system, files are translated into flexible formats, such as XML; they will therefore be accessible by technologies in the future. Lockheed Martin argues that it would be impossible to develop an emulation system for the National Archives ERA because the volume of records and cost would be prohibitive.

Replication

Creating duplicate copies of data on one or more systems is called replication. Data that exists as a single copy in only one location is highly vulnerable to software or hardware failure, intentional or accidental alteration, and environmental catastrophes like fire, flooding, etc. Digital data is more likely to survive if it is replicated in several locations. Replicated data may introduce difficulties in refreshing, migration, versioning, and access control since the data is located in multiple places.

Emulation

Emulation is the replicating of functionality of an obsolete system. Examples include emulating an Atari 2600 on a Windows system or emulating WordPerfect 1.0 on a Macintosh. Emulators may be built for applications, operating systems, or hardware platforms. Emulation has been a popular strategy for retaining the functionality of old video game systems, such as with the MAME project. The feasibility of emulation as a catch-all solution has been debated in the academic community. (Granger, 2000)

Raymond A. Lorie has suggested a Universal Virtual Computer (UVC) could be used to run any software in the future on a yet unknown platform. The UVC strategy uses a combination of emulation and migration. The UVC strategy has not yet been widely adopted by the digital preservation community.

Jeff Rothenberg, a major proponent of Emulation for digital preservation in libraries, working in partnership with Koninklijke Bibliotheek and National Archief of the Netherlands, has recently helped launch Dioscuri, a modular emulator that succeeds in running MS-DOS, WordPerfect 5.1, DOS games, and more.

Metadata attachment

Metadata is data on a digital file that includes information on creation, access rights, restrictions, preservation history, and rights management. Metadata attached to digital files may be affected by file format obsolescence. ASCII is considered to be the most durable format for metadata because it is widespread, backwards compatible when used with Unicode, and utilizes human-readable characters, not numeric codes. It retains information, but not the structure information it is presented in. For higher functionality, SGML or XML should be used. Both markup languages are stored in ASCII format, but contain tags that denote structure and format.

Trustworthy digital objects

Digital objects that can speak to their own authenticity are called trustworthy digital objects (TDOs). TDOs were proposed by Henry M. Gladney to enable digital objects to maintain a record of their change history so future users can know with certainty that the contents of the object are authentic. Other preservation strategies like replication and migration are necessary for the long-term preservation of TDOs.

Digital sustainability

Digital sustainability encompasses a range of issues and concerns that contribute to the longevity of digital information. Unlike traditional, temporary strategies and more permanent solutions, digital sustainability implies a more active and continuous process. Digital sustainability concentrates less on the solution and technology and more on building an infrastructure and approach that is flexible with an emphasis on interoperability, continued maintenance and continuous development. Digital sustainability incorporates activities in the present that will facilitate access and availability in the future.

Digital preservation standards

To standardize digital preservation practice and provide a set of recommendations for preservation program implementation, the Reference Model for an Open Archival Information System (OAIS) was developed. The reference model (ISO 14721:2003) includes the following responsibilities that an OAIS archive must abide by:

• Negotiate for and accept appropriate information from information Producers.
• Obtain sufficient control of the information provided to the level needed to ensure Long-Term Preservation.
• Determine, either by itself or in conjunction with other parties, which communities should become the Designated Community and, therefore, should be able to understand the information provided.
• Ensure that the information to be preserved is Independently Understandable to the Designated Community. In other words, the community should be able to understand the information without needing the assistance of the experts who produced the information.
• Follow documented policies and procedures which ensure that the information is preserved against all reasonable contingencies, and which enable the information to be disseminated as authenticated copies of the original, or as traceable to the original.
• Make the preserved information available to the Designated Community.

OAIS is concerned with all technical aspects of a digital object’s life cycle: ingest into and storage in a preservation infrastructure, data management, accessibility, and distribution. The model also addresses metadata issues and recommends that five types of metadata be attached to a digital object: reference (identification) information, provenance (including preservation history), context, fixity (authenticity indicators), and representation (formatting, file structure, and what "imparts meaning to an object’s bitstream".

Prior to Gladney's proposal of TDOs was the Research Library Group's (RLG) development of "attributes and responsibilities" that denote the practices of a "Trusted Digital Repository" (TDR) The seven attributes of a TDR are: "compliance with the Reference Model for an Open Archival Information System (OAIS), Administrative responsibility, Organizational viability, Financial sustainability, Technological and procedural suitability, System security, Procedural accountability." Among RLG’s attributes and responsibilities were recommendations calling for the collaborative development of digital repository certifications, models for cooperative networks, and sharing of research and information on digital preservation with regards to intellectual property rights.

Digital sound preservation standards

In January 2004, the Council on Library and Information Resources (CLIR) hosted a roundtable meeting of audio experts discussing best practices, which culminated in a report delivered March 2006. This report investigated procedures for reformatting sound from analog to digital, summarizing discussions and recommendations for best practices for digital preservation. Participants made a series of 15 recommendations for improving the practice of analog audio transfer for archiving:

• Develop core competencies in audio preservation engineering. Participants noted with concern that the number of experts qualified to transfer older recordings is shrinking and emphasized the need to find a way to ensure that the technical knowledge of these experts can be passed on.
• Develop arrangements among smaller institutions that allow for cooperative buying of esoteric materials and supplies.
• Pursue a research agenda for magnetic-tape problems that focuses on a less destructive solution for hydrolysis than baking, relubrication of acetate tapes, and curing of cupping.
• Develop guidelines for the use of automated transfer of analog audio to digital preservation copies.
• Develop a web-based clearinghouse for sharing information on how archives can develop digital preservation transfer programs.
• Carry out further research into nondestructive playback of broken audio discs.
• Develop a flowchart for identifying the composition of various types of audio discs and tapes.
• Develop a reference chart of problematic media issues.
• Collate relevant audio engineering standards from organizations.
• Research safe and effective methods for cleaning analog tapes and discs.
• Develop a list of music experts who could be consulted for advice on transfer of specific types of musical content (e.g., determining the proper key so that correct playback speed can be established).
• Research the life expectancy of various audio formats.
• Establish regional digital audio repositories.
• Cooperate to develop a common vocabulary within the field of audio preservation.
• Investigate the transfer of technology from such fields as chemistry and materials science to various problems in audio preservation.

Updated technical guidelines on the creation and preservation of digital audio have been prepared by the International Association of Sound and Audiovisual Archives (IASA).

Examples of digital preservation initiatives

Further information: List of digital preservation initiatives

• The Library of Congress operates the National Digital Information Infrastructure and Preservation Program
• The British Library is responsible for several programmes in the area of digital preservation. The National Archives of the United Kingdom have also pioneered various initiatives in the field of digital preservation.
• The Safety Deposit Box software from Tessella is being widely adopted by National Archives and has been selected by the National Archives in the UK, Netherlands, Switzerland, Finland, Estonia, Malaysia and Austria as well as FamilySearch and the Wellcome Collection. It was recently awarded the Queen's Awards for Enterprise in the UK.
• Ex Libris Rosetta is commercial software helping memory institutions to collect, manage, archive and preserve their digital collections, ensuring its data integrity and access over time. The system enables managing digital entities end to end—from submission to dissemination. A rule-based workflow engine and open architecture allow institutions using the system to develop unique plug-in tools and applications to enhance the system’s ingest, management, preservation and delivery processes.
• The MetaArchive Cooperative is a library-run, collaborative approach to digital preservation that embeds digital preservation infrastructure and knowledge in each of its constituent member institutions. Comprised mainly of University libraries, the Cooperative functions as a network wherein each preserved file is replicated seven times, is stored in geographically distinct locations across four countries, and is carefully managed from ingest (as a SIP) to dissemination (as a DIP).

Large-scale digital preservation initiatives (LSDIs)

Many research libraries and archives have begun or are about to begin Large-Scale digital preservation initiatives (LSDI’s). The main players in LSDIs are cultural institutions, commercial companies such as Google and Microsoft, and non-profit groups including the Open Content Alliance (OCA), the Million Book Project (MBP), and HathiTrust. The primary motivation of these groups is to expand access to scholarly resources.

LSDIs: library perspective

Approximately 30 cultural entities, including the 12-member Committee on Institutional Cooperation (CIC), have signed digitization agreements with either Google or Microsoft. Several of these cultural entities are participating in the Open Content Alliance (OCA) and the Million Book Project (MBP). Some libraries are involved in only one initiative and others have diversified their digitization strategies through participation in multiple initiatives. The three main reasons for library participation in LSDIs are: Access, Preservation and Research and Development. It is hoped that digital preservation will ensure that library materials remain accessible for future generations. Libraries have a perpetual responsibility for their materials and a commitment to archive their digital materials. Libraries plan to use digitized copies as backups for works in case they go out of print, deteriorate, or are lost and damaged.

See also

• Backup
• Bit rot
• Charles M Dollar
• Data archaeology
• Digital artifactual value
• Database preservation
• Digital asset management
• Data format management
• Digital curation
• Digital Continuity
• Digital library
• Digital obsolescence
• Digital reformatting
• Enterprise content management
• File format
• Information Lifecycle Management
• List of digital preservation initiatives
• New media art preservation
• Margaret Hedstrom
• Preservation metadata
• Section 108 Study Group
• Trustworthy Repositories Audit & Certification (TRAC)
• Universal Virtual Computer
• Web archiving

References

• Garrett, J., D. Waters, H. Gladney, P. Andre, H. Besser, N. Elkington, H. Gladney, M. Hedstrom, P. Hirtle, K. Hunter, R. Kelly, D. Kresh, M. Lesk, M. Levering, W. Lougee, C. Lynch, C. Mandel, S. Mooney, A. Okerson, J. Neal, S. Rosenblatt, and S. Weibe (1996). "Preserving digital information: Report of the task force on archiving of digital information" (PDF). Commission on Preservation and Access and the Research Libraries Group. Retrieved 2009-06-23.{{cite journal}}: CS1 maint: multiple names: authors list (link)
• Gladney, H. M.; Lorie, R. A. (2005). "Trustworthy 100-year digital objects: durable encoding for when it's too late to ask". ACM Transactions on Information Systems. 23 (3): 299–324. doi:10.1145/1080343.1080346.
• Gladney, H. M. (2006). "Principles for digital preservation". Communications of the ACM. 49 (2): 111–116. doi:10.1145/1113034.1113038.
• Granger, Stewart (2000). "Emulation as a Digital Preservation Strategy". D-Lib Magazine. 6 (10). doi:10.1045/october2000-granger.{{cite journal}}: CS1 maint: unflagged free DOI (link)
• Edwards, Eli (2004). "Ephemeral to Enduring: The Internet Archive and Its Role in Preserving Digital Media". Information Technology & Libraries. 23 (1).
• Hedstrom, M., Ross, S., Ashley, K., Christensen-Dalsgaard, B., Duff, W., Gladney, H., Huc, C., Kenney, A.R., Moore, R., Neuhold, E. (2003). "Invest to Save: Report and Recommendations of the NSF-DELOS Working Group on Digital Archiving and Preservation" (PDF). Nsf/Delos. Pisa & Washington DC, USA.{{cite journal}}: CS1 maint: multiple names: authors list (link)
• Jantz, R. & Giarlo, M.J. (2005). "Digital preservation: Architecture and technology for trusted digital repositories". D-Lib Magazine. 11 (6). doi:10.1045/june2005-jantz.{{cite journal}}: CS1 maint: multiple names: authors list (link) CS1 maint: unflagged free DOI (link)
• Ross, S (2000). Changing Trains at Wigan: Digital Preservation and the Future of Scholarship (PDF). London, UK: National Preservation Office (British Library). ISBN 0-7123-4717-8.
• Ross, S. and Gow, A. (1999). Digital archaeology? Rescuing Neglected or Damaged Data Resources (PDF). Bristol & London: British Library and Joint Information Systems Committee. ISBN 1-900508-51-6.{{cite book}}: CS1 maint: multiple names: authors list (link)
• Rothenberg, Jeff (1995). "Ensuring the Longevity of Digital Documents". Scientific American. 272 (1).
• Rothenberg, Jeff (1999). "Ensuring the Longevity of Digital Information" (PDF). {{cite journal}}: Cite journal requires |journal= (help) Expanded version of Ensuring the Longevity of Digital Documents.
• Milne, Ronald -- moderator: Webcast panel discussion, "Economics," Scholarship and Libraries in Transition: A Dialogue about the Impacts of Mass Digitization Projects (2006), Symposium sponsored by the University of Michigan Library and the National Commission on Libraries and Information Science (US).

External links

• National Digital Information Infrastructure and Preservation Program at the Library of Congress
• Digital Preservation page from the Digital Library Federation
• "Thirteen Ways of Looking at...Digital Preservation"
• Cornell University Library's Digital Imaging Tutorial
• What is Digital Preservation? - an introduction to digital preservation by Digital Preservation Europe
• Macroscopic 10-Terabit–per–Square-Inch Arrays from Block Copolymers with Lateral Order. Science magazine article about prospective usage of sapphire in digital storage media technology
• Animations introducing digital preservation and curation
• Capture Your Collections: Planning and Implementing Digitization Projects A CHIN (Canadian Heritage Information Network) Resource
• Digitales Archiv Hessen Digital preservation page by Hessisches Hauptstaatsarchiv Wiesbaden