Digital Preservation: Best Practice and Its Dissemination
Digital information is increasingly important to our culture, knowledge base and economy. Long-term management of this material is a vital part of curation practice. This paper outlines the development and subsequent use of an international guide to digital preservation Preservation Management of Digital Materials: A Handbook [1] and its use in training and professional practice. The Handbook was published in 2001 in hard copy by The British Library and is also available digitally online via the Digital Preservation Coalition (DPC) [2]. It has been used as the basis of a series of one-day training workshops organised by the DPC in the UK and will shortly be integrated into the development of a modular digital preservation training course organised jointly by the University of London Computer Centre, Cornell University, The British Library, and the DPC. In addition the Handbook has been used extensively internationally in both its print and electronic form by institutions and individuals for personal development and taught courses.
The Handbook is intended to help institutions and their staff to manage the rapidly increasing volume of information which exists in digital form. Whether created as a result of digitising non-digital collections, created as a digital publication, or created as part of the day-to-day business of an organisation, more and more information is being created digitally and the pace at which it is being created is accelerating.
This activity is occurring in an environment in which there is a growing awareness of the significant challenges associated with ensuring continued access to these materials, even in the short term.
The combination of these two factors is both challenging and troublesome. On the one hand, there are considerable opportunities offered by digital technology to provide rapid and efficient access to information. On the other hand, there is a very real threat that the digital materials will be created in such a way that not even their short-term viability can be assured, much less the prospect that future generations will also have access to them. The need to create and have widespread access to digital materials has raced ahead of the level of general awareness and understanding of what it takes to manage them effectively. A clear picture has emerged of a complex and rapidly changing environment in which those creating and/or acquiring digital resources will require guidance in how to manage those resources most effectively.
Given this conjunction of factors, it seemed timely in 1999 to embark on a writing a Handbook which aimed at both identifying good practice in creating, managing and preserving digital materials as well as providing a range of practical tools to assist in that process. The Handbook was produced at a time when an important body of experience was emerging from recent research projects into digital preservation and from established data archives in the sciences and social sciences. Although many challenges remain, it is now possible to point to many examples of good practice and to suggest ways in which institutions can begin to address digital preservation. By providing a strategic overview of the key issues, discussion and guidance on strategies and activities, and pointers to key projects and reports, the Handbook aims to provide guidance for institutions and individuals and a range of tools to help them identify and take appropriate actions.
An Advisory Group consisting of experts in the field of digital preservation was formed, all of whom had first-hand knowledge of the range of complex issues involved. An early decision was that a Handbook would be the most appropriate mechanism to provide the range of advice and guidance required for such a diverse audience. Research to compile the Handbook combined traditional desktop research, using the World Wide Web as a source of freely available current information, as well as subscription-based print and electronic journals, supplemented by case studies and specialist interviews. Three very different case studies were selected to help develop the practical nature of the Handbook and to ensure that it addressed key issues currently being faced by organisations. Through structured interviews with selected specialists, workshops and conference presentations, and the case studies, it was possible to gauge the overall level of awareness and understanding of digital preservation and to transfer that knowledge to the development of the Handbook.
A consultation period for peer review and assessment of a pre-publication draft was provided. Comments were also accepted up until the end of the project to allow those wishing to comment to do so.
Figure 1: Cover of the print edition of Preservation Management of Digital Materials: A Handbook
In general, the research for the Handbook found that the level of awareness of and interest in digital preservation is gradually increasing but is not keeping pace with the level of digital resource creation. In particular, institutions that have not played a role in preserving traditional collections do not have a strong sense of playing a role in preserving digital materials. Individual researchers were keen to 'do the right thing' but frequently lacked the clear guidance and institutional backing to enable them to feel confident of what they should be doing. The difficulties of allocating responsibilities for preservation and maintenance in an environment in which digital resource creation is frequently a by-product of collaborative projects, which may well be funded by yet another external agency, was also mentioned. Overall, it appears that there is still a need to raise the level of awareness of digital preservation, particularly among funding agencies and senior administrators with responsibility for the strategic direction of an institution. This needs to be combined with more detailed guidance and training at the operational level. Moreover, the guidance needs to be able to accommodate people with varying levels of awareness and understanding of digital preservation, in a wide range of institutional settings, all of whom have significant constraints on their time.
Audience and Purpose
Digital preservation has many parallels with traditional preservation in matters of broad principle but differs markedly at the operational level and never more so than in the wide range of decision makers who play a crucial role at various stages in the lifecycle of a digital resource. Consequently, the Handbook is aiming at a very broad audience. In the first instance it is intended to provide guidance to institutions at national, regional and local levels which are involved in or are contemplating creation or acquisition of digital materials. Within those institutions, the Handbook is aiming at both administrators and practitioners and is accordingly structured to include a mix of high-level strategic overviews and detailed guidance. In addition, the Handbook is aimed at service providers who may be in a position to provide all or part of the services needed to preserve digital materials. It is also relevant to funding agencies which will need to be aware of the implications of creation of digital materials. Finally, it will be of interest to data creators whose involvement in the preservation of their digital publications is still crucial, despite being restricted by the overarching business needs of their organisation.
The Handbook fully recognises that these groups may have different interests and involvement with digital materials at different times. By adopting the life-cycle approach to digital preservation it aims to help identify dependencies, barriers and mechanisms to assist communication and collaboration between these communities.
The need to tailor the Handbook to the needs of individual institutions, including those where digital preservation may be outsourced and those where digital preservation may only be short-term, means that the Handbook needs to be seen as acting as a catalyst for further concerted action within and between institutions.
The broad issues associated with digital preservation are global in nature and examples of good practice, research activity and sources of advice and guidance have been drawn from around the world. However there is a UK focus in terms of the background to the study and some examples, e.g. legislation, are UK-specific. The text of the Handbook indicates a UK focus whenever relevant. The Handbook has proved relevant to an international audience: many of the models and references provided are not UK-based and are in any case applicable to any country. Wherever their country of origin, the users of the Handbook will need to tailor it to their specific needs.
The overall theme of the Handbook is that while the issues are complex and much remains to be clarified (and may never be definitively resolved), there is nevertheless much that has already been achieved and much that can be undertaken immediately by all involved in creating or acquiring digital materials. This activity will help to protect the initial investment in digital materials creation and offer considerably improved prospects for the long term.
Major Objectives of the Handbook
1. Awareness raising: as mentioned earlier, this was regarded as still being very necessary and relating largely to the considerably different approach needed for digital preservation than that for traditional preservation. The word 'preservation' seems not to resonate with people who are not archivists or preservation managers. They tend to assume a traditional model of preservation: an 'end of the line' activity, undertaken by highly specialised staff, that does not really involve them to any degree. In reality, digital preservation is very different from this traditional model: it requires intervention and management at, or close, to the point of creation, and the active engagement of a wider range of players who are not preservation specialists. So this Handbook plays a role in trying to reach a large number of people who may not, to date, have considered that digital preservation has anything to do with them.
2. Collating existing relevant advice and guidance: there is a great deal of helpful information out there, especially in terms of creating digital resources. But these resources are not necessarily readily accessible, especially to those people who are relatively new to this area. The sheer volume of information can be quite bewildering so an important aim is to help people navigate their way through the mass of existing information, making it simpler to find the resource which best suits their particular purpose.
3. Providing advice and guidance: in addition to providing links to guidance documents which already exist, the Handbook aims to provide guidance in the form of a decision tree, checklists and tables which go through the stages in creating and acquiring digital resources, drawing attention to issues which may not necessarily have been thought of. A major theme running through the Handbook is that it is much better to take decisions as early as possible, preferably at creation, to avoid the risk of losing access to the digital material at a relatively early stage.
4. Empowering organisations to take action: this final aim is an important one, given the complexity of the issues and the speed of developments. It would be easy to simply defer developing any corporate policies and strategies relating to digital preservation until the whole scene has settled down and the results of research are known. This is an area where this Handbook meshes quite well with the Cedars [3] and CAMiLEON [4] projects, which have conducted critically important research into preservation methods. In particular their work has been crucial in helping to identify the most appropriate long-term digital preservation strategies for particular categories of digital materials. However it is important for institutions to know that they can and should take some action now; they do not need to wait until everything is resolved. Indeed there is likely to continue to be a state of uncertainty and rapid change for the foreseeable future, but that need not prevent institutions from developing an approach to creating and acquiring digital materials based on sound principles and policies. This approach will help to provide those materials with a significantly improved chance of survival.
Figure 2: Digitisation checklist: an example of guidance and tools in the Handbook
Testing and Feedback on the Handbook
A consultation period for peer review and assessment of a pre-publication draft was provided between 8 August and 4 September 2000. A total of sixty-seven individuals, including the advisory group, case study interviewees, and a number of experts either in digital preservation or a related area of expertise, were invited to comment on the Handbook. The invited respondents represented a very diverse geographic and sectoral constituency, in keeping with the broad audience the Handbook intended. A total of twenty-seven responses were received by the end of September 2000, representing around a 40% response rate. The overall quality of the responses was very high with several incredibly detailed commentaries. The value of this feedback, in terms of testing the effectiveness of the Handbook and providing constructive suggestions for improvements, would be difficult to overestimate.
Sales and Reviews
The work undertaken to road-test the Handbook as a pre-publication draft has paid substantial dividends both in terms of reviews and subsequent sales. Reviews of the Handbook have been exceptionally positive and over 1,300 copies of the print edition of the Handbook have now been sold.
Sales have been pre-dominantly to the UK, elsewhere in Europe, and North America, as follows:
UK and mainland Europe - 1021
North America - 375
Web Version of the Handbook
In such a rapidly developing environment, it was clearly necessary to maintain the Handbook to ensure its currency. This has been achieved through an electronic version and supporting materials available on the Web, which are updated on a regular basis to ensure currency of Web references and cited projects. A commercial link-checking service has been used to provide monthly reports on broken and modified url links cited in the Handbook. These reports are then used as the basis for making updates to the electronic edition. It is envisaged that part of the development of the training course noted below will be to undertake a more extensive process of reviewing and extending the content of the Handbook. This will effectively produce the first major new edition of the Handbook online.
Digital Preservation Coalition Training Workshops
The DPC was formed in 2001 as a membership organisation [5]. Its mission is to foster joint action to address the urgent challenges of securing the preservation of digital resources in the UK and to work with others internationally to secure our global digital memory and knowledge base. From its foundation one of its goals has been producing and disseminating information on current research and practice as well as building expertise among its members to accelerate their learning and generally widen the pool of professionals skilled in digital preservation.
The DPC responded to the recognised need for training by organising one-day training workshops for its members. The one-day workshop was structured around the Handbook which was provided in advance of the workshop to participants. Presenters spoke on aspects of digital preservation, such as myths and legends, developing policies, advocacy and awareness raising, as well as roles and responsibilities. Breakout groups worked through developing a policy together with a matrix of roles and responsibilities for different categories of digital materials. Each workshop included a practical case study provided by the host institution. The workshops were limited to a maximum of 30 participants, and were held in Belfast, Edinburgh and London.
Structured feedback on the workshops was obtained via delegate evaluation forms. These demonstrated very positive feedback on the Handbook and the workshops but also highlighted the opportunity for more extensive training to be developed.
Training Needs Analysis
At the same time, a number of reports and studies funded by the Joint Information Systems Committee (JISC) under its Continuing Access and Digital Preservation Strategy had identified training in long-term management and preservation of digital assets as a major issue for the Higher and Further Education sector in the UK [6]. It remains difficult to recruit individuals with appropriate skills and the need for such skills is growing. The potential impact of this long-term skills gap on institutions in the UK was likely to be severe and is a cross-sectoral issue. Concurrently, a DPC survey of its members also revealed the need for training as a high priority.
Consequently, both the JISC and the DPC wished to explore commissioning the development of a training programme for their respective constituencies. As a founding member of the DPC, the JISC recognised that such an initiative would benefit from input from a wide range of partners within the membership and might ultimately be more sustainable on a cross-sectoral basis. The JISC therefore grant-aided the DPC and Cornwell Management Consultants plc to prepare a study of training needs and a report with recommendations.
The purpose of the project was to determine training needs for digital preservation, determine the course content as well as identify training providers, delivery methods and likely training costs. The consultants gathered information by surveying and interviewing selected Subject Matter Experts identified by the DPC and JISC. They developed a semi-structured interview questionnaire and a short survey. These aimed to elicit the challenges faced by interviewees, their perceptions of various strategies for digital preservation, and implications for training provision. Following completion of the analysis of these inputs, the consultants developed a course outline, and contacted potential suppliers (selected in agreement with DPC and JISC) to request training course design and estimated costs. The final report was then published on the JISC Web site [7].
Next Steps: A Modular Digital Preservation Training Programme
In the summer of 2003 JISC issued a call for proposals for Projects in Supporting Institutional Digital Preservation and Asset Management [8]. One theme of the call invited proposals for 'Training programmes for staff development, aimed at developing the appropriate level of knowledge and skills necessary for digital preservation and asset management at institutions.'
A proposal in response to the call was submitted by a consortium led by the University of London Computer Centre. This proposal was successful and is now underway. It will develop a modular training programme in digital preservation, with class-taught, online and off-line components to meet the needs identified in the training needs analysis study, carried out by Cornwell Management Consultants. It builds on existing exemplars of training and information provision, including the Cornell University digital preservation course, the DPC's one-day workshop, the Handbook, and existing training from JISC-funded services. The training will be developed at multiple levels, to meet the needs of senior managers as well as practitioners and new staff. By including self-paced material as well as taught components and group exercises, it will impart information in a way suited to a variety of requirements. With the backing of the DPC, the course will be made available outside the JISC community, bringing in revenue which will aid the long-term sustainability of the project's outputs.
The Cornell Training Workshops and Online Tutorial, developed with National Endowment for the Humanities funding, offered a model for the kind of programme envisaged. Excellent links have already been formed with the Cornell team by the UK consortium. They propose building on this still further by developing with Cornell a core set of generic training modules, which can also be tailored for specific UK or US settings. The intention is to develop a core set of training modules which are equally applicable in either country.
The modular content developed will include accommodating different skill levels (basic, intermediate, and advanced) and will also be aimed at different organisational staffing levels (senior managers, operational staff, technical/non technical staff).
The programme will produce a revised and updated online version of the Handbook, enabling not only more recent developments to be included, but also introducing interactive features. It will also form the basis for some of the modules in the programme, enabling a structured and easily understood introduction to digital preservation and providing a continuing source of guidance following the training. The DPC Web site already provides a focal point for digital preservation, so this will continue to be the primary mechanism for access to the online Handbook and details of the programme.
In summary, the following characteristics of the training programme will facilitate these wide-ranging requirements:
- Modular programme accommodating different levels of knowledge and expertise
- In collaboration with Cornell, development of a set of core modules
- Extensive use of practical case studies and visits
- Extensive use of UK, other European and overseas experts
- Limiting course attendance to a maximum of circa 25 delegates
- Developing the online edition of the Handbook to enable preparatory work and ongoing support and guidance post-training
Conclusion
The Handbook is part of a growing number of resources and initiatives focusing on digital preservation. The increasing prominence given to digital preservation world wide is an indication of how seriously it is being treated. There is unlikely ever to be a single definitive solution and certainly nothing will preclude the need for individual institutions to commit time and effort to addressing their specific requirements. However there is now much to build on and increasing practical examples to provide both inspiration and guidance.
The analysis of training needs and feedback from existing training workshops in the UK have demonstrated the value of the Handbook both as a stand-alone tool and in conjunction with structured tuition. The proposed future development of the Handbook and its integration with online tutorials and modular workshops promises to address a growing need in institutions.
The overall theme of the Handbook is that while the issues are complex and much remains to be clarified (and may never be definitively resolved), there is nevertheless much that can still be undertaken immediately. This activity will help to protect the initial investment in digital resource creation, improve knowledge transfer between practitioners, and offer considerably improved prospects for the long-term management of these materials.
Acknowledgements
I am grateful to Maggie Jones and Kevin Ashley for reading and commenting on a preliminary draft of this article.
References
- Beagrie, N. and Jones, M. 2001, Preservation Management of Digital Materials: A Handbook (British Library, London)
- The online version of the Handbook can be accessed from the home page of the Digital Preservation Coalition Web site at http://www.dpconline.org/
- The Cedars project completed in 2002. The project Web site is at http://www.leeds.ac.uk/cedars/
- The Camileon project completed in 2003. The project Web site is at http://www.si.umich.edu/CAMILEON/
- Beagrie, N. 2002, An Update on the Digital Preservation Coalition, D-Lib Magazine Volume 8, Number 4, April 2002. Available online from: http://www.dlib.org/dlib/april02/beagrie/04beagrie.html
- JISC 2002, The JISC Continuing Access and Digital Preservation Strategy 2002-5. The Strategy is available from: http://www.jisc.ac.uk/index.cfm?name=pres_continuing
Details of the feasibility studies funded under the Strategy are available from: http://www.jisc.ac.uk/index.cfm?name=programme_preservation - Cornwell Consultants Ltd 2004, Digital Preservation Coalition Training Needs Analysis Final Report: A study funded by the Joint Information Systems Committee, 251345v2.0 October 2004. Available from: http://www.jisc.ac.uk/uploaded_documents/finalReport.pdf
- JISC 2003, JISC Circular 4/04, Call for Projects in Supporting Institutional Digital Preservation and Asset Management. Available online from: http://www.jisc.ac.uk/index.cfm?name=funding_circular4_04