Happy World Digital Preservation Day from New Zealand!November 28th, 2018
It has been the season for celebrations at the National Library of New Zealand. We celebrated the 10-year anniversary of the National Digital Heritage Archive (NDHA) on the 24th of October. We hosted the Library’s first international conference in our newly refurbished auditorium two weeks ago for the International Internet Preservation Consortium (IIPC) General Assembly and Web Archiving Conference. And now we celebrate World Digital Preservation Day with you.
Looking back over the last 10 years since the NDHA was launched, some of the major milestones we can point to include:
- Developing a digital preservation strategy that we share with Archives New Zealand
- All of the planning, configuration, testing, and management to put five versions of the Rosetta software into production
- Conducting seven whole of domain crawls of the New Zealand web space
- Migrating the back-end storage twice
- Conducting two format migrations
- Conducting two Twitter harvests
What’s less evident is all of our day-to-day planning; troubleshooting; writing of code, papers, reports, specifications, policies, strategies and presentations; auditing; testing; outreach activities; gathering of requirements; collaborating and general management that it takes to run a digital preservation repository and programme. In the rest of this post, we’d like to invite you to hear from each of us what we’re up to in our day-to-day work.
- Andrea Goethals, Digital Preservation Manager, National Library of New Zealand
What are we up to now?
Ben O’Brien, Digital Preservation Web Engineer (Kaipūkaha Tukutuku Matapopore Matihiko)
Several tasks will be occupying my time this week:
- Primarily I will be working on getting the latest release of the Web Curator Tool, v2.0, into our Production environment. This is the culmination of eleven months of collaborative development with the National Library of the Netherlands. I will be going through the last of the testing in our QA environment, working through our release process, and tidying up documentation.
- Meeting with a vendor and colleagues to finalise the contract for our next whole-of-domain Web crawl.
- Starting work on migrating our CDX server instances (OutbackCDX) onto a new server.
- Spend some time on R&D, exploring Hitachi Content Intelligence and its potential use in computational and researcher access to our web archives.
- Assisting our web curators with any web harvesting or harvest replay issues.
Jay Gattuso, Digital Preservation Analyst (Kaitātari Matapopore Matihiko)
My work this week will span a few key areas:
- We are working on a pan-library process that helps us in the digital preservation team to undertake preservation treatments on “at risk” items. This process has a number of important internal stakeholders, so the current drafting phase very important to get right.
- I’m working on the very small pool of items we have that do not get assigned a valid PUID by DROID. I’m working through this set, trying to make sure we can figure out what these format these files are, and how to give them a valid file format identifier.
- Part of my day-to-day involves working with colleagues to lift our collective digital capability. This week I have few tasks that include facilitating an online computer science course, as a group of digital practitioners working through some workshop material that we saw at the recent IIPC conference, and supporting colleagues where needed on processes that are adjacent to the preservation systems and policies.
- One of my main tasks is supporting ingests where we encounter problems with digital content. I’ll be keeping an eye on the Technical Analyst Workbench in Rosetta and helping out with any content that lands there.
- I’ll be working with our Digital Preservation Policy Analyst to explore a new policy that we’re drafting that helps us to conceptualise the type of treatments we might need to undertake. The informational/artefactual policy will leverage a paper we wrote last year that describes how we want to classify content in to two broad types, and each of these types supports different preservation treatments. Our new policy will help us work consistently across the library.
Andrea Goethals, Digital Preservation Manager (Kaiwhakahaere Matapopore Matihiko)
Like most weeks, this week I’ll be working on a mix of tasks and projects. Here are some of the things that will keep me busy:
- Meeting with two librarians from the University of Canterbury Library to advise them on getting started with digital preservation
- Representing the Library’s requirements in a project kicking off this week to improve our ICT (aka IT) environment
- Writing a memo for one of the Library’s strategic projects that was initiated to increase the capability, capacity and diversity of staff within the Library
- Helping to finalise a contract for our next whole of domain Web crawl
- Planning for a new process for conducting digital preservation treatments
- Helping to get the iPRES 2019 call for contributions out as a member of the program committee
- Participating in a brainstorming session to imagine the next-generation version of the NDHA
- Having morning tea with my digital preservation colleagues from the Library, Archives New Zealand and Victoria University to celebrate WDPD!
Peter McKinney, Digital Preservation Policy Analyst (Kaitātari Kaupapahere Matapopore Matihiko)
During this week of International Digital Preservation Day, I’ll be undertaking three main tasks:
- Policy work
Specifically: refining our malware policy and scoping a new policy on new definitions for digital content in the Library: we’re trying to move towards classifying content as informational or artefactual.
This package of work describes and reimagines the lifecycle of digital “things” (objects and metadata) in the Library. This week the group will be focussing on getting ready to present to Library leadership.
- Format Library
I’ll be releasing the next version of the Global Format Library for Rosetta installations around the world.
Sean Mosely, Digital Preservation Technical Specialist (Mātanga Hangarau Matapopore Matihiko)
I joined the Preservation, Research and Consultancy group in 2014, having previously worked for Archives New Zealand in a similar role. With my colleague Cynthia Wu, we are primarily responsible for maintaining the Digital Archive and its supporting software. As such, a large chunk of our time will be devoted to monitoring the servers and identifying any potential issues with the network, storage, database or the application itself.
We will also be responding to users if they are experiencing any problems depositing content into the NDHA. We tend to be the first port of call for any issues, so we will triage any issues and determine if we can address the issue ourselves, or if we need to enlist the help of our Digital Preservation Analyst or our broader IT staff.
I will also be focusing on assisting our digital collecting staff in developing tools for preparing bulk content for deposit into the NDHA. Recently we have been harvesting electronic periodicals and automating the deposit process, so I will continue on with that. We are also identifying, cataloguing, and depositing a large corpus of digital files that was provided to the Library under Legal Deposit, so we will be finessing our scripts and updating the system configuration to ensure we can deposit those files smoothly into the digital archive.
Kirsty Smith, Test and Assurance Manager (Kaiwhakahaere Whakamātau me te Whakaū)
Main focuses for me this week will be:
- Rosetta Working Group (RWG)
I’m the current RWG Co-ordinator and a chunk of my time is spent on working group activities. This week we have our monthly working group call with the Rosetta Product Manager.
- NDHA Development Programme
We have some funding for ongoing development work to enhance the tools, plugins and work processes that support the digital preservation programme. A new release of out Indigo deposit application has been finalised and this week we are working with our IT Department to get it released into Production.
with the Optimising Storage implementation date looming this week we will be running some pre-cut over tests in our environment to use as benchmarks for post migration.
Cynthia Wu, Digital Preservation Technical Specialist (Mātanga Hangarau Matapopore Matihiko)
The focus of my work week has evolved quite dramatically since the Digital Preservation programme launched at the National Library of New Zealand in 2008. While my primary focus remains on maintaining the overall health of our digital preservation system and infrastructure together with my colleague Sean Mosely, a significant part of my week is spent on participating in the Library’s project to optimise its digital storage. During this week of International Digital Preservation Day, the following key areas will be:
- Application deployment and change management
The Preservation, Research and Consultancy group is looking to release a new version of its ingest application to users. I will be working with my colleague Kirsty Smith and our project management office on finalising work request to package and deploy the ingest application. We will also meet with our IT partnering company’s Change Manager to build better awareness around our workflows.
- Digital Preservation System Topology Refresh
I will be working with my colleague Sean Mosely and our counterparts from Archives New Zealand to brainstorm measurable outcomes and create a pathway to ensure the project’s success.
- Optimising Digital Storage Project
I will be running a testing workshop to gather testers’ feedback on automated storage access request workflows. The gathered feedback will help fine tune the automated workflows and the same information will be incorporated into the user guide I create. As the project approaches its implementation phase, I will be working with various business units within the Library on network performance testing and pre-implementation tasks such as storage rationalisation, access permission reviews and outage communications.
Suman Sajwan, Digital Preservation Operations Analyst (Kaitātari Whakahaere Matapopore Matihiko)
Reports, testing, training, website and more…my job involves several things. Here I will try to briefly cover the following:
- Change requests
I work on change requests from staff. (A change request often arises when a staff member wants an alteration to the metadata of items in the NDHA or to the actual item itself e.g. unwanted intellectual entities or deletion of files work). To understand this well, let’s think of it this way: A staff member, let’s call her Xena, has deposited some publications as ‘open access’ in the preservation repository. She is looking at what she has done so far. Overall, she is pleased with her work. But, then comes the request from the publisher, ‘Can you make the item restricted? Now, Xena doesn’t have enough rights in the system to make that change to the item, she then contacts you as you have more power (rights in the system). So, you make that change. My work involving change requests, is somewhat like this.There are different types of change requests, or change forms, that are typically actioned in different ways. These requests keep coming and I am doing my best to keep up! So far, I have already worked on 95 change requests this month and (still) counting.
“Working on numbers….” NDHA reports are produced for Rosetta staff users to support the ingest workflows and for statistical analysis. We have 38 + scheduled reports delivered automatically to our business users. I monitor and manage the distribution of these reports. I also work on ad-hoc report requests from business units. The latest report I worked on was for a staff member who was presenting at the IIPC conference and wanted to include some statistics in her presentation on web harvests collection growth over time. Here is the link to the full presentation, if you would like to look at that: 'From e-publications librarian to web archivist: a librarian’s perspective on 10 years of web archiving ...'
“…. to make sure that everything is working as it should.” A new version (3.1) Indigo (deposit tool) is currently being developed and tested. I am working with Kirsty (Test and Assurance Manager) on system testing. I am also working to develop a document for end-users which will have detailed information on the new features of the system including defect fixes, known errors and workarounds similar to this document - Indigo Version 3.0: Enhancements and Known defects.
Part of my job is to train and support Rosetta and Indigo users. The next training session is scheduled for Tuesday, December 11.
I manage this (digital preservation) website. We have recently updated content on some of the pages, so stay connected and check out the new content, explore, and share it with others. You can share your experience too by posting comments on the page below or through the email firstname.lastname@example.org.
Posted by the PRC Team