Opening our data

The National Library recently launched the new Open Data section of our website, which supplies access to sources of raw library information for anyone to download, examine and use. We do this happily, as part of the New Zealand Open Access and Licensing framework and Open Government Information and Data Re-use Work Programme.

We have always endeavoured to supply our patrons with easy-to-use search interfaces, so that anyone can find our materials using only a web browser. But for those who might have more sophisticated needs, the ability to work with large amounts of data and have access to a particular collection's records all at once can be a key part of crafting a more powerful solution to their research problems.

For these "power users", the new Open Data section consolidates our "big data" offerings from around the site, and some work behind the scenes means those datasets now stay up-to-date. We encourage anyone who is interested to download and use them, including:

• Researchers interested in analysing the entire holdings of the library at once.

• Library, archives and museum technologists wanting to make connections between their institution's holdings and ours.

• "Data hackers" who get excited by unearthing the interesting stories hidden in mountains of numbers and database records.

• Entrepreneurs who will use it to build innovative new applications we've only dreamed of: visualisations, collection explorers, new kinds of searches.

• Students interested in exploring the data kept by a large public institution, and learning how to manipulate it.

In addition, by publishing this data regularly and in standard formats, the Library itself can become its own best customer for open data. An institution the size of the National Library can have dozens, or even hundreds, of internal systems that it uses to keep track of its information, and getting those systems to talk to each other can be a monumental task.

By understanding the most common ways our data is being used, we can design better and easier-to-use information management and delivery systems and improve our services, both inside the institution and out.

Our initial Open Data offerings include:

Publications New Zealand metadata – Approximately 385,000 bibliographic records from New Zealand's national bibliography. Downloadable and updated every week.

DigitalNZ metadata – Metadata describing over 26 million pieces of digital content from all over the country, with real-time access via the application programming interface (API).

Papers Past metadata – A large and popular part of the DigitalNZ metadata is the Papers Past historical newspaper collection. Over 24 million items from this collection can be accessed through the DigitalNZ API.

We're planning to add much more in the future. Please keep an eye on the Open Data section, and follow us on Twitter to keep updated when new data sets are added, or when the Library is part of any new policy developments in the world of open and free content. We'd love for you to take this data and build something cool with it. And if you do, be sure to get in touch and let us know!