Working towards measuring library impact

Seek and discover.
Discover and know.
Know and become enlightened.

Background

This section includes, alignment of this tool to other work, the purpose of this tool and how it is intended to be used our development approach.

How this tool supports other key work

This initiative is part of a suite delivered through the New Zealand Libraries Partnership Programme to support the long-term sustainability of the libraries sector in Aotearoa. Early stakeholder engagement highlighted concerns about the time-limited nature of funding support and the need to consider ways to strengthen the sector for the longer term.

In response, the Programme carried out a structured process (Investment Logic Mapping, ILM) with experienced sector members to identify key issues that threaten the health of the sector, and potential responses. The process revealed three key problems and produced wide-ranging response options, which were then prioritised considering their relative cost, time needed to deliver and the impact on the problems identified.  The resulting package of measures was widely supported by sector representatives.

The diagram on the right shows how the ILM and prioritisation processes lead to the package of initiatives. Collectively, these initiatives will go some way towards addressing the issues identified through the ILM, ensuring library services in Aotearoa continue to evolve and thrive.  Areas not yet invested in could inform any future opportunities to collaborate across the libraries sector.   

Figure 1: NZLPP outcomes (supplied by the National Library)

A tool to help demonstrate the impact of libraries

Strengthening libraries’ ability to demonstrate impact 

The Libraries Applied Impact Framework is a tool designed to be used by the libraries sector 1 to improve their understanding of the impact of libraries. The tool can be used by libraries to help them:

• share with key stakeholders current evidence that demonstrates the difference libraries make

• identify gaps in the existing evidence and areas for further exploration.

The tool was developed by using a high-level summary of the key impact areas identified in research for the libraries sector. As such, the tool supports libraries to understand the current research with regards to the impact of libraries from multiple perspectives. This information can then be shared with key stakeholders to illustrate the difference libraries can make.

A set of tools to support illustrating impact

How the Libraries Applied Impact framework aligns with other tools

The Libraries Applied Impact Framework along with the DRE Maturity Model (a self-assessment tool) have been designed to support the implementation of the DRE Strategy. The DRE Strategy is a five-year Strategy to guide the direction of the libraries sector in collecting, analysing and using library-related data, research and evidence (see DRE Strategy Narrative). Using the DRE Strategy, DRE Maturity Model and the Libraries Applied Impact Framework together should help libraries illustrate the importance of library services more effectively, including the contribution they make to the wellbeing of people in Aotearoa. Using the Libraries Applied Impact Framework should support libraries to identify key focus areas for research.

Understanding the Framework  

The framework underpinning the tool 

The tool consists of two elements, the conceptual framework and the application of the current literature and evidence against the framework to create a usable tool.

The framework consists of four perspectives; the library user, the host organisation, communities, hapū and iwi perspectives and an Aotearoa perspective.

For each of these perspectives key questions have been added to help library staff think about the different areas/ways that library services can make a difference, and how this could be explained to maximise the impact when reporting, sharing stories and/or advocating on the value of the library.

The four perspectives   

The library user — this is the foundation of the framework. A primary outcome of library services is to add value to their users. When using the framework it asks library staff to consider:

• which library services provide value to library users?

• how do libraries staff keep informed of library users changing needs and as a result the service offering?  

• who uses the library? How do these users’ needs align with the priorities of the host organisation?  

The host organisation — almost all libraries in Aotearoa are part of a larger organisation. For example, school libraries are part of the school, public libraries are part of local councils, tertiary libraries are part of a tertiary institution and special libraries serve a larger organisation. This dynamic has an impact on libraries such as their focus, funding, resourcing models and services. 

This perspective is included in the model to reflect the need for libraries to remain relevant and add value to their host organisation and the host organisation’s priorities.

When using the framework consider the following:

• How do library services support the priorities of the host organisation and do the current library users needs align with the host organisations priorities?

Communities, hapū, and iwi — this is the third perspective in the framework. Every host organisation is part of a broader community, for example schools and kura are part of both a local community local hapū and iwi, and the education community. Communities, hapū, and iwi need to see value in the services library provide. At both the community, hapū and iwi, and Aotearoa perspective levels libraries need to continue to demonstrate that society approves and has confidence in the value libraries add. This is important for communities, hapū and iwi continued acceptance that libraries have a legitimate role and contribution to play. 

When using the framework consider the following:

• How are library services supporting the priorities and needs of their communities?

The final perspective is that of Aotearoa. Libraries have a long and established role in our society. This level helps library staff to explore the public goods that are obtained from having libraries throughout our communities, hapū and iwi.    

When using the framework consider:

• How do library services provide public goods for Aotearoa?

• What does Aotearoa require from library services that benefit people living here, both today and tomorrow?

One of the key concepts underpinning the framework is the relationships between the perspectives.  

For example, for libraries to remain relevant they need to meet the needs of their library users; they also need to understand who does not use their services and why. At the same time, the purpose of the host organisation and its strategic direction will play an important part in determining what services the libraries can and do provide. While ideally these different needs would align, often there are areas where there is a lack of alignment. Libraries need to be able to address these competing pressures.

At the same time, as libraries receive public monies, they also need to think about their contribution from the perspective of broader society (i.e., Aotearoa), communities, hapū and iwi or host organisation.

How to use the tool to help demonstrate the impact of libraries

Adding the impact areas to the framework  

Findings from the review of selected literature (Towards a value proposition for Aotearoa) was applied to the framework to illustrate impact areas by perspective, by sub-sector. The purpose and strategic direction of the host organisation will determine the impact areas of interest for each library. For example, as local authorities are the host organisations for public libraries, then the impact areas need to include the four areas of wellbeing that are part of their purpose in legislation (i.e., economic, social, cultural, and environmental wellbeing). Other areas of interest for a local authority could include contributing to improved health, education and knowledge, and equity of access. While for school libraries it is likely to be tied to improved student learning outcomes and a greater sense of wellbeing. Demonstrating the libraries’ contribution to these impact areas should resonate more with key stakeholders in the host organisation; this is because they can draw a direct line of sight between the activities and impact of library services and the performance of the host organisation. In essence library staff are then ‘talking their language’.

Similarly, in terms of meeting community needs, based on the evidence available public libraries make a wide range of contributions. There is a significant overlap between benefits to communities, hapū and iwi and the host organisation. The difference to some extent is in the language used, not the actual benefit itself.

Approach

This section explains our approach to developing the tool.

How the tool to help demonstrate the impact of libraries was developed

The approach to develop this tool involved three key stages. The first involved a review of selected literature 2 to find out what was currently known about the value of library services in Aotearoa. Information from 88 journal articles or reports were used. The literature included material from comparable jurisdictions including Australia, United States, Canada, United Kingdom and Europe. The literature review did not include the past value of libraries, the future sustainability of libraries, and challenges and barriers that limit the value of libraries.

The second stage involved developing the concept framework, which is based on the four key groups of stakeholders who benefit from library services. In addition, where possible existing frameworks were drawn on. In particular the New Zealand Treasury’s Living Standards Framework3 and the four aspects of wellbeing that are set out in the purpose of Local Government.

The third phase involved reviewing all of the current evidence set out in the literature review to identify emerging impact themes for the different library sub-sectors. These themes were then considered from the four perspectives: library user, host organisation, communities, and Aotearoa (society). For example, a tertiary student is likely to use the library to gain access to resources that will help them improve their study outcomes. From a host organisation perspective, i.e., tertiary institution, evidence indicates that students who use libraries perform better (e.g., have improved student outcomes). In addition, students learning contributes to improved knowledge and skills at a national level (Aotearoa society). In the framework diagram these have been referred to as Key Impact Themes.

Impact themes

This section sets out the key impact themes identified in the review of selected literature for each library sub-sector.

Applying the literature

Understanding impact themes and research opportunities by sub-sector 

The following section illustrates how the Key Impact Themes from the literature have been applied for the following  library sub-sectors:

• public libraries

• school libraries

• tertiary libraries

• heritage libraries

• special libraries

Areas of potential impact have been applied to the concept framework. These have been identified from a review of selected literature. The list is not exhaustive, but indicative of the literature and evidence. This tool is intended to be a working document which changes over time as more information becomes available. To help with this process, opportunities for further research have been identified.  

Segmenting by sub-sector has highlighted the differing amount of evidence across the sector and the lack of applicability of certain overseas research in specific sub-sectors due to the unique arrangements of libraries in Aotearoa. 

Public Libraries

Key impact themes	Opportunities
Health, knowledge and skills, engagement and voice, safety	To articulate better the contribution public libraries make to Aotearoa society.
Leisure and play, work, care and volunteering	To explore the current implications inequities have on public library outcomes
Cultural capability and belonging, family and friends, income, wealth and consumption	To explore public libraries contribution to the environment and subjective wellbeing.
Equitable free access, civic engagement	To identify to what extent communities, hapū and iwi benefit from public library services
Digital literacy and inclusion, revitalisation of te reo Māori	To identify which communities are not having their needs met by public libraries
Safe space, community cohesion and resilience, social and cultural value	To explore the implications of inequities on the services communities’ receive and whether Te Tiriti obligations are being met
Education and knowledge, equity of access	To more effectively measure the impact of public libraries (who, when, the service, how many, impact statements)
Health information	To identify the contribution a public library makes to their host organisation meeting its Te Tiriti obligations and achieving its’ goals
Wellbeing: Social, economic, cultural	To identify contribution a public library makes to the environment.
Reduced spending, learning and education, equity of access	To understand benefits of libraries from public library users’ perspectives
Health literacy and wellbeing, subjective wellbeing	To develop an understanding of the value of libraries to non-library users
Sense of belonging, support into employment, leisure, having a voice	To understand the barriers to using library services
	To understand the definition of what is a library from different people’s perspective

School and Kura libraries

Key impact themes	Opportunities
Health	To better establish the impact of school and kura libraries in Aotearoa
Knowledge and skills	To identify the impact of different levels of resourcing on school and kura library outcomes
Leisure and play, safety	To identify contribution of school and kura libraries to the environment & subjective wellbeing
There was no evidence in the selected literature about the benefits of school libraries to their broader communities	To explore the contribution school and kura libraries make to their broader community, including addressing inequities
School reputation, student outcomes	To more effectively measure the impact of school and kura libraries (who, when, the service, how many, impact statements)
Equitable access to resources, Improved student wellbeing	To identify and articulate the contribution the school/kura library makes to the strategic direction of the school/kura
Personal development, health information
Literacy and learning, student achievement, digital citizenship	To capture benefits from school and kura students’ and teachers’ perspectives
Access to resources, reading for pleasure, safe space	To develop an understanding of which students and teachers do and do not use library services, and why
Social cohesion, personal development, health information

Tertiary libraries

Key impact themes	Opportunities
Health, knowledge and skills	To better establish the impact of tertiary libraries in Aotearoa
Leisure and play, cultural capability and belonging	To explore tertiary libraries contribution to the environment & subjective wellbeing
Work, care and volunteering, safety
Sharing Skill & Practice	To explore the contribution tertiary libraries make to their broader communities, including addressing inequities
Sharing of information	To explore the contribution tertiary libraries make to their broader communities, including addressing inequities
Economic	To explore the contribution tertiary libraries make to their broader communities, including addressing inequities
Institutional reputation, economic return	To more effectively measure the impact of tertiary libraries (who, when, the service, how many, impact statements)
Improved student outcomes, Access to reliable information	To effectively identify contribution of tertiary library to institution meeting its Te Tiriti obligations and strategy
Cultural centre, improved skills, responsible information stewardship	To capture benefits from tertiary library users’ perspectives (i.e. student, academic staff, other university staff, others)
Student wellbeing, developing skills	To develop an understanding of which students and lecturers do and do not use library services, and why
Support teaching and learning, research support

Heritage libraries

Key impact themes	Opportunities
Knowledge and skills	To articulate better the contribution heritage libraries make to Aotearoa society
Cultural capability and belonging	To explore heritage libraries contribution to the environment & subjective wellbeing
Cultural preservation	To explore the value of heritage libraries for hapū and iwi
	To explore the value of heritage libraries for broader communities
Institutional reputation	To effectively identify contribution of heritage library meeting its Te Tiriti obligations and strategy
Economic value	To effectively identify contribution of heritage library meeting its Te Tiriti obligations and strategy
Cultural sustainability	To effectively identify contribution of heritage library meeting its Te Tiriti obligations and strategy
Equity of access	To identify impact of access and ownership of taonga, Te reo, and Mātauranga Māori for hapū and iwi
Cultural reconnection	To capture benefits from library users’ perspectives more broadly
	To develop an understanding of who uses or accesses the heritage collections, and why

Special Libraries

Key impact themes	Opportunities
Health, knowledge and skills	To articulate better the contribution special libraries make to Aotearoa society
Work, care and volunteering	To articulate better the contribution special libraries make to Aotearoa society
Safety	To articulate better the contribution special libraries make to Aotearoa society
Help bridge health inequities	To explore the value of heritage libraries for broader communities
	To explore the contribution special libraries make to broader communities, including addressing inequities
Information skills, ROI, knowledge development	To more effectively measure the impact of special libraries (who, when, the service, how many, impact statements)
Organisation reputation, contribution to strategy	To effectively identify contribution of special library to institutions meeting their Te Tiriti obligations and strategy
Access to information, improved practice	To capture benefits from special library users’ perspectives
Research support	To develop an understanding of who does and does not use library services, and why
Collegial space

Acknowledgements

The project team would like to thank Alex Thursby and Lewis Brown from the New Zealand Libraries Partnership Programme, within the National Library, who provided access to key documents, ongoing support, and guidance. We would also like to thank Sue Sutherland, our Senior Expert Advisor, for her advice and insights. In addition, we would like to thank NZLPP who funded this work.

Footnotes

1. The libraries sector includes publicly funded libraries (e.g., public, school, tertiary, special libraries (e.g., central government, health, Crown Research Institutes, National Library), and libraries which provide free access to resources, and/or benefit from public investment (e.g., some law and private libraries).

2. See Towards a Value Proposition of Libraries in Aotearoa: Review of selected literature.

3. Developing the themes at an Aotearoa perspective, uses the 2021 Living Standards Framework

Download Working Towards Measuring Library Impact

Working Towards Measuring Library Impact (pdf, 2.20MB)
Working Towards Measuring Library Impact (pptx, 18MB)