Impact Survey: Measuring Your Library’s Impact
In October 2014, Impact Survey, a project of the University of Washington Information School (iSchool), launched its third release, the culmination of six years of work creating and refining a survey tool that empowers public libraries to easily evaluate how patrons use their services and also track the positive impact the library has on patrons’ lives. The 2014 release enables Impact Survey to go beyond public technology outcomes to begin offering surveys that question other areas of library services, such as children’s programs. The 2014 release also featured upgrades that make the process of surveying patrons even more accessible for libraries of all types.
Prior to the creation of Impact Survey in 2009 and the publication of Opportunity for All1 in 2011, the first large-scale research study about the impact of public access technology in libraries, many librarians only had an output-focused understanding of how patrons use library technology services. This was typically determined through the number of computer sessions libraries hosted or how many one-on-one questions librarians answered. Libraries lacked a more comprehensive picture of how patrons not only use, but also benefit from the technology services in libraries. Without this information, librarians were left to make strategic service decisions based on outdated or incomplete statistics and anecdotes that weren’t very helpful for planning or when advocating for greater support for technology services. Impact Survey was specifically designed to provide data, useful reports, and advocacy materials on outcomes achieved by patrons using the library’s services.
Librarians, and many others who do evaluation as part of their jobs, are often confused about the difference between outputs
and outcomes. In a library, outputs are the services, programs, and activities produced for patrons, such as storytime or public computer classes; outcomes are usually things you can easily count. Outcomes are what changes as a result of having access and using the outputs. Taking a count of the number of children and parents attending a story hour would be counting up an output (program attendance). Asking parents of young children if story hour helps them read to children more frequently than before they started attending story hour is counting a desired outcome: parents reading to their children more often is an important outcome on the road to literacy.
In the October 2014 release, the Impact Survey team incorporated many new features based on feedback received from libraries. These include the ability to run multiple surveys in a single calendar year and an improved dashboard that will make it easier for libraries to schedule surveys, keep track of survey progress, and increase survey responses from patrons. The new release also enables library groups, like regional associations or cooperatives, to run surveys together, allowing them to coordinate and implement surveys on behalf of their members, while still returning individual results to each of the participating libraries. This is especially useful for library groups that share resources.
One of the most exciting features of the new release is that the public library surveys will now be available to Canadian libraries, with the intent that, as demand rises, other countries outside the U.S. will also be able to use Impact Survey in their libraries. Impact Survey now has the capability to host custom surveys and reports for public libraries and other nonprofit organizations that want to develop and deploy outcomes-focused user surveys across multiple locations.
In order to become self-sustaining, Impact Survey now asks libraries to purchase a yearly subscription fee for a minimal
cost. All subscription fees go directly to maintaining and improving Impact Survey. State libraries, library consortia, and other groups of ten or more libraries can purchase subscriptions for their members at deeply discounted rates.
Impact Survey now has 856 registered libraries from 46 different states, ranging from large urban systems serving thousands of patrons to single-outlet libraries in rural communities. Since October 2013, 449 libraries have used Impact Survey, yielding more than 45,000 survey responses, giving those libraries valuable data on how patrons use public library services. The data are analyzed by Impact Survey so that libraries end up with ready-to-use reports and advocacy materials that have been customized with their patrons’ responses immediately after they are done running the survey. It can’t get any easier!
Real Libraries, Real Success
Ramsey County Library
Automation Manager Chuck Wettergren from the Ramsey County (MN) Library (RCL) near St. Paul implemented Impact Survey in 2013 and 2014 to help guide decision-making around new technology purchases and inform the library’s three-year strategic technology plan. “Library users are becoming more sophisticated in how they use our technology services,” Wettergren said.2 He found that RCL patrons use the computers and Internet access to not only check email and stay connected with friends and family, but also to search and apply for jobs, and to sign up for healthcare benefits.
A major change facing RCL this year is the remodeling of one library branch and the pending update to a second branch. Wettergren noted their results from Impact Survey have started playing a key role in how the library makes decisions around technology purchases as branches are updated. “The data lets us know exactly what technology services our patrons are using and ways we can improve and add new services to best meet their digital needs,” he said. “It helps demonstrate why we need to purchase new technology and helps us make informed buying decisions to ensure the community will value services at the updated branches.”3
As a member of the Metropolitan Library Service Agency (MELSA), RCL was subscribed to Impact Survey through the consortium, which allows MELSA to coordinate surveys among their members. Group membership for MELSA means that the member libraries can easily compare their results; MELSA also has access to customized aggregate reports so they can show the collective impact of their member libraries. Wettergren and his team also plan to take advantage of another new feature, running the survey twice a year to better track usage patterns.
Timberland Regional Library
Timberland Regional Library (TRL), which serves five counties in southwestern Washington, participated in the inaugural iteration of the Impact Survey in 2009. They implemented Impact Survey again in 2014, finding the data invaluable to their advocacy and planning efforts. They decided to repeat the survey in February 2015 to measure changes in satisfaction and perception of library technology services. “We always knew our library’s technology services helped improve the lives of people in our community,” said TRL Library Director Cheryl Heywood.4 Impact Survey revealed that many of the library’s patrons were using the computers and Internet access to search for jobs and find health and fitness information including nutritious recipes and fitness videos. “It’s one thing to say our library’s technology services have a positive impact on the community. It’s quite another to demonstrate the tangible results of our services using data from a validated research survey,” Heywood said.5 As a result, the library renewed its focus on promoting its online health and employment resources and partnered with other organizations to provide training and access to additional online health and business resources.
Heywood infuses Impact Survey data (see figure 1) into presentations to civic and business organizations and to high-level leaders and policymakers, including the county commissioners of each of the five counties comprising TRL. Heywood explained, “Having hard numbers to support our case for new technology services was a real ‘a-ha’ moment for our advocacy audience. In presentations I’m able to say for example; over 2,500 of our patrons who took Impact Survey find our library’s technology very helpful or this past year 142 people used our computers and Internet access to apply for a job, of which 42 found employment. The data gives us credibility.”6
Tulsa City-County Library
Charlotte Frazier, deputy director of support services at the Tulsa City-County (OK) Library (TCCL), an independent library district, believes high-quality library services start by understanding what customers are looking for at the library. “It’s important for us to listen to our customers. A survey like Impact Survey allows us to see how individuals are using our computers and I believe knowing how they use those services helps us to provide great customer service,” Frazier said.7 TCCL’s customer-first approach is helping to determine the technology services that will be offered at the newly renovated Tulsa Central Library that is currently under construction. The renovated central library will be a twenty-first-century library equipped with a STEM lab, a makerspace called “The Creation Station,” and an educational center for children and educators.
To help generate support for the project, library leaders met with the Tulsa City Council armed with data from various sources that demonstrated the invaluable impact 0the library has on the community—including data on use of technology services from Impact Survey. When speaking with council members, library staff focused on the outcomes that their library’s services produced, not just the services themselves. They also focused on how these outcomes align with the city’s key policy objectives like increasing employment and supporting education.
A city funding proposal that included ten million dollars for the Central Library renovation was ultimately adopted by the Tulsa City Council and voted in by residents. Frazier said, “The passing of the library renovation proposal was a huge win for us, to know the Tulsa community support our work and understands the value we bring to the community.”8 With buy-in from the city, the library was able to leverage support from additional funders to fully fund the $50 million project.
Impact Survey Evolution
As previously noted, Impact Survey grew out of a national research study conducted by the iSchool with funding from the
Institute for Museum and Library Services (IMLS) and the Bill & Melinda Gates Foundation, which resulted in the 2011
Opportunity for All report. The research was the first large-scale study assessing how people use technology services at
public libraries, why they use them, and how these services have helped improve their lives.9 The study used a mixedmethod
approach that included case studies, interviews, focus groups, and a national telephone survey. To augment the survey results, researchers developed a supplemental web survey and enlisted libraries across the country to help collect responses through their websites.
In appreciation for their help with the research, participating libraries received reports of the survey responses generated by their particular patrons. These libraries found themselves with rich data illustrating exactly how patrons use and value their library’s technology services and how these services impact their lives. In response to the overwhelmingly positive feedback from the public library community about the online patron survey, the research team developed a more sophisticated and cost-effective version of the survey tool, Impact Survey, so that libraries could continue to use the tested and validated survey for local evaluations and advocacy. The project is being funded by the Bill & Melinda Gates Foundation with expectations that it become self-supporting in the future.
Looking Ahead
The Impact Survey team is excited about the new features available to libraries and is looking forward to creating more surveys that can show how libraries help people and communities. Feedback from libraries continues to shape Impact Survey, and ideas from libraries on what survey topics to focus on next are always welcome. The team wants to provide public libraries with an invaluable tool that helps all librarians provide their communities with optimal services and supports robust, data-driven advocacy and fundraising for these services. Librarians can learn more about the Impact Survey and how to run the survey at their own libraries by visiting www.impactsurvey.org. As of October 1, 2014, libraries that register for a subscription are asked to contribute a small annual fee that will directly support maintenance of the Impact Survey and its services.
References
- Samantha Becker et al., Opportunity for All: How the American Public Benefits from Internet Access at U.S. Libraries (Washington, D.C.: Institute of Museum and Library Services, 2011), accessed May 13, 2015.
- Chuck Wettergren, telephone interview with the author, Dec. 19, 2014.
- Ibid.
- Cheryl Heywood, telephone interview with the author, Dec. 15, 2014.
- Ibid.
- Ibid.
- Charlotte Frazier, telephone interview with author, Dec. 11, 2014.
- Ibid.
- Becker et al., Opportunity for All: How the American Public Benefits from Internet Access at U.S. Libraries.
Tags: Impact Survey
