Moving Toward Outcomes

by Carolyn A. Anthony on July 7, 2014

Recently I received an email message from a fellow library director recommending a librarian for employment. She noted that the librarian was skilled in readers’ advisory and a good team member, but had been let go due to continuing cuts to the public library’s budget by the municipal authorities, despite the fact that the town is sufficiently affluent to afford sustained support for public library services. Unfortunately, the local officials do not see the extent of the public library’s contribution to the well-being of community residents and to the town. We need to show more effectively that libraries are not only busy and efficiently run institutions, but that public libraries have multiple direct and indirect impacts on our communities.

Measures currently reported by public libraries generally do not reflect the outcomes and impact of library services. Circulation, door count, reference questions answered, and program attendance better capture library activity than purposeful realization of service objectives. Of course, public libraries do have programs with service objectives. One program offered by most public libraries is summer reading. The purpose of the program is to avoid the common summer slide in students’ reading skills by involving them in reading throughout the summer. For the most part, we don’t know if it really is effective in accomplishing that objective as it is daunting to get information from schools on the results of testing. Nevertheless, intuitively it seems that getting more children reading and coming to the library regularly throughout the summer must produce positive results, so an objective of increasing the number of participants is substituted for the original intent of improving reading skills. To engage more participants, the fun of a contest is introduced, challenging students to read a certain number of books in order to win a prize and/or a certificate. Ironically, because the children are eager to win the prize, they may choose shorter books at a lower reading level in order to rush through the program and claim a prize. The tactic produces a high completion rate for the program and librarians talk about the large number of children participating and completing the program. But to what end?

My intent is not to disparage summer reading, but to suggest that the programs can be demonstrably more effective if we just remind ourselves why they began in the first place. Author Simon Sinek talks about the primacy of why and puts it in the center of his “golden circle,” with how and what in the concentric circles that radiate out from it. He says, “Knowing your WHY is not the only way to be successful, but it is the only way to maintain a lasting success and have a greater blend of innovation and flexibility. When a WHY goes fuzzy, it becomes much more difficult to maintain the growth, loyalty, and inspiration that helped drive the original success.”¹ By tackling the matter of measuring the outcome of summer reading, it would be possible to see the effectiveness of the program as currently offered and then to try variations such as a targeted amount of time spent reading over the summer or a system of assigning points for books read, with books at or above grade level worth more points than those below, in order to see which

The Mid-Continent Public Library has partnered with the Kansas City (Mo.) Area Education Research Consortium to study how participation in the summer reading program can positively affect students’ reading assessment scores. Early results suggest that summer reading program participants demonstrated gains in reading achievement from spring to fall.²

PLA’s Performance Measurement Task Force is working on a set of outcome measures for public libraries that should be available next year, but any library interested in improving its ability to demonstrate its effectiveness can begin the transition to becoming an outcome-oriented organization now. Starting with why and the outcome sought is a great way to begin. Taking the example of summer reading, think about the consequences if the public library program were to significantly reduce the summer slide or even help students increase their reading skills over the summer. Teachers who typically spend the first six weeks of the school year reviewing lessons from the prior year might need to spend only two or three weeks of the new year on review, gaining three or four additional weeks to work on new skill development.

You may comment that it is unlikely that the public library could achieve such success in student outcome. It is true that “there is scant evidence that isolated initiatives are the best way to solve many social problems in today’s complex and interdependent world. No single organization is responsible for any major social problem, nor can any single organization cure it.”³ Therefore, another step that public libraries can take toward outcome orientation is to cultivate partnerships with other agencies and organizations that have objectives within the scope of the public library’s mission. For example, to reach more children for summer reading, the library might work with the parks department as many children are enrolled in summer day camps.

Another area of broad community interest is public health. The health department has conducted a survey of local health needs, and issued a plan identifying “access to health care” and “obesity” as two of four community health goals for the next five years. Both areas of concern have information and programming components that naturally lend themselves to involvement by the public library. The public library could work with the health department and local hospital or clinics, setting one or more objectives for health-related outcomes in line with community goals.

Becoming outcome-oriented requires developing deep knowledge of the community and its aspirations, as well as working relationships with key persons in other agencies who can be collaborative partners in working toward meaningful solutions. Such partnerships require trust and a level of communication that are best built over time. It’s never too soon to start establishing those relationships. Partners can assist the library in identifying local needs and in connecting with the target population. Are staff members at your library looking for a way to reach homebound residents? Your area department of social services and Meals on Wheels may be able to help librarians identify and connect with those who have difficulty getting out of their homes, especially during the winter months. Working with partners has the further benefit of building allies, spokespersons outside the library who can speak knowledgeably about the library’s work and accomplishments.

To achieve meaningful outcomes, librarians need to develop focused service objectives for specific audiences. A local concern for economic development and financial health could translate into several library objectives. Depending on local need and the library’s capacity, staff might develop one objective to help job seekers, another to support entrepreneurs, or an objective to assist individuals making financial decisions. Each initiative would be oriented to a different target group, likely involve different partners, and have different target outcomes.

Staff may protest that it is impossible to take on new initiatives developed to respond to identified community needs. Certainly, no library begins service development with a blank slate. There are myriad ongoing programs. A review of current programs may be a helpful step in moving toward outcomes. Look at current offerings and ask such questions as “Why are we providing this service? Who is the target audience? What do we expect to accomplish? How will we know if we succeeded?” The review may help to identify programs that are no longer needed or objectives that could be accomplished in a simpler or more collaborative way, involving fewer library resources. Martin Cole observed that, “Broadly speaking, a public service organization generates public value when it delivers a set of social and economic outcomes that are aligned to citizen priorities in a cost-effective manner.”⁴

Typically, the outcomes one would look for as a result of a targeted library service would be a change in the behavior, attitude, skills, or knowledge of participating individuals. This is sometimes referred to as a logic model “based on the if/then principle: If we offer this program/service, then a specific impact/value will be realized.”⁵ The collective outcome of such a service on a number of individuals or on a community over time may be referred to as the impact. Of course, the library service is not offered in a controlled environment so the library can’t take all the credit for an impact achieved, but if outcome measures are used repeatedly and adjustments made in the service delivery according to findings, one can feel quite confident that the library service has at least contributed to the positive result. Rhea Rubin defines outcome measurement as “a user-centered approach to the planning and assessment of programs or services that are provided to address particular user needs and designed to achieve change for the user.”⁶

Clearly, not everything a library does can be evaluated using outcome measures. Providing a broad selection of adult fiction may respond to a community need for culture and leisure, but no target group is identified, and no change in the user is generally anticipated as a result of reading fiction. It could, however, be possible to use fiction reading in a program designed to build empathy and reduce bullying among middle school students, for example. With the clear audience and focused objective, an outcomes assessment of a program of guided fiction reading could be conducted. Programs for literacy, digital literacy training, job skills, health and wellness initiatives, financial literacy, content creation, and twenty-first-century skills can all be developed for a focused user group with measurable outcomes.

What will it take to implement outcome measures for these services? Sometimes, we’ll need to ask individuals what they hope to accomplish before they participate and then later ask if the library program helped them achieve that goal. Other times, the outcome is clearly demonstrated as in the case of a person who wants to learn how to download an e-book to a mobile device and succeeds in doing so after being shown the procedure. When achieving a goal will take participation in multiple sessions, staff will need to ask for some basic information about the user to track them through the course of a series of programs. Other times, as in the case of a job skills program, the desired outcome is for the person to obtain an interview and get a job. Some follow-up is necessary to know if this has occurred. If not, the librarian has the opportunity to help that individual further and perhaps also to make changes to the program based on the user’s feedback. Beginning to think in terms of relationships with users and groups rather than in terms of transactions will prepare the way for outcome measurement, assessment that is expressed in terms of users’ gains rather than activity of the library.

New ways of thinking about program design and assessment at the time of program development will take time to implement. Even if the new measurement tools are not yet ready, it’s a good time to begin thinking in these terms. The rewards are immediate, for staff members who will see the value in the outcomes for people served, for managers who have a tool for continuous innovation and improvement, and for the library that can clearly demonstrate to budget-conscious civic leaders the real value it is delivering to the community.

References

1. Simon Sinek, Start with Why: How Great Leaders Inspire Everyone to Take Action (New York: Portfolio/Penguin, 2009): 50.
2. Lauren Barack, “Mid-Continent Public Library Proves Summer Reading Programs Boost Student Achievement,” School Library  Journal, Feb. 12, 2014, accessed May 9, 2014.
3. John Kania and Mark Kramer, “Collective Impact,” Stanford Social Innovation Review (Winter 2011): 38, accessed May 9, 2014.
4. Martin Cole and Greg Parston, Unlocking Public Value: a New Model for Achieving High Performance in Public Service Organizations (Hoboken, N.J.: John Wiley & Sons, 2006): xvi.
5. Moe Hosseini-Ara and Rebecca Jones, “Overcoming Our Habits and Learning to Measure Impact,” Computers in Libraries 33, no. 5 (June 2013): 5, accessed May 9, 2014.
6. Rhea Joyce Rubin, Demonstrating Results: Using Outcome Measurement in Your Library (Chicago: ALA Editions, 2005): 3.