Outcomes + Outreach The California Summer Reading Outcomes Initiative

by Natalie Cole, Virginia Walter, & Eva Mitnick on May 7, 2013

Public library summer reading programs make a difference. They help children and teens retain and enhance their reading skills during the summer. They help adults model reading activity for youth. And they provide a haven and a community for readers. But how do we share the impact of our work? How do we extend our programs effectively to underserved communities? And how do we ensure that our programs remain relevant? With the support of the California Library Association’s (CLA) California Summer Reading Outcomes Initiative,¹ public libraries in California are beginning to implement outcomes-based summer reading programs that accomplish all of this and more. In the sections that follow, we will discuss the California Summer Reading Outcomes Initiative, the early results that participating libraries are seeing, and a case study from the Los Angeles Public Library (LAPL)—one of the early adopters of outcomes-based summer reading in California.

The Value of Outcomes-Based Programming

The value of outcomes-based programming is well-documented. Outcomes-based programs are relevant to the community;² they generate results that enable us to demonstrate their value;³ and they are cost-effective because they are tailored specifically to local need.⁴ However, despite this, summer reading programs are not commonly planned or evaluated according to outcomes-based principles. Consequently, the impact of summer reading programs is undocumented, although we suspect that they are not fulfilling their potential. We know that they primarily reach confirmed readers and the library regulars⁵ and they are at risk of becoming irrelevant to large segments of the community. As Francine Fialkoff stated in a call to action in Library Journal:

Libraries are no strangers to delivering community services, nor to goal-setting, but they need to ensure that they’re participants in these larger movements (often called collective impact). It’s critical that they’re providing what their communities need, and that they, too, are collecting data that illustrate the outcomes of their efforts and tell the stories of their successes to funders, donors, and voters.⁶

California Summer Reading Outcomes Initiative

With the support of a Library Services and Technology Act (LSTA) grant from the California State Library, CLA has created the California Summer Reading Outcomes Initiative to help California librarians go beyond collecting output data—such as how many patrons participate in their programs, how many books those patrons read, and how many activities they participate in—and begin collecting outcomes data that provide richer and more meaningful information about the impact of summer reading. Outputs continue to provide libraries with valuable evidence about the popularity of their programs. However, outcomes data go one step further and provide us with answers to questions such as: “So what? Why does it matter that thousands of children take part in summer reading each year?”

The initiative was developed between 2007 and 2010 by Natalie Cole, Virginia Walter, Cindy Mediavilla (library programs consultant for the California State Library), and a task force of librarians from nine California library jurisdictions.⁷ Together, we created, tested, and refined two statewide outcomes; a set of survey and focus group questions; and a comprehensive collection of resources. The process was librarian-led to ensure that the initiative would be relevant to California’s summer reading programs, rigorous enough to capture valid data, and streamlined enough for librarians to implement in busy and under-resourced libraries. Because the initiative has been designed to relate to the successful summer reading programs that are presented in California libraries each year, librarians do not have to significantly alter their existing programs to participate.

California’s Statewide Summer Reading Outcomes

Two statewide outcomes that capture both the value and the potential of California’s summer reading programs are at the heart of the initiative. We developed statewide outcomes to enable CLA to gather consistent data from California’s summer reading programs and to provide libraries with a ready-made set of outcomes that can be implemented easily. However, we encourage libraries to set outcomes of their own in addition to working with those we have developed.

The statewide outcomes are specific enough to capture the essence and value of the summer reading program and general enough to be meaningful to libraries across our diverse state. They can be adopted by libraries that present children’s, teen, adult, and family summer reading programs. They are:.

1. Children (or teens, adults, or families) belong to a community of readers and library users.
2. Targeted community members participate in the summer reading program.

As we planned the initiative, we discovered that, although California’s summer reading programs are highly regarded by those who participate in them, summer reading participants tend mainly to be readers and library regulars.⁸ In all communities, there are children, teens, and adults who do not participate in summer reading and who would benefit greatly from more extensive outreach efforts by librarians. As a result, the initiative is focused on outcomes that relate both to typical summer reading participants and to California’s underserved communities.

Outcome One

Outcome one addresses the most common objective for summer reading programs: to promote reading for enjoyment. The results that are derived from programs that are designed to achieve outcome one help libraries demonstrate the value and quality of their summer reading programs. Summer reading participants tend to be active and engaged readers who already use libraries. And people who come to identify as readers are most often those who have found a social setting in which their peers also enjoy reading or those who are more introverted and like the escape that reading provides. Outcome one speaks to the first category of readers and allows for a more subtle interpretation of the library as a haven for those who find reading to be a more personal and private activity. Community is broadly defined. It assumes that any child, teen, adult, or family group that chooses to participate in a summer reading program identifies with the library and with the program, thereby becoming part of a community of interest. While the focus of the summer reading program is obviously on reading, it also aims to increase library usage and to draw new users into its community. Librarians who use the language of community to present the summer reading program to elected officials, teachers, and other significant stakeholders will find that it resonates with current political and educational values. It positions the library as a place for readers—not just children and teens, but families as well.

Outcome Two

This outcome uses the language of outputs, but it also indicates a change in behavior in the target group. It addresses the concern that most summer reading programs only reach the library regulars. It also challenges librarians to identify target groups of underserved children, teens, or families who have not traditionally participated in the summer reading program, and to devise strategies for bringing them into the library—or for bringing the library to them. Outcome one refers to libraries and community. Many specialists in child and youth development talk about the importance of building a web of community support to help families engage in the increasingly difficult task of raising healthy kids. The public library can be a critical node in that web of community support. Summer reading programs can help to draw people of all ages into that web so they can be nurtured by it and can be enabled to nurture others as well.

By being proactive about proclaiming the library’s important role in community building, librarians will build political capital at the same time that they are serving their patrons. Because we believe that this is such an important part of the library’s mission, we are committed to helping staff reach out to those community members who have not yet found their way to the library. The outreach outcome is intended to institutionalize libraries’ ongoing efforts to open their hearts and minds and doors to every man, woman, and child in their service areas.

Project Framework and Resources

Participating libraries adopt California’s two outcomes and set them as goals for their summer reading programs. They plan their summer reading programs with the outcomes in mind—which includes designing programs, activities, and outreach strategies that will help them achieve the outcomes. They use surveys and focus groups to collect data to determine whether the outcomes have been achieved. They report their results to CLA. And they use their results to improve their program, demonstrate the value and impact of their work, and set targets for the following year. The initiative is an ongoing project and we encourage participating libraries to keep presenting outcomes-based summer reading programs.
To help libraries transition to outcomes-based summer reading, CLA has developed comprehensive resources that are available for download from the association’s website. Our ready-made surveys and focus group questions make it easy for libraries to collect data to determine whether their outcomes were achieved. Each survey includes a brief list of questions and a set of indicators of community. We advise libraries to collect at least one hundred completed surveys from patrons at each participating site. Libraries can use the combined results they receive from the surveys to demonstrate that their patrons feel part of a community of readers and library users, and they can use responses to individual survey questions to gather quantitative data about the value of their program. Additionally, our focus group questions help libraries collect richer and more in-depth feedback from patrons.

Our project checklist takes library staff step by step through the planning, implementation, and reporting process. Tips and guidelines on administering surveys and convening and conducting focus groups help libraries collect their data successfully. And further resources include informational brochures; programming ideas; advice on community mapping and conducting community needs assessments; information on developing successful community partnerships; and guidance on using outcomes data to demonstrate the value of the library and improve library service. We have presented webinars on outcomes-based summer reading in partnership with Infopeople, California’s library training agency. And CLA staff is available to provide support by telephone and email.

Proven Results

The initiative has seen a variety of early and encouraging successes. In 2011, fourteen library jurisdictions participated, and in 2012, 183 main and branch libraries in eighteen library jurisdictions took part.⁹ As we hoped, survey results from both years are demonstrating that summer reading participants feel part of a community of readers and library users—a positive finding that suggests that these patrons will experience all the benefits that we know reading and library use confer.¹⁰ Furthermore, participating libraries are successfully engaging new members of underserved communities in the summer reading program.

In 2012, 175 main and branch libraries in thirteen library jurisdictions planned and evaluated their programs with the goal of achieving California’s first statewide summer reading outcome: children (or teens, adults, or families) belonging to a community of readers and library users. Of the tens of thousands of patrons who took part in summer reading in those thirteen library jurisdictions, 7,764 completed outcomes surveys (4,946 children, 1,783 teens, 171 adults, and 864 families). Taken as a whole, the data from these surveys show that most respondents see the library as a place where readers can find a community of like-minded people. They indicate that summer reading participants view the library as:

• A place to find things to read (77 percent of children, 75 percent of teens, 89 percent of adults, and 89 percent of families).
• A friendly place (49 percent of children, 53 percent of teens, 56 percent of adults, and 85 percent of families).
• A peaceful place (53 percent of children, 64 percent of teens, 69 percent of adults, and 68 percent of families).
• A place for them (48 percent of children, 51 percent of teens, 49 percent of adults, 74 percent of families).

Even more respondents reported that:

• They like to share books or talk about the books they read (65 percent of children, 63 percent of teens, 90 percent of adults, 95 percent of families).
• They enjoyed the summer reading program (87 percent of children, 77 percent of teens, 88 percent of adults, 97 percent of families).
• They will return to the library after the summer (84 percent of children, 84 percent of teens, 91 percent of adults, 96 percent of families).

Notably, a full 56 percent of children, 60 percent of teens, 44 percent of adults, and 35 percent of families reported that 2012 was the first year in which they had taken part in the summer reading program. We anticipate that these numbers reflect both the increased library use that we are also seeing at other times of the year, and outreach efforts at participating libraries.

Ninety-four main and branch libraries in thirteen library jurisdictions planned and evaluated their summer reading programs with the goal of achieving California’s second statewide summer reading outcome: targeted community members participating in the summer reading program. As a result of their efforts, 7,466 previously underserved community members participated in the summer reading program. These new summer reading participants included children and teens in a transitional housing facility; teens in a group home; Head Start parents; kids from targeted preschools, elementary schools, community centers, and housing projects; and families in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC).

Of these previously underserved community members, 3,063 participated in summer reading within a coordinated outreach program that was facilitated by CLA and is part of a statewide strategy (“Summer Matters: A New Vision for Summer Learning and Enrichment in California”). The strategy sees schools, after-school providers, libraries, parks, and other groups coming together to provide quality summer learning programs for low-income children across California. Public libraries are contributing to the strategy by partnering with community-based summer enrichment programs to engage children and their families with summer reading and the library. Summer Matters is spearheaded by the organization Partnership for Children and Youth and funded by the David and Lucile Packard Foundation.¹¹

Since 2010, 10,120 Californians have taken part in summer reading as a result of outreach efforts at libraries that have participated in the California Summer Reading Outcomes Initiative. The effects of that outreach can continue long after the summer. In 2011, Sacramento Public Library took the summer reading program to a youth detention facility. More than sixty teens participated and twenty-four finished the program. All participants received books chosen with their interests and reading levels in mind and the library has since begun to provide donated books for the facility’s library. An LAPL staff member reported on the outcomes survey form: “The success to me was seeing so many new faces in the library. I continue to see some of them even after the close of the program, saying hello and of course checking out books.”

Los Angeles Public Library

LAPL has embraced outcomes-based summer reading. Serving the City of Los Angeles with its Central Library and seventy-two neighborhood branches, LAPL has offered a summer reading program since 1930. Until 2011, success was measured only by output measures such as the number of children and teens registered for the program; the number of events and activities offered; and the number of people attending those events and activities.

While these numbers are important, they only tell part of the story. Many children and teens who register for the summer reading program come to the library all summer long. But what are they getting out of the program? And who are these kids? Are they library regulars or new library patrons? To answer these questions and to plan a summer reading program that actively sets out to achieve its goals, LAPL participated in CLA’s outcomes-based reading initiative in 2011 and 2012.

LAPL has three main goals for its summer reading program: (1) to encourage kids and teens to read for pleasure; (2) to encourage kids, teens, and families to see the library as a welcoming and engaging place; and (3) to attract non-library users to the library. These mesh neatly with CLA’s two outcomes—LAPL’s first two goals could be restated as CLA’s outcome one, while LAPL’s third goal works perfectly with CLA’s outcome two.

In 2011 and 2012, all seventy-two branches and the central library participated in measuring outcome one for both the children’s and teen programs. LAPL used CLA’s survey, adding several questions at the end to gather additional data needed for in-house reporting. Every agency distributed surveys during the last month of the summer reading club and sent the completed surveys to the youth services coordinating office for collation. Because the data was being collated for the library system as a whole and not for each branch, agencies were encouraged to collect as many surveys as possible but were not given a minimum required number.

In 2011, the final number of received and collated surveys was somewhat disappointing: out of the more than 20,000 registered children, only 1,984 turned in completed surveys; and out of the more than 5,000 registered teens, only 611 turned in completed surveys. In 2012 we did much better, collecting 3,836 surveys from over 26,000 registered children and 1,478 surveys from over 6,700 registered teens.

We gleaned intriguing and important information from these surveys. The most popular answer to the question “Which of the following words describe your library?” for both children and teens was “The library is a place to find books to read.” Both children and teens indicated that they like to talk about and share the books they read, and both kids and teens love the summer reading program. In 2012, 67 percent of both children and teen survey respondents were first-time summer reading program participants, and more than 80 percent planned to visit the library after summer was over.

LAPL also measured CLA’s outcome two. Each children’s and young adult (YA) librarian was asked to work with branch managers to identify a group in the community that was not currently participating in the library’s summer reading program. Once the group was identified, librarians set a goal of how many members of that group would participate in the summer reading program at their branch. Finally, librarians specified how they planned to achieve this goal. All librarians received an orientation to this process during a professional information meeting.

In 2011, all LAPL branches took part in this process, but six branches were selected to be pilot sites. The children’s and YA librarians at these branches received more orientation and training on outcomes-based planning and were encouraged to conduct focus groups in addition to collecting surveys. Only these six branches reported back to CLA on the results of outcome two. In 2012, all branches and the Central Library participated fully in measuring outcome two, reporting at the end of summer on how many members of their targeted group actually participated in summer reading as well as on any success stories or lessons learned.

The underserved groups identified by the librarians, as well as the number they hoped would participate in the 2011 and 2012 summer reading programs, varied widely from branch to branch. Examples of goals include:

• five recently arrived Chinese-speaking families;
• twenty-five kids from a local low-performing elementary school;
• five families from a local housing project;
• twenty-five Latino teens; and
• forty teens ages fifteen to eighteen.

In 2011, when asked how they planned to achieve their goal, most librarians outlined an outreach plan, ranging from distributing flyers in their identified group’s home language to visiting a targeted school to offering incentives to those who actually come to the library to sign up. However, in only rare cases did librarians indicate that they would run the summer reading program in their branch specifically to appeal to their identified group.

In 2012, more emphasis was given during staff training on how to use desired outcomes to plan a relevant and successful summer reading program. If a children’s librarian wanted to offer a summer reading program that was welcoming to newly arrived Spanish-speaking families, then the programming and collection should reflect that goal. If a teen librarian wanted older teens to take part in the summer reading program, there needed to be something to attract these older teens and keep them coming back. Children’s and teen librarians were invited and encouraged to try new approaches in order to attract new summer reading participants and excite returning participants, including taking the program to summer camps, recreation centers, and housing shelters using our group summer reading kits. Many of our librarians rose to the challenge and created vibrant and successful programs—from enlisting teens to work with local artists to create a “library kazari” for the Los Angeles Tanabata Festival
to offering storytimes to children on the autism spectrum at the Help Group.

The 2012 results for outcome two were heartening. Librarians reported that 3,329 children and 818 teens from specifically targeted underserved groups participated in the summer reading program. When added to the fact that 67 percent of survey respondents indicated they were firsttime
summer reading participants, it’s clear that our librarians are offering and marketing a summer reading program that appeals to new library users.

LAPL will continue planning our summer reading program based on the outcomes we are trying to achieve. It’s a challenging process but well worth the effort. Our librarians have seen first-hand what happens when they plan programs with their communities in mind. Our stakeholders
have seen that the library is actively reaching out to all members of the community, not just the ones that already visit the library regularly. And our summer reading program just keeps getting better!

What’s Next?

The California Summer Reading Outcomes Initiative is enabling librarians to move towards more thoughtful and relevant summer programming and it is generating data that will help California’s summer reading programs stay vital, dynamic, and relevant.

Transitioning to any new service model can take time and resources. California public librarians are as overworked and underfinanced as any in the country. Many find it difficult to take on a new way of thinking and doing business at a time when they are challenged simply to provide basic services. Yet those who have participated have found that the initial investment in time has paid off with some unexpected dividends. Staff  members have acquired new evaluation skills. Administrators, board members, and other significant stakeholders are impressed with the results. And the more intentional and focused approach to summer reading has given it new life in many libraries.

Participating librarians tell us about the benefits they’ve derived from developing outcomes-based summer reading programs. For example:

• “Outcome two forced us out of our comfort zone and made us think about who is out there that is not being served.”
• “Focus groups gave tons of insight about what to do and how to communicate better with children and teens.”
• “The purposefulness of creating a community of readers helped greatly in making tough decisions about programming in our small and short-staffed library.”

However, despite these successes and the positive results generated by participating libraries, our numbers are small and the initiative is still nascent. To encourage more libraries to begin presenting outcomes-based summer reading in the future, we plan to: leverage the positive results and feedback to introduce more California libraries to outcomes-based summer reading; continue working with participating libraries to help them use their results as benchmark data to be improved on in future years; and disseminate information to libraries in other states. The Illinois Library Association’s iREAD summer reading program now includes California’s outcomes materials in the print and electronic resource guide it distributes to libraries nationally and internationally, which we anticipate will encourage and enable more libraries to embrace outcomes-based summer reading.

We will continue to develop new resources and provide extensive materials on CLA’s website. However, we know there is no substitute for in-person training to help librarians learn the techniques they need to present successful and enduring outcomes-based summer reading programs. We held a successful ALA preconference workshop on the initiative in June 2012, and we will conduct another ALA preconference in June 2013. In addition, we are beginning to provide smaller, personalized trainings for individual library jurisdictions.

And now that the initiative is beginning to generate data that demonstrate the value and impact of summer reading, CLA will combine these results with output statistics and with research on the value of reading and library use, and begin using them to publicize—to library communities, the media, politicians, funders, and other stakeholders—the value and impact of California’s public library summer reading programs.

We are asking for a culture shift in libraries at a time when staff and other resources are limited and we know that it will take time to facilitate  outcomes-based summer reading more widely. However, at times of limited resources, we all need to be targeting our resources to community need and demonstrating the results of our efforts, and outcomes-based programming is the way to do this. The road to widespread outcomes-based summer reading might be a long one, but the results are definitely worth it!

REFERENCES AND NOTES

1. The California Summer Outcomes Initiative is part of CLA’s California Summer Reading Program which provides California libraries with summer reading materials developed by the Illinois Library Association’s iREAD program, and with training and resources to help libraries make the most of those materials. The California Summer Reading Program is supported by the U.S. Institute of Museum and Library Services under the provisions of the Library Services and Technology Act, administered in California by the state librarian.
2. Rhea Joyce Rubin, Demonstrating Results: Using Outcome Measurement in Your Library (Chicago: ALA, 2006), 12–13.
3. Eliza T. Dresang, Melissa Gross, and Leslie Edmonds Holt, Dynamic Youth Services through Outcome-Based Planning and Evaluation (Chicago: ALA, 2006), 15–16.
4. Virginia A. Walter, “Documenting the Results of Good Intentions: Applying Outcomes Evaluation to Library Services for Children,” Advances in Librarianship 35 (2012), 47-62.
5. Susan Roman, Deborah T. Carran, and Carole D. Fiore, The Dominican Study: Public Library Summer Reading Programs Close the Reading Gap (River Forest, Ill.: Dominican University Graduate School of Library & Information Science, 2010), 47, accessed Mar. 27, 2013.
6. Francine Fialkoff, “As Communities Shift to Evidence-Based Measures, Libraries Must, Too,” Library Journal, Jan. 26, 2012, accessed Mar. 15, 2013.
7. County of Los Angeles Public Library, Fresno County Free Library, Long Beach Public Library, Monterey Public Library, Oakland Public Library, San Bernardino County Library, San Diego County Library, Santa Cruz Public Libraries, and Santa Monica Public Library.
8. Roman, Carran, and Fiore, The Dominican Study.
9. Arcadia Public Library, Glendora Public Library, Imperial County Library, Long Beach Public Library, Los Angeles Public Library, Moorpark Public Library, Oakland Public Library, Orland Free Library, Sacramento Public Library, San Bernardino Public Library, San Francisco Public Library, San Luis Obispo Public Library, Santa Monica Public Library, South Pasadena Public Library, Tulare County Library, Whittier Public Library, Willows Public Library, and Yolo County Library.
10. Stephen D. Krashen, The Power of Reading: Insights From the Research, Second Edition (Westport, Conn.: Libraries Unlimited; Portsmouth, N.H.: Heinemann, 2004); Catherine Sheldrick Ross, Lynne McKechnie, and Paulette M. Rothbauer, Reading Matters: What the Research Reveals about Reading, Libraries, and Community. (Westport, Conn.: Libraries Unlimited, 2006); Joan C. Durrance and Karen E. Fisher, How Libraries Help: A Guide to Identifying User-Centered Outcomes. (Chicago: ALA, 2005).
11. Full information about the strategy can be found on the website of Partnership for Children and Youth.