A library system’s evolving role leads to evolving data-tracking practices.
- York County, Pennsylvania
- 100 Staff
Data is dynamic, not static, and so are organizations—which means that over time, the data they need to track changes as they change. Identifying those evolving data points takes effort and thoughtfulness, especially for an organization whose staff is already burdened tracking data for funder-mandated reporting requirements.
Pennsylvania’s York County Library System, which serves as many as 450,000 people each year through a 13-library network made up of a combination of branches with a shared central board of directors and management and independent libraries with their own budgets and directors, is going through that scenario right now. The library system tracks multiple data points as a means of monitoring the services it provides patrons and the ways in which it interacts with them, and wants to do more. It’s also working to identify and define new data points as the way the library serves patrons evolves over time.
But staff spends a lot of time tracking rigorous state funding requirements, said Director of Information Services Lora-Lynn Kahler.
“The state requires us to capture and report certain data twice a year, every year,” she said. “We also use data to inform us around decisions so we can measure achievement of progress and achievement of goals to help us decide which way to go on an issue. We want to be able to look at actual data to help us make decisions.”
Making the job more challenging is that the states data-tracking requirements dont particularly overlap with those the library would like to do for its own use.
“Every March or May, the state asks libraries for a ‘reference survey,’ a count of how many traditional reference questions were asked,” she said. “It’s an outdated measurement.” Similarly, other mandated data requirements “don’t, in our opinion, necessarily give a full picture of what it is a library does to provide service,” Lora-Lynn said.
“A lot of the data they ask for is relevant—most of it, even—and we turn it around however we can for our own internal use. It’s everything from collections to local funders. Some of it is benchmarks: the amount of circulation, how many things do we put in peoples homes, and how many get returned, which measure how effective we are in putting things into people’s homes.”
“There are some pieces of data I know my interlibrary loan staff literally spend four hours a month to cull out just to dump into an annual report once a year thats required by the state,” she said. “Would we collect that data otherwise? I don’t know—our executive director did a scrubbing of all our statistical reports two years ago to look at what were keeping, why, who’s keeping it, and how, and we eliminated some of the stuff that we didn’t need.”
That mandated data collection puts a strain on staff time that might otherwise be spent using data to improve services.
“Our biggest piece of the puzzle is strategic planning,” Lora-Lynn said. “Since 1999 we’ve engaged in strategic planning every year. Part of that is agreeing on routine critical measures of success. What is the data we keep year after year to measure overall success and growth? It paints a very interesting picture of everything from door counters to number of people who have committed to our legacy society—people who’ve told us they’ve mentioned us in their will or estate—to the number of materials that we own. So many things.”
“We’re also trying to get our arms around how we can envision a new service and look at outcomes and data measurement to determine success immediately, she said. We’re not there yet, but there’s where we want to go—basically, whether we’ve done what we set out to do.”
That practice has been tested as the role libraries play has changed with time, she said. What was essentially once a place where patrons sought the help of reference librarians has become a place where they go to use public access computers. The role of the library staff has evolved with those changes.
“We spend more time helping people learn to use a mouse and access Gmail and helping them to apply for a job by sending a resume over Gmail than we do teaching them to use encyclopedias, and that’s a real change,” Lora-Lynn said. “The questions people are asking and the things they need from us have changed drastically. The whole rise of PCs and mobile devices has been transformative.”
“We need to rethink the services were providing and how we measure them. We’ve been able to automate some of that process with auto sign-on and sign-off and timers on the computers, things like that that have made it much easier to calculate unique users each month, the amount of time they stay on a PC—we don’t have to use log books anymore,” she said. “Automating the process is always the better way to go. The more you can have that raw data automatically captured, the more accurate its going to be. It also removes the human element.”
Even identifying new data points is often not enough, she said—it’s also critical to define what those data points actually measure for consistency. As an example, she cited “check outs,” which could mean the initial time a book is checked out or each renewal. “We’ve had some difficulty this past year in just clearly defining what a particular data point is, how we arrive at it, and where do we get it from, so that everyone who refers to it knows exactly what were talking about,” she said. “When you run different reports, you think you’re measuring the same thing, but you’re not. It depends on how its measured and who defines it.”
Lora-Lynn sees herself as a big believer in the value of data. She also sees herself as more of the exception than the rule.
“What I’m not sure about is how many of our individual libraries actually go to our systems and pull out reports and actually get data,” she said. “We have a monthly report and its pulling teeth for some people. They are provided with some data on a monthly basis and maybe that’s enough for them and their boards, but that’s not enough for me. I do some of the analysis, a lot of it is done by the CEO and president of the main library, and the administration, and our director of automation.”
“There’s a lot of advising from senior management and some frontline staff, but I think we need to do more with data, if anything,” she said. “More reviewing whats captured and applying more of it to programs and coming up with new ideas for programs. What is the purpose of this program and how do we measure it?”
“We need to get that level of thinking into our professional staff because it’s not there yet. We have a tough time letting go of things, things we’ve always done that no longer have a good purpose. There’s an emotional attachment sometimes. We can’t stay here and do more, so we need to decide.”
This case study is part of the research project in 2012 conducted by NTEN with the help of Idealware. See the State of Nonprofit Data report for more information about how nonprofits are–and aren’t–making data part of their decision-making processes, and the key challenges that affect an organization’s ability to be more effectively “data-driven.”