- Lexington, KY
- 5 Staff
How does an advocacy organization measure its effectiveness in a data-driven way? Rich Seckel, director of the Kentucky Equal Justice Center, admits it can be a challenge. It’s difficult to be in a multi-variant world trying to prove causality,” he said.
The poverty law organization advocates on behalf of Kentuckians in need, serving as a watchdog for the state legislature, keeping an eye on bills that are filed and lobbying lawmakers. Rich said measuring effectiveness through data boils down showing funders data that supports not just his nonprofits impact on policy changes, but on the lives of people who benefit from its victories.
In 2003, with the state under intense pressure from the federal government to cut Medicare spending, the governor announced a series of changes that would save $45 million but would terminate nursing home or home healthcare coverage for 3,300 disadvantaged Kentuckians. To challenge the cuts, KEJC enlisted the National Senior Citizens Law Center and sued the state on the basis that the measures were solely to save money and therefore illegal. However, they didn’t have data to support the case that the cuts were unjust.
The Medicaid cuts were fairly unpopular with the electorate, so during the next gubernatorial election both candidates promised “not to kick people out of nursing homes.” When a new governor was elected the following year, counsel for the state agency expressed interest in settling the case. In the settlement, the agency pledged a return to the earlier standards, to review all people who had been denied or terminated from long term care, and to report on the results. Using that data, Kentucky Equal Justice Center found that 97 percent of those who had been denied benefits or whose benefits had been terminated had been restored long term care coverage – about 3,300 of them.
Three of the 10 plaintiffs named in the lawsuit KEJC filed contesting the cuts died before the issue was resolved, demonstrating not just the seriousness of the issue but that access to the right data can make or break an advocacy groups case.
KEJC’s work directly improved the lives of more than 3,000 Kentuckians, but often the situation isn’t so clear-cut, Rich said. Just about every funder he works with wants KEJC to evaluate its success in human terms, which can be tricky though; policy successes can be easily measured, the results cannot. The data he needs to demonstrate these results often come from such external sources as state agencies, he said, and can be hidden under layers of bureaucracy. One way around that obstacle that has worked for him, Rich said, is to establish relationships with friendly legislators or state government employees who will share reports that never get released to the public, but which have the data that he needs to show his organizations impact.
As part of a foundation grant given KEJC to boost its infrastructure, Rich was required to meet periodically with a trained evaluator who helped him understand the science behind the statistics the organization was tracking. That statistical analysis training showed him how many different variants come into play in the organizations work, and the different ways they can and should be interpretedin other words, he said, it gave him a conscience about ascribing too much change to the efforts of his nonprofit.
“We don’t brag too much,” he said, but conceded that maybe the organization should work a little harder to publicize its measurable results. “We probably need to get better at that.”
This case study is part of the research project in 2012 conducted by NTEN with the help of Idealware. See the State of Nonprofit Data report for more information about how nonprofits are–and aren’t–making data part of their decision-making processes, and the key challenges that affect an organization’s ability to be more effectively “data-driven.”