Still, for all the advocates behind the evidence-based policy movement, others are reluctant to embrace it, partly because they worry that critical assessments will lead to funding cuts. Traditional rigorous evaluations can also be expensive and especially intimidating for small nonprofit groups that manage shoestring budgets and operate on goodwill grounded in compelling anecdotal evidence.
“You will often hear nonprofits push back against evaluation and evidence-based practice by saying, ‘You can’t really measure what we do and the impact we have,’” said Mat Despard, Ph.D. ’15, a former clinical associate professor at UNC’s School of Social Work. “Nonprofits are capable of evaluating their efforts, as long as they consider a full range of methods appropriate for the questions they are trying to answer. For some, this may mean using semi-structured interviews to assess highly complex issues. For others, use of a control or comparison group and standardized measures or analysis of administrative data may be feasible. The point is to discern in some systematic way whether the people engaging in the nonprofit’s services are experiencing quality of life improvements.”
Randomized controlled trials, also known as RCTs, are gaining more attention, including bipartisan support for low-cost RCTs, as a less expensive way for determining an intervention’s effectiveness. Low-cost RCTs track outcomes for a group of people who receive treatment and a comparison group that does not. These low-cost studies are considered more affordable than traditional RCTs because they rely on data that public, private or nonprofit agencies already collect, such as emergency room visits, the number of children in foster care, or graduation rates.
Testa is among those who think low-cost RCTs can offer states more flexibility for testing promising ideas and for increasing the amount of sound evidence in public child welfare. This spring, he was awarded a $96,000 grant from the Laura and John Arnold Foundation to conduct a low-cost RCT for Safe Families for Children. The national nonprofit recruits and oversees a voluntary network of host families with whom parents in crisis can briefly place their children in times of need to prevent them from falling into the foster care system.
Because the nonprofit helps to reunite children with their parents usually in less than two months, supporters say the program saves taxpayer dollars by avoiding the costs of foster care, which in Illinois, averages $41,000 per child over a typical two-year length of stay. Testa’s study could offer evidence Safe Families for Children needs to demonstrate that its program is just as safe as foster care to justify future funding.
Ultimately, evaluations can help nonprofits better understand what is working—a goal all should be aiming for, added Despard, now an assistant professor at the School of Social Work at the University of Michigan.
“If given the choice of understanding whether your programs are achieving their intended outcomes and impacting the people you are trying to help versus choosing not to understand, the choice is pretty simple,” he said. “If you assert that you care about the community you serve, then it follows that you would want to know that your efforts are making a difference.”