State higher education performance funding has not improved retention and graduation rates

State higher education performance funding is falling short of its intended goals of raising student retention and degree completion rates at community colleges, according to new research published today in Educational Evaluation and Policy Analysis, a peer-reviewed journal of the American Educational Research Association. The results of the study, which focused on Washington State's Student Achievement Initiative (SAI)--widely recognized as a model for performance accounting systems in the United States--are in line with recent results in Tennessee and Pennsylvania indicating that performance funding has not improved retention and graduation rates.

VIDEO: Co-author Nicholas W. Hillman discusses key findings: https://www.youtube.com/watch?v=OQuv-ls2R4M&feature=youtu.be

Using data for 2002 through 2012 from the U.S. Department of Education's Integrated Postsecondary Education Data System, authors Nicholas W. Hillman of the University of Wisconsin--Madison, David A. Tandberg of Florida State University, and Alisa Hicklin Fryar of the University of Oklahoma, found that retention and degree completion at Washington's community and technical colleges were not distinguishable from the performance of colleges in states without similar accountability policies.

"Tying state financing to college 'performance' is expected to create an incentive for colleges to produce more degree recipients and retain students at higher rates," said Hillman. "However, evidence from the Washington experiment suggests these results do not occur in systematic ways."

Introduced in 2007, Washington's initiative is one of the nation's longest-operating performance funding programs currently in place, and it has been widely recognized as one of the most carefully-designed performance accountability systems in the country.

"Considering the popularity of Washington's performance funding model, we are surprised the impacts on associate's degree productivity are so modest," wrote the authors.

While Washington did not experience a significant increase in retention and associate's degree production following the implementation of SAI, it did see noticeable growth in students earning short-term certificates which, overall, hold little value in the labor market.

The authors noted that original SAI guidelines gave community colleges incentive to expand short-term certificate programs, which are designed to be completed in less than one year, even though it was not one of the primary goals of the state's policy.

"Short-term certificates are an easier approach than two-year completion or graduation. They're the path of least resistance for schools." said Tandberg. "However, short-term certificates often don't provide any benefits over a high school diploma."

In December 2012, Washington's policy was revised and now awards achievement points only for short-term certificates that lead to middle and higher wage jobs or to a more advanced credential.

Since the mid-2000s, performance funding programs, which fund colleges based on institutional "output" measures--including retention and graduation rates--rather than "inputs" such as the number of students enrolled, have gained in popularity. The authors noted that this is in part due to the efforts of a number of influential national organizations, including the Bill & Melinda Gates Foundation, Complete College America, the Lumina Foundation, the National Conference of State Legislatures, and the National Governors Association.

The authors said that the difficulty of retaining students may be a factor more complicated than SAI allows for, noting that improving retention rates requires significant campus resources to address a broad range of factors, including student engagement levels, academic support services, campus climate, student satisfaction, and financial aid.

"There is no easy solution to improving college performance," said Hillman. "Most schools do not have the capacity to make improvements with current resources. That's especially true of community and technical colleges, which are already known for having to do the most with the least amount of resources."