Questions raised on research on Florida neovouchers

A recent report from Urban Institute has been widely trumpeted as documenting college attainment benefits for students receiving a neovoucher through the Florida Tax Credit scholarship program. But a new review raises concerns and cautions that proper use of the study begins with an understanding of its limitations.

Jaekyung Lee, of the University at Buffalo, SUNY, reviewed The Effects of the Florida Tax Credit Scholarship Program on College Enrollment and Graduation: An Update. Professor Lee found that the report, while making timely and relevant contribution to the research base, provides little guidance for policy and practice.

The report finds neovoucher students were 6-10 percentage points more likely to enroll in some type of college and were one to two percentage points more likely to earn a bachelor’s degree. The study uses expanded data tracking and appropriate research methods that attempt to match neovoucher students to non-neovoucher students with similar baseline characteristics. However, Professor Lee raises three critical questions about methods, findings and conclusions.

First is the problem of selection bias. The study’s attempt to match students resulted in the comparison group having two to three times more students receiving reduced-price lunch. This is on top of the problem of possible unmeasured differences, which the report’s authors briefly acknowledge. Professor Lee notes that choosers tend to be more advantaged in unmeasured aspects than nonchoosers, which tends to result in an upward bias (positive effects on achievement).

Second, the study skips over the question of whether neovoucher students have any measured achievement benefits, at a time when research has raised serious questions about whether students do less well academically when they receive vouchers. Without real achievement benefits, the estimated impact of Florida neovouchers on college enrollment may reflect college matching effects rather than true program effects on students’ college readiness.

Third, the neovoucher program’s reporting of much greater apparent effects on college enrollment than on graduation suggests that the conditional neovoucher effect on college completion (i.e., conditional on college entry) could be null or even negative. Even while many more neovoucher students are heading to college, their likelihood of doing well once they get there is much lower.

These concerns call into question the use and misuse of this study by voucher advocates. The study was used by Florida Governor DeSantis in his State of the State address to advocate for vouchers, and it was dutifully covered by The 74 and The National Review, and even as part of voucher debates in Kentucky (The Courier-Journal) and Nebraska (The Omaha World-Herald). Proper use of the study begins with an understanding of its limitations and sees the need for confirmatory research and future exploration of potential mechanisms driving any increased college attainment.

Find the review, by Jaekyung Lee.

Find The Effects of the Florida Tax Credit Scholarship Program on College Enrollment and Graduation: An Update, written by Matthew Chingos, Tomas Monarrez, and Daniel Kuehn, and published by Urban Institute.