Grit better than GRE at predicting success in STEM fields

Selecting graduate students in the fields of science and engineering based on an assessment of their character instead of relying almost entirely on their scores on a standardized test would significantly improve the quality of the students that are admitted and, at the same time, boost the participation of women and minorities in these key disciplines.

That is the argument made in the essay "A test that fails" published in the June 12 issue of the journal Nature. The authors are Associate Professor of Physics Casey Miller of the University of South Florida and Keivan Stassun, professor of physics and astronomy at Vanderbilt University and Fisk University, who are both involved in successful bridge programs designed to improve PhD completion rates among all students and to boost women and underrepresented minority participation in the fields of science, technology, engineering and math (STEM).

According to the authors, the primary reason that half of all American PhD students fail to graduate, and the primary barrier holding back women and minority students is American academia's over-reliance on the GRE, the graduate record examination, a standardized test introduced in 1949 that most US graduate schools require for admission. The problem is that the exam's quantitative score – the part measuring math ability – is not a good predictor of a student's ultimate success, particularly in the STEM fields. Women, on average, score 80 points on average lower in the physical sciences than men and African Americans score 200 points below whites. At the same time, studies performed by ETS, the company that administers the test, have found that the test's predictive ability is limited to first-year graduate course grades and even that is questionable in STEM fields.

"In simple terms, the GRE is a better indicator of sex and skin color than of ability and ultimate success," the article states.

Despite its demonstrable demographic bias, graduate-admissions committees routinely use minimum GRE scores to filter applications. A typical procedure is to reject the application of any candidate scoring less than 700 on the 800-point quantitative section, despite the fact that this practice violates ETS guidelines.

"The misuse of GRE scores to select applicants may be a strong driver of the continuing under-representation of women and minorities in graduate school. Indeed, women earn hardly 20 percent of US physical sciences PhDs and underrepresented minorities – who account for 33 percent of US university-age population – earn just 6 percent. These percentages are striking in their similarity to the percentage of students who score above 700 on the GRE Quantitative Measure," the article points out.

Miller and Stassun propose an alternative approach to the selection process, which has proven successful in the bridge programs with which they are involved: Using a 30 minute face-to-face interview that examines an individual's college and research experiences, key relationships, leadership experience, service to the community, and life goals. This provides committee members with a good indication not only of the person's academic training and aptitude but also of the other competencies that point to a likelihood of success in graduate school and a STEM career.

The validation for this approach is the track record of the students in their programs. At the Fisk-Vanderbilt bridge program, for example, 85 percent of the students would have been eliminated by the 700-point GRE cutoff. However, 81 percent of the 67 students who have entered the program – including 56 underrepresented minorities and 35 women – have earned, or are making good progress toward their PhDs and all the students who have received their doctorates have found employment in the STEM workforce, as post doctoral students, university faculty members or staff scientists in national labs or industry. This 81 percent success rate is significantly better than the 50 percent national average, which most policy makers agree is an enormous waste of precious human resources.

Miller and Stassun make it clear that they are not advocating the admission of unqualified minorities in the name of social good. Instead, they argue that the nation can swell the ranks of its STEM workforce – a goal that is broadly agreed upon by policy makers to improve the nation's international competitiveness – by adopting a more accurate graduate school admission process that improves the quality of admitted PhD students and that doesn't eliminate large numbers of talented minority and women students who have the right stuff to succeed in these challenging careers.