Subjective Evaluations in Graduate Education

This study introduces a model of the admissions process based upon standard agency theory and explore its implications with economics PhD admissions data from 2013-2019. The authors show that a subjective score that aggregates subjective ratings and recommendation letter features plays a more important role in determining admissions than an objective score based upon graduate record exam (GRE) scores. Subjective evaluations by references who write multiple letters are not only more influential than those of references who write one letter, but they are also more informative. Since multiple-letter references are also more highly ranked economists, this implies that there is a constraint on the supply of high-quality references. 

Moreover, the authors find that both the subjective and objective scores are correlated with job placement at a top economics department after the completion of the PhD. These indicators of individual achievement have a smaller effect than an undergraduate degree from an Ivy Plus school (i.e., Ivy League + Stanford, MIT, Duke, and Chicago). In the self-selected pool of applicants, Ivy Plus graduates are twice as likely to be admitted to a top 10 graduate program and are much more likely to obtain an assistant professor position at a top 10 program upon PhD completion. Given that Ivy Plus students must pass a stringent selection process to gain admission to their undergraduate program, it is possible e hypothesis that admission committees use information efficiently and fairly. However, this also implies that there may be a return to attending a selective undergraduate program in order to be pooled with highly skilled individuals.