AI-generated college admissions essays exhibit male, privileged bias

 In an examination of thousands of human-written college admissions essays and those generated by AI, researchers found that the AI-generated essays are most similar to essays authored by students who are males, with higher socioeconomic status and higher levels of social privilege.

The paper, published in the Journal of Big Data, also found the AI-generated writing is also less varied than that written by humans.

“We wanted to find out what these patterns that we see in human-written essays look like in a ChatGPT world,” said AJ Alvero, assistant research professor of information science at Cornell University and co-corresponding author of the study. “If there is the strong connection with human writing and identity, how does that compare in AI-written essays?”

Alvero and the team compared the writing style of more than 150,000 college admissions essays, submitted to both the University of California system and an engineering program at an elite East Coast private university, with a set of more than 25,000 essays generated with GPT-3.5 and GPT-4 prompted to respond to the same essay questions as the human applicants.

For their analysis, the researchers used the Linguistic Inquiry and Word Count, a program that counts the frequencies of writing features, such as punctuation and pronoun usage, and cross-references those counts with an external dictionary.

Alvero and the team found that while the writing styles of large language models (LLMs) don’t represent any particular group in social comparison analyses, they do “sound,” in terms of word selection and usage, most like male students who came from more privileged locations and backgrounds.

For example, AI was found on average to use longer words (six or more letters) than human writers. Also, AI-generated writing tended to have less variety than essays written by humans, although it more closely resembled essays from private-school applicants than those from public-school students.

Additionally, humans and AI tend to write about affiliations (with groups, people, organizations and friends) at similar rates – despite the AI not actually having any affiliations.

As LLMs like ChatGPT become more popular and more refined, they will be used in all sorts of settings – including college admissions.

“It’s likely that students are going to be using AI to help them craft these essays – probably not asking it to just write the whole thing, but rather asking it for help and feedback,” said Rene Kizilcec, associate professor of information science at Cornell and co-author of the paper. “But even then, the suggestions that these models will make may not be well aligned with the values, the sort of linguistic style, that would be an authentic expression of those students.

“It’s important to remember that if you use an AI to help you write an essay, it's probably going to sound less like you and more like something quite generic,” he said. “And students need to know that for the people reading these essays, it won’t be too difficult for them to figure out who has used AI extensively. The key will be to use it to help students tell their own stories and to enhance what they want to convey, not to replace their own voice.”

For additional information, read this Cornell Chronicle story.