Can Artificial Intelligence Crack College Admissions?

Since the unveiling of OpenAI’s ChatGPT in late November 2022, the large language model (LLM) has been the source of endless speculation—experts have predicted that the technology will do everything from abolishing high school English classes to displacing white-collar workers to highjacking democracy. The program has caused particular consternation within the education sector, as school systems across the country have banned its use, prompted by fears that it will become a tool for widespread cheating. The concerns are understandable given the program’s sophistication—this week, ChatGPT passed both the United States Medical Licensing Exam (USMLE) and a final exam in a Wharton School MBA course.

Given the program’s ability to pass a high-level course at a prestigious university, can it successfully earn a student admission to a similarly high-caliber school? Can ChatGPT craft an admissions essay that stands out in the competitive world of Ivy League admissions?

Analyses of the program’s performance on the USMLE and MBA exam offer some helpful insights into what ChatGPT can and cannot do within the landscape of college admissions. In both cases, perhaps the most remarkable aspect of the program’s performance was its ability to respond to case studies, arriving at and explaining conclusions that did not pull directly from dataset input. LLMs, of which ChatGPT is perhaps the most sophisticated, operate by using a massive amount of language input to generate and predict text based on probability. ChatGPT is particularly remarkable due to its impressive and complex ability to mimic the nuances of human speech and to generate responses that are highly contextually informed.

Both the USMLE and the MBA exam included questions with nuanced scenarios, often with difficult or misleading wording—in other words, questions and problems that ostensibly require deductive reasoning. Despite the dense and tricky nature of the questions, ChatGPT was able to provide well-reasoned and largely correct answers. Christian Terwiesch, Co-Director of the Mack Institute for Innovation Management and the researcher who administered the MBA exam, writes that ChatGPT “does an amazing job at basic operations management and process analysis questions including those that are based on case studies. Not only are the answers correct, but the explanations are excellent.” Similarly, the researchers analyzing the program’s results on the USMLE found that “AI-generated responses also offered significant insight, role-modeling a deductive reasoning process valuable to human learners…At least one significant insight was present in approximately 90% of outputs. ChatGPT therefore possesses the partial ability to teach medicine by surfacing novel and nonobvious concepts that may not be in learners’ sphere of awareness.” While other prior iterations of LLMs cull answers from data directly input into the programs, ChatGPT was able to detect information adjacent to that which had been input, creating the appearance of independent reasoning.

However, appearance is the critical term. While it is an incredible feat that the program could respond to nuances in the scenarios put forth—nuances that we typically associate with human reasoning—using inferences to relate the scenarios to adjacent information contained within its dataset is not the same as human reasoning.

So what does this mean for the college admissions essay?

While the program is well equipped to pass even complex tests, it cannot be a substitute for innovation, creativity, and critical thinking—attributes that students are expected to showcase on the college admissions essay. The most successful essays clearly convey a student’s unique personality. As Forbes’ Emma Whitford has demonstrated, with clear instructions, ChatGPT can produce a straightforward admissions essay, but the examples generated lack the spunk and imagination that stand out to top schools.

The University of Chicago’s “uncommon essay” prompts further illustrate ChatGPT’s inability to conduct human reasoning. The prompts require students to approach topics with ingenuity and originality. One of the most notorious prompts in the past asked students to simply “Find x.” When I plugged this prompt directly into ChatGPT, the program provided a standard explanation: “x is a variable that can represent any number or mathematical expression. It is usually used in equations to represent the unknown quantity that needs to be solved for.”

Even with more specific instructions, the program failed to craft a unique or truly creative response. When I instructed the program to “write a creative essay for the prompt ‘Find x,’” it generated this essay:

It was the summer of my senior year of high school, and I was on the brink of discovering something extraordinary. I had spent the past few months studying mathematics and physics, absorbing every detail with an eagerness that only a curious mind can appreciate. One day, my teacher gave us an assignment to “find x” – an equation that could unlock the answers to a number of complex questions.

At first, I was overwhelmed with the task. I felt as if I was searching for a needle in a haystack – and the stakes were high. I knew that I had to find x if I wanted to fulfill my dream of becoming a successful scientist. With a determination that rivals a lion’s, I set to work.

I read and reread my textbooks, pouring over each equation and concept with a magnifying glass. I experimented with different approaches, running into dead ends and wrong turns, but never giving up. I was determined to find x.

Finally, after weeks of hard work and dedication, I had a breakthrough. I had an epiphany – I had found x! It was an equation that had the potential to revolutionize the world of mathematics and physics.

I was ecstatic. I was so proud of myself for never giving up, even when the odds were against me. I had done the impossible – I had found x!

This experience taught me that perseverance, hard work, and a little bit of luck can help you achieve anything. Even if you feel like you’re looking for a needle in a haystack, never give up – you may just find x.

Although the prose reads like an excited high school student, the specifics of the essay itself leave much to be desired. In fact, ChatGPT circumvents the prompt entirely by describing the process of finding x rather than actually finding x—or at the very least, defining it.

Of course, this is all not to mention that a student’s essay, like ChatGPT, is only as good as its input. Prestigious colleges are devoting increasing attention to students’ activities and how those activities provide a cohesive and compelling picture of who the student is and what they are passionate about. A great essay will tie all of those activities together in a holistic narrative with gusto and creativity, but no essay—however well-written—can artificially manufacture four years of actionable engagement with one’s interests.

Understanding ChatGPT’s limitations is almost as important as recognizing its strengths. While the program has much to offer students in a variety of different fields, it is not a substitute for a student’s own voice and reasoning. Advances in the world of artificial intelligence should propel colleges to increasingly craft essay prompts that encourage critical thinking and originality and continue to emphasize student’s activities and demonstrable interests as a primary consideration in the admissions process.