ChatGPT articles are in danger of using up all available stock photos of typing robots
I’m the director of an essay contest for local high school students, and as I prepared this year’s prompt, I couldn’t resist the urge to let ChatGPT take a shot at it. The results highlight several of the reasons that ChatGPT is not likely to overtake the top spot in English classes across the country.
Margaret Feldman graduated from our local high school as a standout athlete and scholar. After college, she became part of the fledgling Office of Strategic Services (OSS), the precursor of the CIA. When her father died after WWII, she came home to take over his business and became one of the first women to run an industrial company. Then she entered the classroom, inspiring and mentoring a generation of students and fellow teachers. After her death, a foundation was established in her name; that foundation has long run an essay competition. In recognition of her work and her devotion to the area, the prompt is always literature based and additionally asks students to make a connection to local young people.
This year’s prompt asks the writer to show how a character in a major is affected by their relationship with the larger community, and then to show what local teens can learn from that work.
I gave ChatGPT three tries, selecting a work for it to use in responding to the prompt, partly out of curiosity, but also to test the prompt itself. If a chatbot can write a great response to the prompt, then the prompt is wasting students’ time.
First Try
The first attempt used William Faulkner’s Light In August. The bot’s response was smoothly composed, but it highlighted ChatGPT’s tendency to write around the point. Take this section:
Through these experiences, Joe Christmas gradually comes to a deeper understanding of his own identity and the ways in which he is shaped by the larger communities around him.
That sounds great—but what is Christmas’s understanding, exactly? What are the ways he is shaped? Like much of this essay, the sentence promises there’s a point coming, but it never arrives.
Second Try
For the next attempt, I added a specific requirement to include quotes from the work. This is a major weakness of ChatGPT. For the second attempt, ChatGPT was told to use Romo and Juliet, and it included this:
Through his relationship with Juliet, Romeo also experiences a sense of rebellion against the social norms and expectations of his community. He is willing to risk everything, including his own life, to be with Juliet, even though their love is forbidden by their families:
"O, let us hence! I stand on sudden haste." "Wisely and slow. They stumble that run fast" (Act 2, Scene 3).
The quote does not have anything to do with Romeo’s willingness to face risk, nor does the bot add anything to explain how the quote supports the point it is positioned to support. ChatGPT uses quotes exactly like a freshman who, realizing the assignment called for quotes, opens the text and drops their finger at random, then just inserts the quote into their essay.
Third Try
Finally, ChatGPT took a whack at Hamlet. This essay highlighted ChatGPT’s trouble with accuracy. In the bot’s version of the play, Hamlet “sacrifices his own life in order to achieve justice for his father and to challenge the corrupt power structures...” and includes this interpretation:
He recognizes the role that his family and the court play in perpetuating the cycle of violence and betrayal that has defined their lives, and begins to question the legitimacy of revenge as a means of achieving justice:
We could at best call these unconventional readings of the play. And once again, the bot backs up its stated idea with a random quote:
"I'll take the ghost's word for a thousand pound. Didst perceive? Very well, my lord. Upon the talk of the poisoning? I did very well note him" (Act 3, Scene 2).
Mysteriously, ChatGPT has combined quotes from two different characters into one line, and again provides no explanation of what that quote has to do with the point being presented.
Other issues
Since ChatGPT works by aggregating and smooshing together everything it has read from the internet, it seems possible that works like Shakespearean plays are so well-covered that it gives the bot way too much to work with, leading to widely varied chunks of content. There were other issues as well.
Perhaps because the prompt asks the writer to select and write about a single character, none of ChatGPT’s essays mention any other characters by name.
As for the requirement to connect the ideas to the local area, ChatGPT was stumped, and simply resorted to generic platitudes that could be applied to any community in the country.
All three essays stayed away from concrete and clear specifics. English teachers will be reminded of essays they’ve read based on the text printed on the covers of books.
What ChatGPT has in spades is a sort of fluency and facility that makes it sound as if it’s saying something. It’s a print version of Will Stephen’s TED Talk, “How To Sound Smart in Your TED Talk,” in which Stephens says nothing at all, but uses the cadence and tone of his voice to simulate a meaningful speech.
ChatGPT creates a very shiny vessel, and that will undoubtedly fool many folks, particularly those who have no familiarity with the content that the shiny vessel is supposed to contain. But if Margaret Feldman were alive today, she would not be fooled for ten seconds.
