Mission: Accepted! U.S. College Admissions Insights for International Students

links to homepage

Can AI Detection Tools Really Catch AI-Written College Essays?

October 29, 2024

Curious about how reliable AI-detection tools are, I ran an informal experiment using ChatGPT 4o to generate three versions of a college essay. I tested these essays with a range of free AI checkers—ZeroGPT, GetMerlin, Scribbr, GPTZero, and QuillBot—to see if they could accurately identify the text as AI-generated. I also ran them through Claude to see if it would flag them as AI-written.

Key Findings

The first essay was a straightforward response based on the Common App's essay prompt #6. AI detection scores ranged from 21-71%, meaning some tools rated it up to 79% "human-generated." For the second version, I prompted ChatGPT to sound more natural and "human." Interestingly, the scores increased to 45-74% instead of lowering the AI content. Finally, I refined the third version even further by asking for a more varied sentence structure; this approach reduced the AI scores down to 21-47%. Only Claude consistently flagged all three versions as "most likely AI-generated," possibly because, as a large language model, it recognized familiar language patterns used by AI.

These results highlight a concerning issue: the free AI checkers struggle to accurately identify even 100% AI-generated texts. On the other hand, Claude's consistency suggests that advanced LLMs may be better suited to recognize the distinct patterns in AI language. As AI's influence in writing grows, college admissions and other fields face a real challenge in reliably identifying AI-generated content. Today's tools still need significant improvement to keep up with AI's generative abilities. In addition, college admissions must find effective strategies to address the growing use of AI in their policies and evaluation approaches.

Experiment Details 

Effectiveness of AI-Detection Tools on AI-Generated College Essays

Step 1: Initial Essay Generation

I used the following ChatGPT prompt to generate the first essay version:

Write a college essay (650 words or less) for the following prompt: Describe a topic, idea, or concept you find so engaging that it makes you lose all track of time. Why does it captivate you? What or who do you turn to when you want to learn more?


This resulted in Essay Version 1:

It starts as a faint hum, a rhythm that seems woven into the fabric of existence itself. I’m always drawn to this intangible beat, which turns into harmony as I start picking up frequencies, intervals, and resonances in my surroundings. The concept of sound and its relationship to the universe—a symphony unfolding in invisible wavelengths—fascinates me. From birdsong and ocean waves to the hum of machinery, each sound feels like an invitation to explore the science and art behind it. When I’m lost in this world, time fades; there’s only the beauty of sound and its hidden secrets.

Music and sound go beyond just vibrations in the air—they embody energy, physics, mathematics, and emotion. I lose myself in the process of understanding how each element of a sound wave shapes what we hear. As a young musician, I was always aware of sound’s artistic side, but once I learned that every note, every noise, is measurable and can be studied, a whole new dimension opened up. Sound became a bridge between science and art, a way to measure and understand the world around me and to connect with it on a deeper level.

I became captivated by the physics of sound, specifically in how sound waves interact with one another. I started experimenting with tuning forks, realizing that the frequency of one fork could cause another to resonate if they shared a harmonic relationship. It was thrilling to witness the laws of physics come to life in this way. Sound became a language of its own, one that I could "speak" through careful observation and experimentation.

To understand more, I often turn to people who share my fascination. My music teacher, Mr. Simmons, has been an invaluable resource. While he primarily teaches me the piano, he’s also introduced me to the mechanics of sound production, inspiring me to learn more about acoustic physics. We’ll spend hours talking about topics like resonance, harmonics, and even how the shape of a piano influences its sound. These conversations often spill over into my online research, where I’ll find myself deep in articles about wave interference or psychoacoustics, the study of how humans perceive sound.

But my curiosity isn’t confined to the classroom or internet searches. I find myself listening closely to the world around me. The sound of wind rushing through trees and even the rumbling of city buses take on new dimensions as I identify the pitches, frequencies, and rhythms inherent in each. To some, these might seem like random noises; to me, they’re melodies waiting to be understood.

The more I learn, the more I want to experiment. Last year, I borrowed my school’s lab equipment to study the effects of different materials on sound waves. I set up an experiment where I measured how various surfaces affected sound frequencies. For hours, I adjusted the setup, analyzed data, and reveled in every small discovery. I didn’t realize how much time had passed until the school librarian reminded me it was closing time.

This constant sense of discovery fuels me, and I find joy in every layer I uncover about sound. I think it’s the endless complexity that pulls me in. Sound is both simple and complex; it’s something as ordinary as a clap yet as intricate as a symphony. It’s math and art intertwined, science and soul coalescing into one. And as I continue exploring this symphony of the universe, I’m driven by a desire to connect the dots between what I know and what I have yet to learn.

Each new insight is a reminder that I am only beginning to scratch the surface, and that the true beauty of sound lies not just in what I can understand, but in the endless possibilities waiting to be discovered.


I then ran this 100% AI-generated text through various free AI-checkers, resulting in the following detection rates: 

AI Detection Analysis for Version 1

Tool AI Detection Rate (%)

Step 2: First Rewrite to Sound More Human

In the next step, I modified the ChatGPT prompt to: 

Rewrite the text to make it sound more human while keeping the word count.


Essay Version 2

It starts as a faint hum, barely noticeable, like a hidden rhythm woven into the air around me. But soon enough, I’m swept away, listening intently to the sounds that surround me—birdsong, passing traffic, wind in the trees, even the distant hum of machinery. Each sound feels like a new invitation, urging me to understand the science and art behind it. Before I know it, time fades, and I’m completely immersed in a world of sound.

Growing up, I’ve always been drawn to music and sounds in general. But when I learned that each noise or note could be explained scientifically, that the way we hear things is a combination of physics, math, and emotion, I was hooked. Sound became more than a series of vibrations; it turned into a bridge between science and art—a new language I wanted to learn. What started with a love for music has now become a passion for the science of sound, or “acoustics,” as I’ve learned to call it.

Take resonance, for instance. One day, I brought two tuning forks to class. I tapped one, and the other started vibrating as if on its own. It felt almost magical—yet it was all physics! Sound, I realized, is a language of energy, and I could "speak" it by understanding these laws. My music teacher, Mr. Simmons, became my partner in crime. Even though he mainly teaches me piano, he’s always open to talking about the science behind sound. We’ve spent hours going over topics like resonance, harmonics, and how different instruments create their unique tones. When he talks about the physics of a grand piano, it feels like he’s sharing a secret with me.

The more I learn, the more curious I get, and that curiosity doesn’t stay in the classroom. I’m constantly finding myself “listening” to everyday sounds like they’re music. From wind rushing through trees to the faint hum of a refrigerator, each sound now feels like part of a hidden symphony. I’ll catch myself analyzing pitches, rhythms, even subtle frequencies, as if each ordinary noise has its own part to play in a larger composition.

To dig deeper, I’ve taken my experiments outside of music class. Last year, I borrowed the school’s lab equipment to test how different materials affect sound waves. I measured how sound frequencies reacted against various surfaces for hours, taking notes, adjusting the setup, and getting lost in each new discovery. I was completely absorbed and didn’t even notice how late it had gotten until the librarian reminded me the school was closing.

What fascinates me most about sound is how complex it is and how it reveals layers the more you dive in. Sound is both mathematical and musical, ordinary yet incredible. I think I love it because it’s an endless puzzle, and each time I learn something new, I get a little closer to solving it—though I suspect I never fully will. But that’s the beauty of it: no matter how much I uncover, there’s always more to understand.

Sound is this incredible combination of science and art, and I feel lucky to be able to explore all these sides of it. Each new insight feels like a secret that connects me to the world a little more deeply. And in this symphony of vibrations and frequencies, I’m just a curious listener, captivated by the endless song of the universe.


AI Detection Analysis for Version 2

Tool AI Detection Rate (%)

Step 3: Second Rewrite to Sound Even More Human

Final Prompt: Rewrite the text and vary the sentence structure to make the text more human. Stay under 650 words.


Essay Version 3

Sometimes, it’s just a hum. A faint vibration that I might not even notice, buried beneath the everyday sounds of life. But once I start paying attention, that hum unfolds into a full-blown symphony around me, as if every noise and note is part of a grand, invisible song. When I’m lost in sound, the world narrows, and time becomes a distant thought. I don’t just hear things; I feel them, and I can’t stop myself from wondering what each sound is made of, why it’s there, and how it fits into the bigger picture.

I’ve always been drawn to sound, but music was my first love. For a long time, it was just a beautiful art form to me. However, everything changed when I learned that sound is measurable, that it can be studied scientifically. Suddenly, music transformed into a landscape where physics and math met creativity. It opened a door to a whole new world.

One of the most fascinating aspects of sound is resonance. Last spring, I borrowed a couple of tuning forks from my science teacher. I struck one, and to my amazement, the other started to vibrate! It was an extraordinary moment where physics felt like magic. I had unlocked a small secret of the universe. With every sound wave, there was a story waiting to be uncovered. My music teacher, Mr. Simmons, has been my guide through this world. He’s not just a piano teacher; he’s someone who shares my curiosity about sound. Our discussions often meander into the science of how sound is produced and why different instruments have their unique voices. When he describes how a grand piano's shape affects its tone, I feel like I’m learning something profound.

My curiosity doesn’t stop in the classroom. I find myself actively listening to the world around me, turning everyday noises into music. Wind through the trees, traffic outside my window, or the hum of my refrigerator—it all has a rhythm, a melody that begs to be explored. I’ve become an amateur sound detective, analyzing pitches and rhythms as I walk down the street.

To take my learning further, I’ve turned my curiosity into experiments. I once set up a lab project measuring how different materials affect sound waves. I could spend hours in the lab, fiddling with my setup, measuring frequencies, and observing changes as I switched surfaces. Time flew by without me noticing, lost in discovery until the librarian kicked me out at closing time!

What I love most about sound is its infinite complexity. Each new discovery is like peeling back layers of an onion, revealing more to explore. Sound is both simple and intricate, familiar yet mysterious. It’s like a never-ending puzzle that intrigues me. The more I learn, the more I realize how much more there is to know.

As I continue to explore the fascinating world of sound, I see it as an interplay of science and art, a dance of vibrations and frequencies. Each insight brings me closer to understanding the universe, connecting me to something greater than myself. And so, I remain a curious listener, forever captivated by the intricate and beautiful song of existence.


AI Detection Analysis for Version 3

Tool AI Detection Rate (%)

Summary of AI Detection Rates

Conclusion

The findings from this experiment revealed a troubling trend: free AI detection tools had difficulty accurately identifying even texts that were entirely AI-generated. Only Claude showed a consistent ability to detect such content. The results of this small-scale experiment suggests that college admissions must find effective strategies to address the growing use of AI among students. Their policies and assessment approaches need to be updated to incorporate and consider the impact of AI-generated content.

#AIDetection #CollegeAdmissions #AIWriting #AIinCollegeEssays #AIInEducation #FutureOfEducation #CollegeEssays