Detecting AI Content: Detectors and Their Limits

Detecting AI content in one sentence

There's no reliable method for looking at a single piece of text and knowing for certain whether an AI wrote it. Detectors deliver probabilities, not proof — and they're wrong often enough that their results can't carry a decision on their own.

That's an uncomfortable answer, but it's the only solid one. Knowing this leads to noticeably better everyday decisions than trusting a percentage from a checking tool.

How AI detectors work

Detectors don't analyze the content — they analyze the statistical structure of a text. Two metrics are central.

Predictability. AI text tends to be made up of words that are very likely at their position — after all, that's exactly what the model selects for. Human text contains more surprises: unusual word choices, abrupt shifts, quirks. A detector measures how “expected” a text is at every point.

Uniformity. Humans write unevenly: a long, nested sentence, then a short one. AI text is rhythmically more homogeneous, with sentence lengths clustering closer together. Detectors measure this variation.

In addition, many tools use a classifier trained on collections of sample human and machine text. What comes out in the end is always the same thing: a percentage that expresses how closely a text resembles the patterns the tool considers machine-made. That is fundamentally different from proof of origin.

Why real-world accuracy is low

Marketing claims of “99% accuracy” usually refer to clean lab conditions: unedited AI text against unedited human text. Everyday reality looks different.

Editing destroys the signal. Even light rewriting — a few swapped phrases, a rearranged sentence — largely defeats detection. And practically every professionally used AI text gets edited.

Mixed text is the norm. Most real-world text is neither purely machine-generated nor purely human-written, but alternates between the two: an AI draft, a human edit, an AI trim, a human polish. Detectors aren't built for this reality.

Prompting changes the style. Instructing the model to write unevenly, colloquially, or in a specific voice shifts exactly the traits being measured. How much style can be steered is shown in Writing AI Texts.

The models are moving faster than the detectors. Every new model generation writes more like a human — the detection tools are playing catch-up. The fact that even OpenAI shut down its own detection tool after a few months due to insufficient accuracy is a telling signal.

The real problem: false positives

A detector can be wrong in two ways. If it misses an AI text, that's annoying. If it flags a self-written text as machine-made, real harm results: a plagiarism accusation against a student, an applicant getting rejected, a writer having to justify herself — with no way to prove her innocence.

Especially uncomfortable: the errors don't hit at random. Studies have shown that text written by people who aren't writing in their native language is flagged as AI-generated significantly more often — in one widely cited study on English exam essays, this affected over half the texts. The reason is structural: someone writing in a foreign language draws on a smaller, more conventional vocabulary. That's exactly what the detector measures as “machine-made.”

The same pattern hits professionally standardized text — legal briefs, technical documentation, scientific methods sections, product descriptions. They're formulaic because their genre is formulaic, not because a machine wrote them.

That leads to the most important rule in this article: a detector result should never trigger a consequence on its own. It can be a reason to have a conversation — never proof.

What people often use to spot AI text

More telling than any statistic are content-level traits. Typical of unreviewed AI text:

• Uniform rhythm — every paragraph roughly the same length, every list has exactly three items.
• Filler phrases without substance — “In today's fast-paced world,” “It's important to note,” “In summary.”
• Claims without concrete evidence — lots of adjectives, few verifiable facts, numbers with no source.
• Missing lived experience — no personal examples, no names, no situations that only someone who was there would know.
• A smoothed-over perspective — every statement immediately hedged, no clear position.

These traits are more revealing than a percentage — but they actually describe something else: poor, unreviewed writing. A carefully crafted AI-assisted text won't show them; a hastily typed human text certainly can. Using them as a detection marker actually just measures quality.

Watermarks and provenance records

The more promising direction goes the other way: don't guess after the fact — mark at generation time.

Regulation also plays a role here: the EU AI Act requires providers to label synthetic content in a machine-readable way and requires that people can tell when they're talking to an AI or looking at AI-generated material. The underlying structure is explained in The EU AI Act Explained Simply. The practical effect stays limited as long as only some models participate and markings can be removed — but the direction is right nonetheless: proving origin is more reliable than guessing it.

Practical recommendation

For businesses: The question “Is this AI?” is usually the wrong one to ask. What matters is whether a text is correct, sourced, and accounted for. So govern the creation process instead of buying detectors: a written AI policy, clear external labeling rules, and a quality check that verifies facts. How to build such a policy is shown in An AI Policy for Your Company; the matching review process is described in AI Hallucinations.

For education and exams: Don't rely on detector scores. Process evidence is more solid — version histories, intermediate drafts, oral follow-up questions about a person's own text — as are assignment formats that require personal experience, current data, or personal reflection.

For editorial work and hiring: Use a detector result, at most, as a reason to look more closely. Then check the substance: are the facts correct? Are there concrete, verifiable details? Can the person discuss the content in conversation? That's a more reliable way to find out what you actually want to know.

Conclusion

AI detectors measure linguistic patterns and output probabilities — nothing more. Light editing, mixed text, targeted prompting, and the rapid progress of the models further undermine their reliability, while false positives systematically hit the wrong people. Watermarks and provenance records are the better path, but they need time and broad adoption. For everyday use, that means: shift the check from origin to quality. A correct, sourced, accountable text is valuable regardless of what produced it — and a wrong one stays wrong regardless of who typed it.