perplexity-based detection detector

What we look for in perplexity-based detection

Classifiers trained on any LLM output, plus perplexity and phrase heuristics that catch templated structure.

Short snippets skew scores. Aim for 150+ words when possible — engines calibrate better with context.

Reading the engine grid

Green across the board: likely human or heavily reworked. Red cluster: investigate before you publish or grade.

Split vote: classic sign of AI-assisted editing. That is the case where a single-score tool misleads.

Context

Perplexity asks how surprised a language model is by your next word. Machine text tends to be more predictable, so it scores lower. The intuition that trips people up is that this should punish famous books, since a model has effectively memorised them — measured, it does not: the perplexity engines score pre-1920 literary prose *lower* than modern human writing, and they are among the most dependable detectors in the grid.

Editorial note

SlopTotal does not store analytics cookies. Reports delete on day 30.

Practical tip: March 2026 benchmarks on our grid show short tweets (<80 words) swing wild. Blog excerpts stabilise after ~200 words.

How the 23 engines reach a verdict

SlopTotal does not ask one model for an opinion. It runs 23 independent detectors and shows you all 23 rows, because the interesting information is usually in the disagreement. Nine are neural classifiers fine-tuned to separate human from machine text, among them DeBERTa-v3-large, three RoBERTa variants and a 4.4M-parameter BERT-tiny that answers in milliseconds. Seven are statistical tests that need no detector training at all: they measure how predictable your text looks to GPT-2 Medium, using perplexity, token rank distribution, log-rank and cross-entropy ratios between two different language models. The remaining seven read the prose itself, counting contractions, em-dash habits, sentence-length variance, hedging and stock openings.

The final score is not an average. Averaging lets a single confident engine drag the result, and detectors fail in correlated ways: the neural classifiers were largely trained on similar data, so when they are wrong they tend to be wrong together. Scoring instead anchors on the classifiers that measured both accurate and unbiased in our own evaluation, then blends that against the full weighted set, and treats agreement between independent engine families as the real signal of confidence.

Every engine weight comes from measurement rather than intuition. Each detector's contribution is proportional to how well it actually separated known-human from known-AI text in testing, then reduced if it showed bias against any particular kind of writing. Detectors trained on the same benchmark we test against are damped, because a model graded on its own training distribution always flatters itself.

Tested against two corpora, not one

Accuracy claims mean little without saying what was tested. SlopTotal is evaluated against machine text from six different model families — GPT-4, ChatGPT, Llama, Mistral, Cohere and GPT-3 — across four kinds of writing: news reporting, book prose, poetry and academic abstracts. Measuring one domain is how detectors end up with impressive numbers that collapse in the wild; we found one of our own engines scoring near-perfectly on abstracts while being almost exactly backwards on everything else.

The second corpus is the one most detectors skip. It is prose published between 1532 and 1915 — Machiavelli, Austen, Melville, Kafka — where a high score cannot be anything but an error, because the text predates language models by a century or more. Any detector that quietly punishes older or more formal writing gets caught by that set immediately, and several of ours did. The results are published in the repository, including the failures.

Where detection stops working

Short text is unreliable and no detector honestly says otherwise. Under about 80 words the score swings hard on word choice alone; results settle at roughly 200 words and above. If you paste a tweet and get a confident answer, distrust the answer rather than the tweet.

Lightly edited AI is the hard case. Text that a person has rewritten sentence by sentence carries fewer machine fingerprints with every pass, and there is no threshold at which editing stops mattering. Source code is a genuine gap: these engines are trained on natural language, and in our own testing they neither falsely accuse human code nor reliably catch machine-written code, so we do not claim they can.

A score is evidence, not a verdict. Use the 23 rows to decide where to look and what to ask, then ask. No detector output should by itself decide a grade, a hire, or a publication — and any tool that encourages you to treat it that way is selling certainty it does not have.