Cross-Perplexity

docker run -p 8000:8000 -v sloptotal-models:/app/models ghcr.io/pablocaeg/sloptotal

curl -s -X POST http://localhost:8000/api/analyze \
  -H "Content-Type: application/json" \
  -d '{"text": "Paste at least 50 characters of text here..."}' \
  | jq '.engine_results[] | select(.engine_name == "Cross-Perplexity")'

What Cross-Perplexity measured

On SlopBench (1,626 human texts written before ChatGPT and 1,626 AI texts from 14 current models, across 15 kinds of writing), Cross-Perplexity reached an AUC of 0.618. It scored machine text 0.680 on average and modern human text 0.580. AUC is the probability that it ranks a random AI passage above a random human one, so 1.0 is perfect separation and 0.5 is a coin flip.

One slice tests the failure that matters most: 87 passages published between 1532 and 1915 (Machiavelli, Austen, Melville, Kafka), where any high score is an error by construction, because the writing predates language models by a century or more. Cross-Perplexity scored those at 0.437. That is 0.143 below its score for modern human writing, so archaic prose does not trip it.

Its part in the score

Cross-Perplexity is one row of the 23-row engine table, next to neural classifiers, statistical tests built on GPT-2 models and checks of the prose itself. No single engine sets the verdict.

The overall score comes from a logistic model fitted on SlopBench, which weighs each engine by what it adds once the others are known. Detectors that fail together, such as classifiers trained on similar data, do not count twice.

How the 23 engines reach a score

SlopTotal does not ask one model for an opinion. It runs 23 independent detectors and shows all 23 scores, because the useful information is often in the disagreement. Nine are neural classifiers trained to separate human from machine text, among them DeBERTa-v3-large, RoBERTa-large and a 4.4M-parameter BERT-tiny. Seven are statistical tests that need no detector training: they measure how predictable the text is to GPT-2 models, with perplexity, token ranks, log-rank and the cross-entropy between two models. The other seven read the prose itself: sentence-length variance, vocabulary, structure, hedging and the stock phrases AI models overuse.

The overall score is not an average. A logistic model fitted on SlopBench weighs each engine by what it adds once the others are known, so engines that fail in the same way do not count twice. Very short texts are pulled toward the middle, because a few words cannot carry much evidence.

The verdict bands are set on human writing: 45 is where the top 5% of human texts begin, 55 the top 2% and 80 the top 0.5%. So a score of 55 or more means fewer than 2 in 100 human texts score that high. Other languages have their own fit, and each language is marked supported, experimental or not yet reliable.

Measured on 3,252 texts

An accuracy figure means little without saying what it was measured on. SlopTotal is measured on SlopBench, an open benchmark of 1,626 human texts, every one written before ChatGPT existed, and 1,626 AI texts on the same topics and at the same lengths from 14 current models (GPT-6, DeepSeek, Qwen, Llama, Mistral, Gemma, GLM among them). It spans news, encyclopedias, science abstracts, Q&A answers, Reddit posts, reviews, student essays, non-native English and fiction, because a detector tested on one kind of writing ends up with numbers that collapse in the wild.

Two slices are there to catch unfairness: 87 passages of literature published between 1532 and 1915, where any high score is an error by construction, and essays by non-native English writers, the people detectors most often accuse wrongly. Every headline number is measured on kinds of writing the model was not tuned on, and the results are published in the repository, failures included.

Where detection stops working

Short text is unreliable, in every detector. Under about 80 words the score swings on word choice alone, and it settles from about 200 words. If a tweet gets a confident answer, distrust the answer.

Lightly edited AI text is the hard case. Each pass of human rewriting removes machine fingerprints, and there is no point at which editing stops mattering. Source code is out of scope: the engines are trained on natural language and do not reliably detect AI-written code, so SlopTotal does not claim to.

A score is evidence, not a verdict. Use the 23 scores to decide where to look and what to ask, then ask. No detector output should decide a grade, a hire or a publication on its own.