Russian AI detector

How it was measured in Russian

The human side is 40 Russian Wikipedia articles as they stood in 2018, before language models wrote any meaningful share of Wikipedia, plus 1 piece of Russian literature published before 1920. The AI side is text from DeepSeek, GPT-6 Luna, Mistral Small, Llama 4 Maverick and Qwen 3.8 Flash, asked to write about the same topics in Russian.

Every number on this page comes from five-fold cross-validation within Russian: the model that scored a text never saw it during fitting. As a stricter test, the weights were fitted on DeepSeek alone and scored on the other four model families, and they still flagged 100% of that text. The weights learned what machine-written Russian looks like rather than one model's habits.

What AI Russian overuses

Comparing 40 AI-written encyclopedia passages with 40 Russian Wikipedia articles on the same topics, some words stand out: “включая” in 33% of AI texts and 0% of the Wikipedia articles; “одним из” in 33% of AI texts and 5% of the Wikipedia articles; “остаётся” in 26% of AI texts and 0% of the Wikipedia articles; “в том числе” in 26% of AI texts and 5% of the Wikipedia articles; “известность” in 18% of AI texts and 0% of the Wikipedia articles.

These are tendencies of the models, not proof. Plenty of people use every one of these words, and SlopTotal does not count them: the score comes from the engines. The list is here because it is what a careful reader notices first.

Short texts

Short Russian text is held back. Below 150 words the score is pulled down in proportion to how little text there is. The size of that correction was fitted on French, Spanish, German, Italian and Portuguese texts cut to 50 and 100 words, where it brought human text flagged above 45 from 20% down to 5%; it has not been measured separately on Russian, so treat a short Russian result as a hint.

What to know about Russian

Ukrainian is recognised as its own language and reported as unsupported rather than scored as Russian, because no Ukrainian text was in the fit. Services such as Antiplagiat check for copied text; SlopTotal checks for machine-written text, and the two answer different questions.

How the 23 engines reach a score

SlopTotal does not ask one model for an opinion. It runs 23 independent detectors and shows all 23 scores, because the useful information is often in the disagreement. Nine are neural classifiers trained to separate human from machine text, among them DeBERTa-v3-large, RoBERTa-large and a 4.4M-parameter BERT-tiny. Seven are statistical tests that need no detector training: they measure how predictable the text is to GPT-2 models, with perplexity, token ranks, log-rank and the cross-entropy between two models. The other seven read the prose itself: sentence-length variance, vocabulary, structure, hedging and the stock phrases AI models overuse.

The overall score is not an average. A logistic model fitted on SlopBench weighs each engine by what it adds once the others are known, so engines that fail in the same way do not count twice. Very short texts are pulled toward the middle, because a few words cannot carry much evidence.

The verdict bands are set on human writing: 45 is where the top 5% of human texts begin, 55 the top 2% and 80 the top 0.5%. So a score of 55 or more means fewer than 2 in 100 human texts score that high. Other languages have their own fit, and each language is marked supported, experimental or not yet reliable.

Measured on 3,252 texts

An accuracy figure means little without saying what it was measured on. SlopTotal is measured on SlopBench, an open benchmark of 1,626 human texts, every one written before ChatGPT existed, and 1,626 AI texts on the same topics and at the same lengths from 14 current models (GPT-6, DeepSeek, Qwen, Llama, Mistral, Gemma, GLM among them). It spans news, encyclopedias, science abstracts, Q&A answers, Reddit posts, reviews, student essays, non-native English and fiction, because a detector tested on one kind of writing ends up with numbers that collapse in the wild.

Two slices are there to catch unfairness: 87 passages of literature published between 1532 and 1915, where any high score is an error by construction, and essays by non-native English writers, the people detectors most often accuse wrongly. Every headline number is measured on kinds of writing the model was not tuned on, and the results are published in the repository, failures included.

Where detection stops working

Short text is unreliable, in every detector. Under about 80 words the score swings on word choice alone, and it settles from about 200 words. If a tweet gets a confident answer, distrust the answer.

Lightly edited AI text is the hard case. Each pass of human rewriting removes machine fingerprints, and there is no point at which editing stops mattering. Source code is out of scope: the engines are trained on natural language and do not reliably detect AI-written code, so SlopTotal does not claim to.

A score is evidence, not a verdict. Use the 23 scores to decide where to look and what to ask, then ask. No detector output should decide a grade, a hire or a publication on its own.