Anatomy of the slop score.

Most AI-writing tools give you a verdict and no way to check it. The slop score shows its work. Every point on its 0 to 100 scale traces back to a phrase you can read or a statistic you can recompute, and the whole thing is open source. This post explains how the formula works, how we tuned it without training a model, what an outside review found wrong with it, and how it held up on three public tests.

The score adds up four kinds of evidence. A pattern meter with 279 weighted tells, a 96-term watchlist, and 26 context-gated terms catches the phrases AI overuses. A rhythm channel measures whether sentence lengths scatter the way human writing does. A followability channel flags comma pileups and 38-word sentences. A formatting channel counts emoji, bold spam, and dash density. The four add up to one evidence total, exactly as the formula on the landing page shows.

Then a curve turns evidence into a score. We use a logistic function, the same S-shaped curve behind everything from population models to exam grading. We checked the math in plain Python, with no AI involved. At zero evidence the score is 9.5. It never goes lower, so no text scores a perfect zero. At 9 evidence points the score crosses 50. And the curve is steepest exactly there, moving 6.25 score points per evidence point, so the meter is most sensitive in the ambiguous middle and gentle at the clean end.

One number from that verification surprised us. Known human writing tops out at a score of 21, and readers start noticing slop at 25. In evidence terms those two lines are just 0.9 points apart, less than a single mid-weight tell. The border between “reads human” and “something is off” is one bad phrase wide. That matches how reading actually feels.

No training run picked the weights. We tuned them by hand against two piles of text: public AI output that should score high, and a dozen documents that must never be flagged, including the Gettysburg Address, a Federalist paper, an engineer’s email written in second-language English, and a dense grant abstract. Lincoln runs 2.1 em dashes per hundred words and still scores 20.2, clear, because punctuation alone is never enough to fail a text. Any proposed pattern that flags one of those twelve documents is rejected, whatever else it catches.

In August we asked an outside reviewer to check the scorer’s math and try to break it. The review found three real bugs. Two overlapping rules could match the same phrase, so one bad line was counted twice. A single stray arrow character could push a clean memo from a score of 20.2 to 86.7. And the scoring floor ignored document length, so a long report could never come out clean no matter how well it was written. We fixed all three and reran the numbers. The same arrow now moves that memo to 26.6. The double-counted phrase counts once. A 2,000-word report with a few mild issues scores 31.4 instead of failing automatically. The bugs and the fixes are in the open repo, so anyone can check our work.

Does the score match human judgment? We ran three tests on public data, and the meter was never tuned on any of it. The first test checks that the score falls as humans get more involved. The public Beemo corpus pairs 2,187 model answers with expert-edited versions and fully human answers to the same prompts. Raw model output averages 30.2. The expert-edited versions average 25.3. The human answers average 20.0. Each step of human work lowers the score.

The second test checks that better models earn lower scores. We scored 7,627 documents from a benchmark built on the public RAID dataset. The share of writing that reads as slop falls as model quality rises: 41.7% for Llama 3.3, 30.4% for Gemma, 18.2% for Gemini 3.1 Pro, and 10.1% for DeepSeek V3.

The third test checks the meter against people. Two editors labeled 72 passages as slop or clean without seeing any scores. They agreed with each other 77.8% of the time, and the meter matched their shared verdicts about as often, with an F1 of 0.8.

The last test is the practical one. We took 18 AI drafts, ran them through five open-source editing tools, and scored the results.

The score has limits worth stating. It reads the surface of the text, so it cannot tell whether a claim is true or a paragraph is empty. It does not prove who wrote something: a person can write slop, and a draft written with AI help can read clean. What it gives you is a number you can check yourself and watch fall as you edit. In the table above, that is the difference between 76.3 and 15.4.

Run it on your own writing and see where you land.