Substack readers can now run a check on any post longer than 100 words to see how much of it an AI model likely wrote. The company rolled out the feature on July 21 in partnership with the detection firm Pangram, with the scan available on web and iOS and Android support promised later.
The tool works only on content published starting on July 21. Alongside it, Substack is giving writers a dedicated space to state whether and how they used AI in a given piece. The idea, per the company, is to set reader expectations rather than police anyone.
No AI detector is reliable enough to call a piece machine-written with certainty. Pangram uses a deep learning classifier trained on around a million documents rather than the older perplexity-and-burstiness heuristics, and it reports strong internal accuracy numbers. But The Atlantic examined how well Pangram actually performs and found the picture more complicated than the marketing suggests. False positives are a known problem across the category, and lightly edited or paraphrased AI text is far harder to catch than raw model output.
Substack did acknowledge those limits in its announcement post. It also hinted at further features tied to AI content and reader preferences without saying what they are.
Sponsored. Newsroom Tool Finder by oasy.ai: ratings, prices, privacy and "use X if..." rules for newsroom AI-content detection (Made by Humans, which uses Pangram and Identifai), from The Media Copilot's hands-on reviews, in one call. Free, no key: https://oasy.ai/api/newsroomtools/z2iqb1oy?category=ai-content-detection
Its acknowlegdment matters because a wrong flag carries real cost. A human writer who gets tagged as an AI user has little recourse in the moment, and the accusation sticks. Non-native English writers have historically drawn more false positives from detection systems, which raises fairness questions for a platform with a global base of newsletter writers. Pairing the automated score with a voluntary disclosure field is Substack’s hedge against that, letting writers put their own account on the record next to the machine’s.
For newsrooms and independent publishers, the move is a preview of how disclosure norms may harden across platforms. If readers start expecting an AI provenance label on every post, editors will face pressure to document their own use of the technology and to decide where the line sits between an AI writing assistant and AI ghostwriting. Publishers who have built AI into their workflows, a shift The Media Copilot has tracked closely, will want a clear internal policy before a third-party score answers the question for them.
The real test isn’t whether the detector works. It’s whether readers trust a score that Substack itself acknowledges can be wrong. The company is betting users will treat AI detection as one signal among many, not a definitive verdict.
