When you can’t see something clearly, you can understand the presence and scale of that thing by the reaction it inspires. When it comes to AI content in our information ecosystem, it’s now clear the effect is significant and — surprise — not always good. It turns out when you dial the cost of content down to zero, bad things can happen, and a couple of recent stories underscore the need to take countermeasures.
OpenAI recently said it took action against an Iranian “influence operation” called Storm-2035. There were apparently several ChatGPT accounts creating all kinds of fake stories, which would then be distributed on the web and social media. The group posted about politics, the war in Gaza, and other hot-button issues, though it didn’t exactly go viral — few real people engaged with the content.
A few days later, the FTC announced it would ban “fake” reviews of products created to mislead people. As anyone who has spent more than 5 seconds on Amazon knows, many product reviews aren’t real — they’re created only to portray a product in a positive light. The FTC’s statement highlighted AI-generated reviews in particular, which can be more elaborate than your typical astroturf spam (“great product, would recommend, 5 stars”).
While both developments are welcome news to those who care about the health of our discourse, they don’t do justice to the enormity of the icebergs floating beneath the surface of each.
The fact that OpenAI even could cancel the Storm-2035 accounts betrays how much of a problem AI-powered disinformation could be — and likely is. Using a regular $20-a-month ChatGPT account to create fake stories, then copying and pasting them to a website or social account is amateur hour to the extreme. The group could have just used the API (which OpenAI claims is private), but even if that is monitored in some way, it’s trivial to simply use an open-source model instead. From there, you can plug it into a content platform simply and crank out as much content as you like (provided you pay for the compute costs).
In other words, anyone with a little knowledge and money can create AI-generated content — fake or otherwise — at scale. You certainly don’t need the resources of a nation-state; a few determined individuals could do a lot of damage, and there wouldn’t be any accounts to delete.
Similarly: While no one would argue with the FTC’s new rule, it’s easy to see that it’s nearly impossible to enforce. Proving that the author of a particular review did or didn’t actually use the product is hard enough. With AI in the mix, you’d first need to prove that there’s any author in the first place. And given the scale of online reviews, where to start? It would help if the platforms hosting the content put in safeguards, but the more barriers you put in, the fewer reviews you’ll get. It’ll never be in the interest of Amazon or other sites to do serious vetting.
AI Detection to the Rescue?
These separate-but-similar disinformation problems cry out for a systemic solution, and AI detection — scanning copy for signs that it was written by a bot — could be it. Because AI writes sentences based on statistical probabilities, not actual thinking, it’s relatively easy to detect.
There are a lot of problems with using AI detection as the ultimate filter, however. For starters, AI detectors aren’t that reliable. While the good ones are customizable, they generally tend to flag too many false positives — AI detectors have been known to flag human copy that was simply given a once-over by Grammarly.
That leads to the gray area of legit uses of AI in text, of which there are many. When used in the right process, an article can be both 100% AI-written and 100% accurate and truthful. Remember that Google ad from the Olympics? You can debate whether or not it’s appropriate to write a fan letter with AI, but there’s no question the big AI companies are encouraging us to use it in that way — and a whole lot more. AI copy ≠ bad copy.
Finally, with sophisticated prompting, it is possible to coax an AI into creating text that can pass as something human-created. As foundational models get better with each generation, human-like writing seems destined to become easier, if not simply a feature of using the most advanced AI.
In short, AI detection, although a helpful indicator, is far from the silver bullet it first appears to be, and is probably going to be a less useful tool over time.
If there’s a silver lining to the rise of AI-powered disinformation, it’s this: the Iranian campaign did not get any “meaningful audience engagement,” per OpenAI. Perhaps that’s just because the operation was in its infancy, but I like to think it’s because our own bullshit detectors are more fine-tuned than ever. The rise of disinformation over the past eight years has led to an equal if not greater rise in the awareness of it. You can’t be online anymore and not be always asking the question, “Is this even real?”
That’s not to say we don’t need systemic solutions working against bad actors looking to fool us with bad information — we do. But we also need to accept that those filters will never be perfect, and that AI adds a new level of sophistication to the stuff that gets through. We’ve democratized the power of AI, and there’s no going back on the price we must now pay: eternal vigilance.
Leave a Reply