Nic Newman stood in front of a slide that read “AI can do what journalists do quicker and better.” It was a deliberately provocative framing, and the Reuters Institute senior research associate used it to make a sharper point about where the threat to publishers is actually heading.

Speaking at the Media Voices Publisher Summit in London on 10 July, Newman warned that AI models’ ability to serve up what he calls “liquid content” in whatever format a reader wants is a really serious challenge for news media, as reported by Press Gazette.

He demonstrated with a live example. Asking Anthropic’s Claude about the recent U.K. heatwaves, Newman got answers in a Q&A format, filterable graphs comparing air conditioning uptake across countries and Met Office warnings pulled together on the fly. The same information base used by news outlets, adapted to a user’s specific needs and context.

“Content is becoming much more fluid, whether we like it or not,” Newman said. “It’s going to be remixed in many different ways, and audiences will expect and like that level of personalisation.”

The worry is not just today’s chatbots. Newman pointed to agentic tools that complete tasks without being asked, such as ChatGPT Pulse, which digests a user’s chat history and connected apps like Google Calendar to deliver a morning brief unprompted. He expects AI to increasingly turn publisher newsletters into convenient audio digests too.

“People not having to put words into a search box, but the AI agents knowing what you’re interested in and bringing it to you automatically” is the shift publishers should fear most, he said.

Newman’s 2026 trends and predictions report, based on responses from 264 news leaders, found publishers see novel content as their way forward. Original investigations and reporting from the ground ranked highest, followed by contextual analysis and community-building through events. General news for everyone, Newman argued, is exactly what AI will commoditise.

He also urged publishers to fold AI into their own products rather than cede the ground entirely. On-site chatbots, like Ask The News, a product The Washington Post has been developing, can pair human curation with AI’s ability to answer specific reader questions.

The second disruption he flagged is the rise of personality-led and creator-style journalism. Citing Financial Times analysis by John Burn-Murdoch, Newman noted social platforms have become less social and more about following individuals. Joe Rogan now reaches roughly a fifth of American adults weekly. Audiences describe creator-led media as more trustworthy and relatable, even as they rate it less impartial overall.

For newsrooms, the takeaway is uncomfortable but clear. The funnel model, built around using newsletters, podcasts and social traffic to bring users back to a website, remains relevant but faces long-term pressure as audiences consume news differently. Younger audiences are moving elsewhere: 52% of 18-to-24-year-olds now name social, video and AI platforms as their main source of news, up from 40% five years ago, according to the latest Digital News Report covered on The Media Copilot.

Newman sees the future in show- and talent-led models built around personalities and niche audiences. Goalhanger, a U.K. podcast producer known for series including The Rest Is History and The Rest Is Politics, is adding written content to their model, while The Guardian’s Guardian Studios is expanding the publisher’s work across branded storytelling and commercial partnerships. The growth opportunity, Newman said, is building habit and trust around personalities and brands, then finding ways to turn that relationship into a lasting business.

The post Newman warns AI’s ‘liquid content’ remixing poses serious challenge to news media appeared first on The Media Copilot.

Freelance writers for Take a Break’s Fiction Feast, Bauer Media’s monthly compendium of short stories, have been told their services are no longer required. In their place: stories that appear to be drafted by AI and credited to “The Fiction Feast Team.”

The change was reported by Press Gazette, which says that whole stories now appear to be produced with AI tools rather than commissioned from human authors. Bauer Media has not responded to requests for comment.

Press Gazette’s Dominic Ponsford framed the likely motive as financial. Costs are tight, and cutting freelance commissions may be a way to keep the title running while keeping the remaining named human authors in work. Ponsford presents this as a possibility, not an established fact, and acknowledges he cannot verify the underlying reason.

What makes this case notable is the setting in which the AI is being deployed. Fiction is among the creative forms readers are least likely to embrace when they learn it was generated by AI. That makes a magazine built around human-authored short stories an unusual place to test audience acceptance of synthetic writing, particularly when publishing under a house byline rather than clearly disclosing AI involvement.

The move fits a broader squeeze on paid creative work. The Author’s Guild has repeatedly called for AI-generated works to to be clearly labeled to prevent them from being passed off as human-written and to protect the market for human authors. Publishers introducing AI into creative publications are stepping into that dispute.

The decision comes amid a worsening climate for freelance writers and creative workers. Newsroom and publishing job cuts have accelerated, with the 2026 layoff wave already outpacing the previous year by early spring. Faced with those pressures, publishers may see AI-generated drafts as a way to cut down on freelance spending.

Traffic data in the same Press Gazette briefing shows why money is tight. Most of the top ten U.S. news websites lost more than 20% of their traffic year on year in June, with Substack the lone gainer at 25%. Buzzfeed fell 48% to 46 million monthly visits. Globally, 45 of the biggest news sites saw declines in April 2026 alone. Google’s expansion of AI Overviews and AI Mode in its home market is a major driver of those drops.

For newsrooms and publishers, the Take a Break decision offers an early look at a challenge more titles may soon confront. When search referrals fall and budgets shrink, the temptation to replace paid contributors with AI grows, and fiction is no longer off-limits. The open questions are whether readers notice, whether they care, and whether a house byline like “The Fiction Feast Team” counts as adequate disclosure.

Bauer has not addressed those questions publicly. The outcome could shape how other publishers think about AI in creative work: if readers see little distinction between commissioned fiction and AI-generated stories, publishers may feel more comfortable expanding such experiments. If readers object, the episode could underscore the reputational risks of introducing AI into publications built on human authorship.

The post Bauer’s Take a Break drops freelance writers as AI drafts fiction stories appeared first on The Media Copilot.

The Associated Press has joined SPUR as the coalition’s first U.S. founding member, adding one of the world’s largest news licensing organizations to a publisher-led effort to create standards for how AI companies track, value and compensate journalism.

Founded in March 2026, the Standards for Publisher Usage Rights is a publisher-led coalition aiming to move AI content use away from opaque scraping and toward a usage-based licensing model where publishers can see how their work is accessed and used. Its founding members include the BBC, the Financial Times, The Guardian, Sky, The Times of London and European group MediaHaus. The AP now joins 30 publisher members and six affiliates.

SPUR’s central argument is that publishers need more than the ability to block AI crawlers. They need visibility into what happens after AI systems access their content.

SPUR’s technical foundation is a content telemetry standard announced June 12 and open for public comment through July 24. The framework breaks AI content use into five measurable events: content retrieved, grounded, cited, displayed and engaged. It creates a common format for reporting those interactions back to publishers.

The standard also defines the underlying data schema, allowing publishers, platforms and vendors to integrate with the same system.

SPUR has begun testing the framework beyond its membership. Microsoft and CDN provider Fastly participated in a recent London public comment event, while licensing and infrastructure startups including TollBit, Redpine and MonetizationOS have said they plan to implement the standard.

The effort differs from earlier publisher initiatives because it focuses on measuring usage after content enters AI systems. The IAB Tech Lab‘s Content Monetization Protocols, by contrast, focused more heavily on pre-crawl access controls and bot management.

But adoption remains the biggest challenge. SPUR can define how AI usage should be measured, but it cannot force AI companies to provide that information. No single publisher has enough leverage to compel companies such as OpenAI or Google to adopt publisher-friendly standards.

SPUR’s strategy is collective action. If enough publishers adopt the same framework, they may create enough pressure for AI companies to participate. That collective-action logic echoes other recent moves, from Reuters and Time shifting to bot-blocking whitelists to broader efforts to build a global publisher alliance.

“The key here lies in both parts of this being a collective action,” Scott Messer of Messer Media told Digiday in an email. “A divided set of publishers cannot battle the forces of LLMs.”

The approach reflects a broader shift in the publisher-AI debate. Instead of focusing only on payment, SPUR members are trying to establish permission and transparency as the foundation for future licensing.

Publisher alliances, however, have a complicated history. During the rise of programmatic advertising, shared industry systems often created value for platforms while leaving publishers with limited control.

Alessandro De Zanche, a former News U.K. executive and founder of media strategy consultancy ADZ Strategies, argues SPUR differs because publishers are approaching AI through the lens of content ownership rather than advertising inventory.

“The teams that drove the advertising channel into a wall are not the ones now dealing with content, IP and LLMs,” De Zanche said.

With AI, he said, publishers are not selling volume. They are selling accuracy, provenance and reliability, and the stakes are “completely different.”

The post AP joins SPUR as publishers build a telemetry standard to track AI content use appeared first on The Media Copilot.

In 2023, a company called Alabama Community News LLC supposedly spent $3.2 million to buy 47 weekly newspapers across the state. The corporate owners fired the local staff, replaced them with an artificial intelligence system that scraped high school sports scores, and promptly drove the entire network into bankruptcy. The story even named a specific 26-year-old campaign staffer who generated 70 percent of the copy.

None of it actually happened. The entire 1,900-word saga was a fabrication published by a site called The Editorial, according to an investigation by Nieman Lab. The targeted newspapers, including the Shelby County Reporter and the Centreville Press, are still printing. The angry local advertisers quoted in the piece do not exist. The story falsely claimed the roll-up was funded by 1819 News, a real conservative outlet in the state, adding a layer of plausibility to the hoax.

The fake story gained traction among journalists on social media platforms like Bluesky before the operators pulled it down. They replaced the page with a sterile retraction notice citing “fact-verification concerns.” But the Alabama hoax was not an isolated incident.

The Editorial has built a bizarre subgenre of AI-generated obituaries for real American newspapers. The site previously published fabricated stories detailing the collapse of the Chattanooga Times Free Press, the Kenosha News, and the Macon Telegraph. The nonexistent reporters credited with these stories sport fake resumes claiming past stints at ProPublica and Reuters.

The motive behind the site remains murky. Domain registration and payment records point to a Finnish technology company called Nordiso Group, which develops AI study apps. Yet the site’s political sections suggest a different angle. The Editorial publishes a high volume of geopolitical content focused on Taiwan and the South China Sea, heavily pushing narratives that highlight Chinese military dominance.

These geopolitical stories share obvious synthetic fingerprints. Nearly every piece opens with a variation of the exact same scene: a nondescript, windowless conference room where a secret document slides across a table. This repetitive structure aligns with tactics tracked by groups like the Stanford Internet Observatory, which monitors state-sponsored disinformation campaigns. It also highlights how cheap synthetic media allows operators to flood niche topics, a trend we track closely at The Media Copilot.

For publishers, this represents a strange new vector of reputational risk. Newsrooms are used to fighting disinformation about elections or public health, but now they must monitor for synthetic hoaxes about their own business operations. A fake story about a newspaper shutting down or firing its staff can spook actual advertisers and confuse real subscribers before the publisher even realizes the rumor exists.

The barrier to generating convincing local news copy is gone. Operators no longer need to understand the nuances of a community to write a plausible story about it. They only need a prompt and a target, leaving local editors to clean up the mess when the synthetic fallout hits their own backyards.

The post AI fake news network invents the collapse of 47 local Alabama newspapers appeared first on The Media Copilot.

The Centre Daily Times in State College, PA, has voted to unionize after months of pushback against its parent company’s AI tool—a move that, according to The NewsGuild-CWA, makes it the first newsroom in the union to cite AI adoption concerns as a primary reason for organizing.

As Nieman Lab reported, the Centre Daily Times staff voted to join The NewsGuild of Greater Philadelphia last month. All eligible editorial staff signed authorization cards, and McClatchy voluntarily recognized the union. The catalyst, reporters told Nieman Lab, was McClatchy’s Content Scaling Agent (CSA) tool—an AI system that repackages existing articles into short-form summaries for publication or video scripts—and a March internal meeting where Kathy Vetter, McClatchy’s chief of staff for local news, told staff the company would use their bylines on AI-generated content unless union contracts prohibited it.

Josh Moyer, a senior reporter at the Centre Daily Times, took that as a signal. “It was essentially like, if you’re not in a union, your byline gets used; if you are in a union, we’ll follow what the union says,” Moyer told Nieman Lab. “If we want to control what happens to our byline, that’s the company telling us that we need to form a union.”

McClatchy introduced the CSA tool at the paper earlier this year. Reporters initially published at least one CSA-assisted story per week under a generic byline noting AI assistance. But in late February, management changed the policy: AI-generated content would now carry the reporter’s actual name. Reporters objected that it misrepresented their work to readers.

“When our names go on a thing, it says that this article or video is from that person, but that is just not true in this case,” said Trebor Maitin, a service reporter. Maitin was the first reporter at the paper to have his byline changed to reflect AI assistance.

The NewsGuild-CWA’s president, Jon Schleuss, said unionized newsrooms have had more success keeping AI content clearly labeled: “Unionized newsrooms are the ones where McClatchy’s AI slop gets a clear label. In non-union newsrooms, the AI slop may be carrying a real human reporter’s byline.”

Multiple McClatchy publications have seen byline strikes over the CSA tool, and some have taken labor actions over the tool and related workplace issues. For the Centre Daily Times, the union opens the door to formal collective bargaining and the ability to join coordinated actions at sister publications.

“Some of us use AI a lot more and are okay with it,” Maitin said. “But there is an overall understanding that we need to be able to have a say in this, and that unionizing at least gives us a seat at the table.”

The post A newspaper unionized because McClatchy put reporters’ names on AI content appeared first on The Media Copilot.

YouTube is moving from voluntary disclosure to automatic detection when it comes to AI-generated content.

The platform announced Wednesday that it will begin automatically applying AI-generated content labels to videos that feature “significant photorealistic AI use”—even if the creator never disclosed it. As noted by Variety, the change marks a notable escalation of YouTube’s approach to AI transparency, which previously relied entirely on creators self-reporting their use of generative AI tools.

Previously, YouTube labeled AI-generated content only when creators voluntarily disclosed it in their video settings. Now the company is rolling out an internal detection system that will flag videos even without creator admission. Creators can dispute incorrect labels through YouTube Studio, but YouTube says the labels will “remain permanent” in certain cases, including content created using YouTube’s own AI tools like Veo or Dream Screen, and any video carrying C2PA metadata (standards from the Coalition for Content Provenance and Authenticity) indicating full AI generation.

The labels are also getting a more prominent placement. Previously buried in expanded descriptions, AI labels on long-form videos will now appear directly below the video player, above the description. For YouTube Shorts, the label will appear as an overlay directly on the video itself.

“The goal here is context at a glance,” said Rene Ritchie, YouTube head of editorial and creator liaison, in a video explaining the changes. “If it looks real but was made with AI, viewers will know immediately.” Ritchie emphasized that the labels do not affect monetization or recommendation algorithms — “This is purely about giving viewers the right information at the right time.”

This push for better AI disclosure follows a broader problem: why AI content labels keep failing the people who need them most. Content Credentials, the metadata-based standard designed to track an image’s AI origins, has existed for years, but social platforms have been inconsistent about adopting it, often stripping out the very metadata that makes the system work. YouTube’s move toward automatic detection is an attempt to close that gap, even if the underlying standards remain patchily implemented.

The move comes alongside YouTube’s expanded likeness-detection program, now available to all creators 18 and older, which helps users identify and request removal of AI-altered facial likeness content.

The post YouTube Will Auto-Label AI-Generated Videos, and Make Those Labels Harder to Miss appeared first on The Media Copilot.

A former startup CEO who pleaded guilty to defrauding investors has been linked to a network of AI-generated fake local news outlets — built, by his own account, to manipulate how both people and search engines perceive people like him.

Drew Chapin, who in 2021 pleaded guilty to investor fraud in connection with a failed tech startup, founded The Discoverability Company, a Philadelphia-based online reputation management firm. According to an investigation by The Florida Trib in partnership with KCRW’s Question Everything podcast, that firm is behind the South Florida Standard, a website that presented itself as a local news outlet but was, in fact, entirely AI-generated.

The South Florida Standard featured fake reporters with AI-created headshots and fabricated biographies. Stories were lifted from legitimate news organizations, run through AI, and republished as original content. The site published three stories a day, seven days a week—including Easter Sunday—under bylines that had no professional history or digital footprint outside the site.

The problem of AI-generated content disguised as legitimate journalism is one our coverage has tracked closely, and it’s getting harder to solve. Even as platforms roll out content credentials and provenance tools, Wikipedia has moved to ban AI-generated text entirely from its 7.1 million articles, citing hallucinations and fabricated citations. The South Florida Standard is a case study in why those bans exist.

Frechette’s analysis traced the South Florida Standard to at least two sister sites: the Charleston Sentinel in South Carolina and the San Francisco Download in California. All three were built from the same source code and controlled by the same entity. Across the network, The Florida Trib identified at least nine “reporters” who share names with people accused of or convicted of fraud or conspiracy.

Chapin acknowledged responsibility for the network, describing it as a six-month experiment to build what he called “geographic topical authority,” testing whether AI-generated news sites could rank alongside legitimate outlets in search results. He said he stood up 17 similar sites across the country, producing more than 3,500 URLs drawing more than 44,000 visitors.

“I don’t know whose job it is to make sure that people are represented fairly and wholly online,” Chapin told the outlet.

The experiment, by Chapin’s own account, didn’t work particularly well. Search engines, he said, could tell the difference between The New York Times and a fake outlet. His sites weren’t breaking through.

But the broader damage is harder to quantify. Known in academic research as “pink slime” journalism—named for the cheap meat by-product used as a filler—these fake news sites now outnumber local daily newspapers in the United States, according to data analysis firm NewsGuard. As of June 2024, NewsGuard identified 1,265 such outlets nationwide, surpassing the 1,213 daily newspapers still operating.

Florida, with the lowest number of news outlets per capita in the continental US, may be especially vulnerable. Researchers have found the state is already home to dozens of pink slime outlets, part of a national network of more than 1,000 sites backed by conservative think tanks, donors, and political operatives.

“Stuff like this has zero value to the public,” said Kelly McBride, a senior vice president at The Poynter Institute. “And in fact it has a negative impact on the news ecosystem, because it clutters the environment.”

The findings arrive as traditional local news continues its historic collapse. Real newsrooms are already experimenting with AI-generated content, and their own workers are pushing back. Since 2005, the country has lost almost 2,900 newspapers and roughly two-thirds of its newspaper journalists—43,000 positions—according to Northwestern University’s Medill Local News Initiative. As real local news disappears, the vacuum doesn’t stay empty for long.

Edited by Pete Pachal

The post A fraudster built a network of fake AI news sites to manipulate search results appeared first on The Media Copilot.

How can you tell something’s AI-generated? When it comes to writing, there are common tells: the excessive use of em dashes, sentences that are too rhythmically clean, and a general smoothness that feels overly engineered.

It’s hardly a perfect science, though, and most humans’ AI detection skills are based on vibes.

If humans are just relying on instinct, what are AI detectors relying on? Here, Zapier shares everything you need to know about how AI detectors work.

What is an AI detector?

An AI detector is a tool that analyzes content like text, images, or videos, and estimates the likelihood that it was generated by an AI model. Instead of giving a definitive yes-or-no answer, most AI detectors will give you:

• A probability score (for example, “74% likely AI-generated”)
• A confidence rating
• Highlighted passages that appear machine-written, if it’s text

Their goal isn’t to “catch” AI with certainty, but to flag content that statistically resembles AI-generated patterns.

How do AI detectors work?

The specifics of how AI detectors work vary depending on what type of content they’re analyzing. For simplicity, this article will focus on AI text detectors. But other types—like AI image detectors—work similarly.

Large language models (LLMs) generate text by predicting the most likely next word based on probability. It’s more nuanced than that, but that’s the idea. AI detectors reverse-engineer that idea: They look at a finished piece of writing and measure how closely it matches those probability patterns. Here are the main techniques they use.

1. Perplexity

Perplexity (not to be confused with the AI-powered search engine) measures how unpredictable a piece of text is to a language model. The lower the perplexity, the more the wording follows patterns the model expects to see.

AI-generated text often has lower perplexity because it’s built from highly common word sequences. It gravitates toward phrasing that’s safe, common, and structurally sound. Which is kind of the point. AI models are trained to predict the most probable next word, not the most chaotic or idiosyncratic—just the most likely.

Human writing, on the other hand, tends to raise the perplexity score because it’s usually less predictable. Unless you have a ruthless editor who’ll set you straight, humans use words that technically work, even if they’re not the exact right ones. They go off on tangents and litter their work with comma splices because those pauses just feel right to them.

2. Burstiness

Burstiness looks at sentence length distribution and structural variation to identify patterns that appear overly consistent.

Humans rarely write in perfect cadence. They mix short sentences with longer ones, occasionally go on tangents, and vary pacing without thinking about it. Earlier AI models, by contrast, tended to produce writing that felt evenly spaced and neatly balanced. Nothing was outright bad, just … suspiciously consistent.

That “too rhythmic” quality is often what sets off our internal AI radar. AI detectors try to quantify that instinct by measuring variation in sentence length, punctuation, and structure. If the tempo barely changes from start to finish, that uniformity can raise a flag.

3. Classifiers

A classifier is a machine learning system trained to categorize text as likely human- or AI-generated. Unlike perplexity or burstiness, which are individual signals, a classifier looks at many features at once and weighs them together.

Developers train their LLMs on large datasets of labeled human and AI text. Through that training, classifiers learn statistical patterns that tend to separate the two categories. Those patterns can include predictability scores, sentence variation, word frequency distributions, and other structural signals.

When you paste new text into an AI detector, the classifier evaluates how multiple signals interact and then produces a probability score. The final output reflects whether the writing, on average, more closely resembles patterns associated with AI-generated text or human-written text.

4. Stylometric analysis

Stylometric analysis is the study of writing style, including vocabulary richness, repetition, and sentence complexity. Think of it as your linguistic fingerprint.

The idea is that humans tend to develop quirks over time. For example, the author Fredrik Backman typically writes stories with a sort of progressive repetition that’s hard to describe, but is uniquely him. It’s what makes his writing so easily distinguishable.

AI writing, by contrast, often clusters around high-probability patterns, generating phrasing that reflects widely represented patterns rather than highly idiosyncratic ones. That’s also what makes much of AI writing feel technically solid but vaguely same-y.

5. Watermark detection

Watermark detection is a way of identifying AI-generated text by looking for a hidden signature baked into the writing itself.

Not all AI models use watermarking, and there isn’t one standard way to do it. But when watermarking is enabled, the model slightly nudges its word choices in a consistent, trackable way. The shifts are subtle enough that you wouldn’t notice anything while reading, but an AI detector that knows what to look for can spot the pattern.

In theory, that makes AI-generated content easier to trace. In reality, even light editing or paraphrasing can blur or erase the signal. So while watermarking sounds like a clean solution, it’s not foolproof.

How accurate are AI detectors?

AI detectors are probabilistic tools, not lie detectors. A detection score reflects how closely writing matches certain patterns. It doesn’t prove who or what actually wrote the text.

Here’s why accuracy gets complicated.

• False positives happen. Some human writing naturally resembles AI-generated text. If you refuse to give up the em dash and sprinkle them liberally throughout your writing, an AI detector may flag it as machine-written, even if it wasn’t.
• False negatives happen. AI models are improving at an alarming speed and learning to mimic human variability more effectively. Humans, for their part, are learning to refine their AI prompts to inject human signals—for example, telling their AI writing generator to mix up sentence patterns or intentionally include errors. As AI writing and human prompting become more nuanced, detection becomes harder.
• Hybrid content blurs the line. Most writing today isn’t purely human or AI. AI detectors struggle in this gray area because the final text contains both human and machine signals.
• Results vary across tools. Different AI detectors use different training data and different models. The same paragraph can receive dramatically different scores depending on the platform. That inconsistency makes it risky to rely on a single detection result for high-stakes decisions.

The bottom line on AI detectors

We’re no longer living in a binary world of purely human or purely AI-generated writing. A lot of content now sits somewhere in between. A draft may start with AI, a human reshapes it, AI tightens a paragraph, a human adds a lived example—the lines blur. And AI detectors have to make probabilistic guesses in that gray space.

This story was produced by Zapier and reviewed and distributed by Stacker.

The post How do AI detectors work? appeared first on The Media Copilot.

ArXiv, the open preprint repository that has become a cornerstone of scientific publishing, is escalating its fight against low-quality, AI-generated research with a new enforcement policy: one year-long ban for authors who fail to take responsibility for LLM output.

As reported by TechCrunch, the rule, announced by Thomas Dietterich, chair of arXiv’s computer science section, targets submissions that contain “incontrovertible evidence” that authors did not check AI-generated content. That evidence includes hallucinated references—fabricated citations—a problem research shows is on the rise in scientific literature. It also covers direct LLM comments or instructions left inside submitted manuscripts.

“If a submission contains incontrovertible evidence that the authors did not check the results of LLM generation, this means we can’t trust anything in the paper,” Dietterich wrote on X.

The penalty is steep. Offending authors face a one-year suspension from arXiv followed by a requirement that all future submissions first be accepted through a peer-reviewed venue before being posted. Dietterich described it as a “one-strike” rule, though moderators must flag violations and section chairs must confirm the evidence before a ban is imposed. Authors retain the right to appeal.

Crucially, arXiv is not banning LLM use outright. Researchers may use AI tools in their workflow, but they bear full responsibility for the final content. That means if an author copies AI-generated text containing plagiarized material, biased content, fabricated references, or factual errors, they face consequences regardless of the tool that produced it.

“Authors take full responsibility for the content of their submission, irrespective of how the contents are generated,” Dietterich said.

The move comes as arXiv navigates a broader transition. After more than 20 years under Cornell University, the organization is becoming an independent nonprofit, a shift expected to give it more resources to address challenges like AI slop. ArXiv has already required first-time posters to obtain endorsements from established researchers, a barrier aimed at weeding out low-effort submissions.

Fabricated citations have become a flashpoint. Peer-reviewed research published in The Lancet found that fake citations are increasing in biomedical literature, a trend researchers link partly to LLM use. Scientists are not alone—lawyers, firms, and other professionals have also been caught submitting AI-hallucinated citations to courts and other authorities.

The policy puts arXiv ahead of most academic platforms in explicitly addressing AI authorship accountability. Whether it deters misuse or simply shifts it elsewhere remains to be seen.

The post ArXiv will ban researchers for a year if they let AI write their papers appeared first on The Media Copilot.

Fake accounts are as old as social media itself. So when it came to light that a “hot girl” MAGA personality named Emily Hart was actually a 22-year-old male medical student in India, it could have been dismissed as just another internet deception story. Just another catfisher, another sock puppet, another scammer—the internet is full of them.

But this case was different. This one had photos. And videos. And thousands of followers across multiple networks with some posts getting millions of views. Emily Hart was a full-on influencer, not just some anonymous egg. The person who created Emily confessed to Wired that while the account was active, he was making thousands of dollars every month from posting softcore videos to an OnlyFans competitor and merchandising.

Emily’s creator is not a developer. He’s just a cash-strapped student with a good sense of American political culture and a Google Gemini account. Yet the Emily Hart story has done more than expose one fraud. It’s put a spotlight on how thoroughly AI has lowered the barrier for almost anyone to produce convincing content and manipulate social media’s engagement systems.

That reality raises a set of urgent questions. Is anyone looking out for us out there? How can you tell what’s real and what’s not anymore? And who is responsible for alerting social media users that the images they’re looking at might have come from AI?

How cheap AI tools made fake influencers scalable

The real significance of the Emily Hart story has little to do with a single fake account. The major implication is that this is the tip of the iceberg. AI has made creating online personas like Emily so easy that it’s enabled deception at scale. The Wired story points to other pro-Trump fake influencers like Jessica Foster, but you don’t have to look very far in your Instagram Explore page before you spot something AI-generated, and it’s rarely disclosed. The Emily Hart case proves that the template is cheap, fast, lucrative, and easy to copy.

Every major social network has policies that address AI-generated content. While they vary in detail, the gist is generally the same: Synthetic images must be disclosed—especially if it could be construed as real and the subject matter involves sensitive subjects like politics, health, finance, and current news. If the account doesn’t identify AI content, it could be frozen, demonetized, or banned.

In practice, those consequences almost never materialize. Enforcement is difficult, partly because detecting AI content is getting more difficult by the day. Most state-of-the-art image generators are light-years ahead of the models that created the first “Will Smith eating spaghetti” video, and telltale artifacts like extra fingers and disappearing background characters have largely become a thing of the past. Without watermarks, even automated systems have a difficult time parsing AI images from real ones just by looking at them.

Content Credentials and the AI labeling problem

A new standard was supposed to fix this. Content Credentials are a way to track how an image was created and modified throughout its life cycle. That provenance data can live in the image’s metadata, so the site displaying it can more easily tell whether it’s AI-generated, potentially passing on a label or warning to the user. The idea is that, as you scroll your social feed, any image would have a tiny icon next to it that would reveal its history when clicked.

However, even though this technology has existed for years and ostensibly has the support of major tech companies such as Adobe, Google, and Nvidia, social platforms haven’t adopted it consistently. Seeing the label is rare, and a Washington Post report found that social networks often strip out the metadata that enables Content Credentials. The stripping isn’t necessarily a deliberate act of sabotage — it follows a best practice from the early days of the web when every byte was precious. But the fact that it’s still happening shows there is little enthusiasm to make the system work.

Does labeling even change behavior? Emily’s creator says he believes many of his followers didn’t care whether the images he was posting were AI or not. That may be true for some, but data suggest labels can alter people’s propensity to engage with AI content. A 2024 study found that labels on AI-manipulated media reduced belief in the claims. The study also found that wording matters: “manipulated” or “false” were more impactful than process-based labels alone.

Put another way: labels work, but toothless labels work poorly. A buried “AI info” tag is not the same as a clear warning that an image might depict a person who does not exist.

The technical capacity to do better clearly exists. Platforms like Facebook, Instagram, YouTube, and TikTok already process and modify content at scale. They’ve spent two decades building the art of detecting copyright violations, nudity, spam, and engagement signals. It is hard to believe they are incapable of building a clearer label for AI-generated people.

Why platforms have reason to keep AI labels weak

The question then becomes: why haven’t they? The uncomfortable answer is that the incentives point the other way. While platforms want to keep bad content out, they are more motivated to keep people posting, scrolling, sharing, and buying. AI-generated material fits neatly into that machine because it is cheap to make, easy to personalize and highly compatible with engagement-driven feeds.

Mark Zuckerberg has been unusually direct about this, describing AI-generated material as “a whole new category of content” that he sees as important for Facebook, Instagram and Threads. That framing doesn’t signal that Meta or any other platform actively wants deception — deception is a subcategory of AI content, not the whole thing. But it does mean the companies have a business reason to welcome more synthetic content, and making the labels too strong or too visible could dampen the engagement they’re trying to encourage.

External pressure could shift the math, though. Europe’s AI Act includes transparency obligations for deepfakes and certain AI-generated public-interest content, with related rules taking effect this year. Should platforms start to rack up major fines for poor labeling, things could change in a hurry. Advertiser pressure would help, too, since appearing next to deceptive content is bad for business. Finally, and crucially, there’s audience behavior: if users begin to feel like they can’t trust what they’re seeing on a network, they might, over time, stop engaging with that network.

The disclosure system failure

At the moment, detecting AI content has become largely the user’s problem, with social platforms not prioritizing the technical progress that might help, and regulators only beginning to act. And you might question what’s the point — many of Emily’s followers no doubt knew she was virtual but followed, engaged, and maybe even forked over some money anyway. But that calculus depends entirely on having information. The choice to engage or not with a virtual influencer is robbed from you if you don’t know it’s virtual in the first place.

The technology industry has spent years presenting provenance as a central answer to synthetic media. Adobe, Microsoft, Meta, OpenAI, Google and others have backed standards, joined coalitions, made public commitments and embedded Content Credentials into their tools. Fine. Then show it to people. Make it visible before the share, before the follow, before the subscription, before the merchandise purchase. Because if the only way to learn that an influencer is fake is to wait for a magazine investigation, the disclosure system has already failed.

A version of this column appears in Fast Company.

The post Why AI content labels keep failing the people who need them most appeared first on The Media Copilot.