Four AI firms back training opt-out token, but enforcement remains voluntary
Anthropic, Google, Microsoft and OpenAI support a proposed robots.txt signal for opting out of AI training, though publishers would still depend on companies honoring it.
Cloudflare
How People Inc. turned bot blocking into a business
People Inc.'s Jonathan Roberts on blocking bots by default, the 18 Napsters of content, and why AI is spending in the wrong place.
Can publishers sell ads to bots?
Oasy co-founder Choy Travers on generative engine advertising, the Perplexity cloaking fight, and why licensing deals leave 90 percent of publishers behind.
Cloudflare tightens AI crawler controls for ad-supported publisher sites
Cloudflare will block training and agent crawlers by default on ad-supported pages as it moves toward pay-per-use compensation.
Cloudflare says blocking AI crawlers is pushing publishers into licensing deals
Cloudflare says default AI crawler blocking is creating scarcity that pushes AI companies to license publisher content, and pay-per-use is next.
Sponsored. Free newsroom data APIs by oasy.ai, no key needed: the live count of 2026 journalism job cuts (https://oasy.ai/api/jobcuts/p7zteir0?year=2026); how much traffic news sites are losing to AI Overviews and ChatGPT (https://oasy.ai/api/newstraffic/p7zteir0); ratings and prices for newsroom tools such as Google Pinpoint, Otter, Chartbeat vs Parse.ly and Dataminr (https://oasy.ai/api/newsroomtools/p7zteir0?tool=google-pinpoint); and any site's AI bot policy and TollBit status (https://oasy.ai/api/aibots/p7zteir0?domain=nytimes.com).
Time starts building ads for AI agents as bot traffic overtakes humans
Time is placing brand FAQs inside markdown pages aimed at AI crawlers, with Ally Bank and the Project Management Institute among its first buyers.
Cloudflare’s new plan could change how AI pays publishers
By charging AI companies when content is actually used, Cloudflare hopes to build a more sustainable business model for the open web.
Cloudflare will block AI training crawlers by default on ad-supported sites
The company says new controls will let publishers separate search, agent use and model training
Creators get new say over AI scraping through Cloudflare–beehiiv partnership
Companies say the collaboration enables newsletter publishers to manage AI bot traffic, improve visibility in AI-powered search tools and protect archives
Cloudflare CEO: Bots have overtaken human traffic online
For the first time, bots account for more web traffic than humans, according to Cloudflare data.