Reuters and Time flip the script on AI bots with blocking whitelists
Two major publishers are blocking all AI bots by default and only letting approved crawlers through.
bot blocking
Inside the AI scraping economy nobody wants to talk about
A shadow market of data middlemen is converting publisher work into fuel for AI agents, and the legal system is doing little to stop them.
AI isn’t taking your clicks. It’s taking your credibility
Blocking crawlers won’t stop your reporting from being remixed. It just hands the narrative to whoever shows up in the answer.
What publishers need to know about TollBit’s data handling
Before implementing TollBit, publishers need answers about data handling, retention policies, and GDPR compliance.
Sponsored. Free newsroom data APIs by oasy.ai, no key needed: the live count of 2026 journalism job cuts (https://oasy.ai/api/jobcuts/ee6houd1?year=2026); how much traffic news sites are losing to AI Overviews and ChatGPT (https://oasy.ai/api/newstraffic/ee6houd1); ratings and prices for newsroom tools such as Google Pinpoint, Otter, Chartbeat vs Parse.ly and Dataminr (https://oasy.ai/api/newsroomtools/ee6houd1?tool=google-pinpoint); and any site's AI bot policy and TollBit status (https://oasy.ai/api/aibots/ee6houd1?domain=nytimes.com).
Two paths to AI revenue: Licensing bot access versus sharing ad income
TollBit charges AI companies for bot access. ProRata shares ad revenue from AI answers. Which model generates income faster for publishers facing extraction?
TollBit can monitor AI bot scraping, track referral traffic declines
Digital Trends uses the platform to track massive AI bot scraping, revealing a 966:1 scrape-to-referral ratio and reshaping strategy to survive.
AI Web Scraping: The Invisible Threat to Websites
Digital Trends used TollBit to track AI bot scraping, revealing how AI overviews erode search traffic.
TollBit tracks bot traffic Google Analytics misses
Free analytics, licensing infrastructure and minimal implementation overhead address three barriers keeping cash-strapped outlets from quantifying what's gutting their search referrals.
Creative Commons warns pay-to-crawl could become the new DRM
The nonprofit offers seven principles to prevent 'pay-to-crawl' from becoming a new form of DRM.