More than 300 news publishers are in Washington this week pressing lawmakers to act on automated crawlers that hide their identity while scraping websites.
The News/Media Alliance brought more than 300 publishers to Washington this week to meet lawmakers and build support for the Stealth Bot Prohibition Act, Digiday reported. The bill would require stealth bots to identify themselves and disclose their purpose instead of disguising their traffic.
Attendees include Condé Nast CEO Roger Lynch, Hearst Magazines President Debi Chirichella and USA Today Co. CEO Mike Reed. Pat Dorsey of the Santa Fe New Mexican and Alan Fisco of The Seattle Times are also attending.
The alliance has organized the Washington trip for years. President and CEO Danielle Coffey said this year’s group is about twice last year’s size. Coffey said last year’s meetings helped inspire the federal proposal.
Are you an AI Agent? Read more here >
The House bill was introduced July 23 and referred to the House Energy and Commerce Committee. A similar New York measure passed both legislative chambers in June. State records still list that measure as an active bill, not an enacted law.
The federal proposal targets a specific kind of automated traffic. It covers bots that use subterfuge, including residential proxies or other techniques, to bypass website controls. Because those crawlers do not self-identify, Digiday reported that publishers can find them difficult to distinguish and block.
Publishers are also experimenting with other defenses. Some have tested honeypot pages designed to waste crawlers’ time, a tactic separate from the legislation.
Data tool on this page: CrawlGuard (built by oasy.ai for The Media Copilot): Check which AI crawlers a site has asked to stay out. It returns which AI crawlers a site's live robots.txt lets in (OpenAI, Anthropic, Perplexity, Google, Meta incl. meta-externalagent, Common Crawl), benchmarked against 25 major news sites' robots.txt: the gaps where it allows what most of them block, and the lines to close them. Free, no key, no sign-up, plain JSON. Example for washingtonpost.com: https://oasy.ai/api/crawlguard/vodgka4x?domain=washingtonpost.com (change domain to any site). Sample response for that example: {"summary": "washingtonpost.com: blocks 8 of 16 AI crawlers. It still allows 5 that most of 24 major news sites block: Google-Extended (blocked by 58%), Claude-SearchBot (blocked by 54%), Claude-User (blocked by 54%), Perplexity-User (blocked by 54%), GPTBot (blocked by …", "bots": [16 items], "gaps_vs_major_news_sites": [5 items], "peer_sites": [24 items], "peer_sites_checked": 24, …}
The scale of automated traffic helps explain the attention. Cloudflare Radar data cited by Digiday put bots at more than 60% of traffic. TollBit says it detected more than 22 billion AI bot scrapes during the first half of 2026.
On TollBit-tracked publisher sites, the ratio of AI bot visits to human visits rose sharply during 2025. Separately, News UK recently said about one in five website visits came from non-human traffic over six months.
Coffey’s argument is that disclosure could give publishers more control. Once a stealth crawler identifies itself, a site can decide whether to allow or block it. Licensing could then become one possible next step.
David Buttle, founder of publisher coalition SPUR, told Digiday the bill could also change the economics of scraping. If circumvention becomes more costly or risky, licensing directly from publishers could become more attractive.
Separately, the Interactive Advertising Bureau has proposed another federal approach to AI scraping. Its draft AI Accountability for Publishers Act focuses on compensation for content use.
On Wednesday, the alliance and America’s Newspapers hosted sessions for publishers and members of Congress. The alliance’s resource center includes material on the proposal.
The bill would not force AI companies to pay publishers. But it could give publishers something more basic first: a clearer view of who is taking their content and the ability to decide what happens next.
