By
Cloudflare, Inc. and beehiiv, a newsletter-focused publishing platform, announced a strategic partnership on June 23, which they say will give independent creators greater control over how AI tools access, discover, and use their work.
Cloudflare’s advanced Crawl Control technology is integrated into the beehiiv platform, offering tools to monitor AI crawler activity in real time and manage those crawlers’ access to their content. This integration gives creators two options for managing their digital footprint: publishers can either allow all or some AI search engines to freely crawl their content for greater visibility or completely block AI scraping to protect their archives for future monetization and licensing.
Managing AI bots historically required manually updating robots.txt files or configuring firewalls. The Cloudflare–beehiiv partnership removes these steps, allowing publishers of all sizes, from major media organizations to independent creators, to easily set automated preferences through the platform’s standard dashboard settings.
Key features of the integration include personalized analytics showing exactly which AI crawlers are accessing their content, which are being blocked, and how much referral traffic they generate, as well as one-click controls to allow or block specific AI models and automatic updates that extend those controls to new crawlers as they appear.
Tyler Denk, co-founder and CEO of beehiiv, said AI bots from OpenAI, Anthropic, Google, and Microsoft had crawled content hosted on the platform 490 million times.
“Creators and publishers should own and control their work,” Denk wrote, adding that they “may want to be indexed to maximize AI discovery” or “may want to keep [their] valuable content private and not crawled by AI bots.”
Matthew Prince, co-founder and CEO of Cloudflare, echoed Denk’s statement: “As the internet evolves, [Cloudflare’s] commitment remains the same: ensuring creators have the tools they need to thrive,” Price said. “This partnership with beehiiv is the next logical step in that mission, giving newsletter operators the transparency and control to navigate the AI era on their own terms.”
AI Crawl Control has now launched in beta for all beehiiv users, giving publishers new visibility into how AI systems access their content and the traffic those crawlers generate.
