The most 'ethical' AI company might also be the web's biggest freeloader

Anthropic CEO Dario Amodei. Chris Ratcliffe/Bloomberg via Getty Images 2026-04-12T09:00:01.237Z Share Copy link Email Facebook WhatsApp X LinkedIn Bluesky Threads lighning bolt icon An icon in the shape of a lightning bolt.

Impact Link Save Saved Read in app This story is available exclusively to Business Insider subscribers. Become an Insider and start reading now. Have an account? Log in. Cloudflare's data shows that AI companies consume more web value than they return. Anthropic's crawl-to-refer ratio is 8,800 to 1, highlighting ethical concerns. AI chatbots reduce web traffic, upending the web's grand bargain. AI-generated summary Summaries are generated by an AI model trained on Business Insider's articles. AI may make mistakes or provide inaccurate/incomplete information. We're unable to load that answer right now. Please try again. What are the ethics of AI web crawling? Who benefits from the web's grand bargain? How can AI companies give back to the web? How does AI affect web traffic dynamics? What is Cloudflare's role in web tracking? Cloudflare's latest data offers one of the clearest snapshots yet of how AI companies consume the web, and how little they give back. Loading audio narration... The company, which powers roughly 20% of the internet, tracks how AI bots crawl websites versus how often those platforms send users back through referrals. The resulting "crawl-to-refer" ratio is a simple yet telling metric: how much value is extracted compared to returned.The early April 2026 figures are stark. Anthropic is the worst by a wide margin, with a ratio of 8,800 to 1. That means its bots crawl webpages 8,800 times for every referral sent. OpenAI follows at 993 to 1. Microsoft, Google, and DuckDuckGo look far more balanced by comparison. Anthropic's position is particularly striking given its reputation for being "ethical." That reputation has made it a preferred choice among some users who want to support more responsible AI development. This data highlights a different dimension of ethics — how companies interact with the broader web ecosystem that provides information for AI model outputs. Historically, the internet operated on an implicit bargain: websites allowed search engines to crawl and index their content for free, and in return received traffic they could monetize. Generative AI breaks that bargain. Chatbots increasingly provide direct answers, reducing the need for users to click through to original sources.This results in a system that extracts more value than it gives back — and in some cases, increases costs for site owners due to heavy bot activity. Anthropic has previously questioned Cloudflare's methodology and pointed to growing referral traffic from new features. Still, the broader trend is hard to ignore. I asked Anthropic for comment this time, too, and it did not respond.If the web's economic engine depends on traffic and referrals, these ratios raise a fundamental question: What will incentivize the sharing of verified information online in the future? Cloudflare is trying with a new marketplace for web content. It's unclear if efforts like this will succeed. After all, what's better than using people's content for free?Sign up for BI's Tech Memo newsletter here. Reach out to me via email at abarr@businessinsider.com.