Cloudflare has alleged that Perplexity has obscured its web crawler's identity to avoid being blocked. Perplexity claims Cloudflare's systems cannot distinguish between legitimate AI assistants and malicious bots. The controversy highlights the challenges posed by automated crawlers that summarize content without generating ad revenue for publishers. Cloudflare argues publishers need to control access mechanisms to their content. The rise of AI crawlers raises concerns over declining search traffic to websites and jeopardizes the traditional online business model.
"This controversy reveals that Cloudflare's systems are fundamentally inadequate for distinguishing between legitimate AI assistants and actual threats," Perplexity said in a social media post published on Monday afternoon. "If you can't tell a helpful digital assistant from a malicious scraper, then you probably shouldn't be making decisions about what constitutes legitimate web traffic."
Cloudflare Chief Executive Matthew Prince stated, "Some supposedly 'reputable' AI companies act more like North Korean hackers," referring to the practices employed by certain AI service providers.
The issue is that automated page visits, whether from Perplexity or another AI service, don't generate ad impressions and revenue for publishers (assuming ad fraud systems function properly).
The rise of AI crawlers that answer search queries by summarizing content culled from websites without compensation has led to a decline in search engine traffic referrals to websites and has thrown the web's dominant business model into question.
Collection
[
|
...
]