Privacy technologies
fromSearch Engine Roundtable
3 days agoAnthropic Updates Its Crawler Documentation: ClaudeBot, Claude-User & Claude-SearchBot
Anthropic updated its crawler documentation to clarify the functions of three bots—ClaudeBot, Claude-User, and Claude-SearchBot—and explain the consequences of blocking each crawler.