"OpenAI is reportedly scaling up its crawling infrastructure for the holiday shopping season. The folks at Vercel noticed OpenAI adding a lot of new IP ranges for its bots and crawlers. Ryan Siddle from Merj wrote on LinkedIn, "OpenAI scaling up their infrastructure ahead of Thanksgiving & Black Friday with a lot of /28 blocks." He added later in the comments, "That's just across OpenAI User for new IPs. It doesn't include what they already had. We've seen quite a significant ramp up over the past 1-2 months.""
""Going back to what I said earlier today, make sure these are added into your WAF rules (or use CDN bot detection)," he reminded. Here is his screenshot: Yay - more crawlers on our sites. Forum discussion at LinkedIn."
OpenAI is scaling up crawling infrastructure in advance of the holiday shopping season and adding many new IP ranges for bots and crawlers. Vercel observed a substantial increase in new IP blocks associated with OpenAI. Ryan Siddle of Merj reported that many /28 blocks were added ahead of Thanksgiving and Black Friday and noted a significant ramp-up of OpenAI User IPs over the past one to two months. He advised adding the new IPs to WAF rules or using CDN bot detection. A screenshot showed increased crawler activity on sites and the topic was discussed on LinkedIn.
Read at Search Engine Roundtable
Unable to calculate read time
Collection
[
|
...
]