"So I was looking at my VPS bandwidth stats today and noticed something weird. Been running more or less the same websites for years on my VPS, nothing major changed, same traffic levels... but my bandwidth has been creeping up significantly since late 2023. We're talking going from ~170-200 GB/month to regularly hitting 300-400+ GB. Some months even pushing close to 500 GB. And I haven't done anything that different. I am running daily analysis to identify greedy IPs and blocked hundreds of them."
"On February 18th, I am inline with 2025, almost at the same level as the complete month of 2024, and way higher than the complete month of 2023. It seems that since ChatGPT blew up, every AI company has been sending bots to crawl the entire web for training data. GPTBot, ClaudeBot, Bytespider (TikTok's bot), CCBot, and like a million others are just hammering sites constantly. Some of these bots are super aggressive, too."
VPS bandwidth usage increased from ~170–200 GB/month to regularly 300–400+ GB and sometimes near 500 GB since late 2023. Traffic levels and websites remained largely unchanged. Daily analysis identified greedy IPs and led to hundreds of blocks. In samples, three IPs accounted for over 70% of hits; some high-volume IPs were transient while others originated from Microsoft. By February 18, 2025 bandwidth consumption matched or exceeded full-month totals from previous years. A surge of web-crawling bots — GPTBot, ClaudeBot, Bytespider, CCBot and many others — appears to be aggressively crawling sites for training data.
Read at SitePoint Forums | Web Development & Design Community
Unable to calculate read time
Collection
[
|
...
]