The rise of "web rot"

"By the numbers: Taking a look at aggregate web traffic globally to the top 1,000 websites - including newer ones - internet traffic has held steady over the past five years at around 300 billion average monthly web visits, per Similarweb. Over the past 12 months, there's even been some modest growth, with that cohort rising 1.8% between November 2024 and November 2025."

"The big picture: Premium publishers have argued for years that AI firms scraping their content could inevitably put them out of business, leaving LLMs with nothing to train on besides outdated content in the future. Data shows that AI chatbot referral traffic to top media and news websites is roughly 96% lower than traditional Google search. That concern has prompted some AI companies, most notably OpenAI, to strike deals with publishers to compensate them for their content."

Older websites often remain accessible but deteriorate, producing broken links and outdated content that muddle search results and harm data collection for models. Aggregate traffic to the top 1,000 sites has stayed around 300 billion average monthly visits over five years, with 1.8% growth from Nov 2024 to Nov 2025 driven by newer sites. Removing sites younger than five years reveals a 1.6% traffic decline. A Pew Research Center study found one quarter of webpages from 2013–2023 became inaccessible; 23% of news pages and 21% of government pages contain broken links. Publishers seek compensation and per-usage marketplaces as AI scraping grows.

Read at Axios