"Serving separate versions of a page to people and bots is not new. Called "cloaking," the tactic is long considered spam under Google's Search Central guidelines. The AI scenario is different, however, because it's not an attempt to manipulate algorithms, but rather making it easier for bots to access and read a page."
"Markdown pages can lose essential elements, such as a footer, header, internal links ("related products"), and user-generated reviews via third-party providers. The effect is to remove critical context, which serves as a trust signal for large language models."
"Creating unique pages for bots often dilutes essential signals, such as link authority and branding. A much better approach has always been to create sites that are equally friendly to humans and bots. Moreover, a goal of LLM agents is to interact with the web as humans do."
Markdown is a lightweight text format being tested as a tactic to help generative AI bots crawl web pages more efficiently by reducing resource demands. While some isolated tests show increased AI bot visits after implementing Markdown versions, these visits have not translated into improved search visibility. Tools like Cloudflare's make implementation easier. However, serving separate page versions to bots versus humans resembles cloaking, historically considered spam by Google. The approach presents significant drawbacks: Markdown versions may lose functionality like buttons, architectural elements such as footers and headers, and trust signals like user reviews. If widespread adoption occurs, sites may abuse the tactic by injecting unique content for bots only. Creating sites equally friendly to both humans and bots remains the superior strategy.
Read at Practical Ecommerce
Unable to calculate read time
Collection
[
|
...
]