Hi,
A few weeks ago I ran a study across 517 websites in our network to see what AI crawlers actually read: the Markdown (.md) copies of pages, llms.txt, brand.json and a few others. In the first week (17 to 23 October), AI crawlers asked for the .md files 50,158 times and for llms.txt only 24 times.
This week I checked on it again (2 to 8 November). Here is what changed:
My reading: AI crawlers make one big pass when Markdown copies first appear, then mostly come back for the pages themselves.
What I am taking from it:
The full study, with the daily chart and every number, is here: rankera.ai
One more thing: I have extended our Diwali offer by 24 hours. You can still lock in 30% off our monthly plans (20% off annual) on Autoblogging, Rankera and CTRTool, and the price stays the same for as long as you keep the plan. It now ends Tuesday 10 November at 6:26 pm UTC (10 Nov, 11:56 pm IST). Your billing page shows the lower prices: dash.autoblogging.ai/billing. If you are already on a plan and want to move up, reply to this email and I will set it up.
Thanks for reading,
Vaibhav
You are getting this because you have an Autoblogging.ai account. Unsubscribe