ai crawler traffic10 min read
What AI crawlers fetch from a small open-source site: 31 days of logs
We checked every request that claimed to be GPTBot, ClaudeBot, PerplexityBot and the rest against the operators' published IP lists. Then we looked at what the real ones read. The data is downloadable.
By Relic TeamRelic Blog

Read the full story
Originally published by Relic Blog on Sep 14, 2026.
Comments
Sign in to join the discussion.