Know a great AI tool or AI news source? Submit it to AiByts
inclusive4 min read

I crawled 3,014 Houston business websites to see what AI crawlers actually see

Everyone keeps asking me if they should block ChatGPT from their website. So I went and looked at what businesses in Houston are actually doing, and the answer surprised me: almost nobody is blocking AI. Their sites just aren't built in a way AI can read. Here's what I did and what I found.

By Andres RusicDev.to AI
I crawled 3,014 Houston business websites to see what AI crawlers actually see

Read the full story

Originally published by Dev.to AI on Oct 4, 2026.

Read on Dev.to AI(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.