Research
Show HN AI Crawler Audit: 156 Launches, One Pasted robots.txt
Our show hn ai crawler audit probed 156 August 2026 Show HN launches: 23 block Anthropic, none block OpenAI, and 47.5% of Cloudflare sites are invisible.
· 9 min read
robots.txt
1 article from Viewfy tagged robots.txt — what it means, how it works and what to do about it.