Brian Lovin
/
Hacker News
Ask HN: AI bots everywhere – does anyone have a good whitelist for robots.txt?