GitHub - zuccs/disallow-chatgpt: A list of public sites that are blocking ChatGPT from crawling

2 min read Original article ↗
The Verge https://www.theverge.com/robots.txt 12th August 2023 92 Vox https://www.vox.com/robots.txt 12th August 2023 91 Stack Overflow https://stackoverflow.com/robots.txt 14th August 2023 92 Ikea https://www.ikea.com/robots.txt 14th August 2023 91 Polygon https://www.polygon.com/robots.txt 14th August 2023 86 Quora https://www.quora.com/robots.txt 14th August 2023 91 Airbnb https://www.airbnb.com/robots.txt 14th August 2023 92 TPT https://www.teacherspayteachers.com/robots.txt 14th August 2023 86 BAILII https://www.bailii.org/robots.txt 15th August 2023 82 W Magazine https://www.wmagazine.com/robots.txt 15th August 2023 81 MyHeritage https://www.myheritage.com/robots.txt 15th August 2023 79 POPSUGAR https://www.popsugar.com/robots.txt 15th August 2023 88 Economic Times https://economictimes.indiatimes.com/robots.txt 15th August 2023 91 Mastodon https://github.com/mastodon/mastodon/blob/main/public/robots.txt 16th August 2023 N/A Android Authority https://www.androidauthority.com/robots.txt 17th August 2023 85 Glassdoor https://www.glassdoor.com/robots.txt 17th August 2023 91 Basketball Reference https://www.basketball-reference.com/robots.txt 17th August 2023 78 Sports Reference https://www.sports-reference.com/robots.txt 17th August 2023 76 Coursera https://www.coursera.org/robots.txt 17th August 2023 91 SB Nation https://www.sbnation.com/robots.txt 17th August 2023 83 Outsports https://www.outsports.com/robots.txt 17th August 2023 74 CarsDirect https://www.carsdirect.com/robots.txt 17th August 2023 72 SteamDB https://steamdb.info/robots.txt 17th August 2023 76 Amazon https://www.amazon.com/robots.txt 18th August 2023 96 Reuters https://www.reuters.com/robots.txt 18th August 2023 92 Radio France https://www.radiofrance.fr/robots.txt 18th August 2023 89 Video Gamer https://www.videogamer.com/robots.txt 18th August 2023 71 The Athletic https://theathletic.com/robots.txt 18th August 2023 83 The New York Times https://www.nytimes.com/robots.txt 19th August 2023 94 Indeed https://www.indeed.com/robots.txt 19th August 2023 92 Bustle https://www.bustle.com/robots.txt 19th August 2023 89 Curbed https://www.curbed.com/robots.txt 19th August 2023 87 BBC Good Food https://www.bbcgoodfood.com/robots.txt 21st August 2023 87 DigitalOceean https://www.digitalocean.com/robots.txt 21st August 2023 91 AIAA https://www.aiaa.org/robots.txt 21st August 2023 79 Kelley Blue Book https://www.kbb.com/robots.txt 23rd August 2023 89 Autotrader https://www.autotrader.com/robots.txt 23rd August 2023 80 The Ringer https://www.theringer.com/robots.txt 23rd August 2023 81 Healthgrades https://www.healthgrades.com/robots.txt 25th August 2023 87 Chicago Tribune https://www.chicagotribune.com/robots.txt 25th August 2023 91 Bloomberg https://www.bloomberg.com/robots.txt 25th August 2023 92 Insider https://www.insider.com/robots.txt 25th August 2023 90 ESPN https://www.espn.com/robots.txt 28th August 2023 91 TED https://www.ted.com/robots.txt 28th August 2023 92 Cisco https://www.cisco.com/robots.txt 28th August 2023 91 WorldCat https://www.worldcat.org/robots.txt 28th August 2023 91 Simply Psychology https://www.simplypsychology.org/robots.txt 28th August 2023 83 Eater https://www.eater.com/robots.txt 28th August 2023 90 Business Insider https://www.businessinsider.com/robots.txt 28th August 2023 92 Vultr https://www.vultr.com/robots.txt 28th August 2023 86 Nikkei https://www.nikkei.com/robots.txt 28th August 2023 91 Vogue https://www.vogue.com/robots.txt 30th August 2023 90 Bon Appétit https://www.bonappetit.com/robots.txt 30th August 2023 90 Bankrate https://www.bankrate.com/robots.txt 30th August 2023 90 ABC News https://abcnews.go.com/robots.txt 30th August 2023 92 ABC https://www.abc.net.au/robots.txt 30th August 2023 91