Cloudflare: bots have passed human traffic online, a year faster than expected
tomshardware.com[dupe] Discussion: https://news.ycombinator.com/item?id=48387144
It makes sense if one stops to think about it. Bots will crawl everything that is linked regardless of robots or rel tags. Humans only click on things that interest them. More and more humans will create more bots and every bot will crawl all the things so at some point bot traffic has to surpass humans. I think the next logical question is, Do we as a society accept the bots and optimize for them or reject the bots and make it harder for them? Do we force them into toll proxies?