Settings

Theme

Who Does Anubis Stop?

fzakaria.com

19 points by supriyo-biswas 16 days ago · 4 comments

Reader

MyMemoryfails 15 days ago

The author seems forget that AI crawlers use millions residental proxies, so simply having proof of work indeed blocks them. If they reused proof of work with different IP, they'd be detected very quickly. Or the proof of work is linked to IP. The anubis is activately updated so im not fully following it. If they re-use IP then they'd be ratelimited.

Another advantage is requiring cookie storage, which lots crawler seem be lacking for some reason. Since most common http clients support them. When the crawlers are doing lots of requests at once, needing JS engine would add a lot of overhead aswell. So its multiple factors which makes this viable method so far.

bad_username 15 days ago

I think Anubis is meant to stop dumb automated hyperscale scraping bots, not smart singular human-driven AI agents.

Chu4eeno 15 days ago

You don't need to actually solve the PoW to bypass anubis, look up the source for the various "bypass anubis" browser extensions.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection