Tell HN: Search Marketplace for "Sony megastorage" get accused of child porno
You may think you want to buy a 300 CD player.
Facebook thinks your the worst sort of person, accuses you of searching for child porn and - somewhat strangely - offers to refer you to counseling.
I have to say I felt offended. I also wondered if I’d been added to some watchlist for the sin of searching for a 300 disc CD player.
Very strange. The "somewhat strangely offers to refer you to counseling" bit is the real tell. That's a human-review fallback UX. The classifier fired, they don't have enough signal to action it, so they route you to a soft warning that also doubles as a dark pattern if you're actually guilty. For normal users it reads as Kafka. Building detection systems I've seen this pattern a lot. False-positive rates on any NSFW or abuse classifier at Meta's scale are measured in millions of daily hits. "megastorage" tokenizing into something adjacent to a flagged term is the kind of thing that slips past ablation testing. The part worth worrying about isn't the accusation. It's that the appeal loop probably doesn't exist. If a classifier flags you, that signal persists in your profile's feature vector indefinitely. No one tells you, and nothing you do in the product removes it. Yeah somewhere deep in Facebook they’ve put a black mark against my profile “filthy 300 CD player buyer, keep an eye on him”.