alternate title: the most annoying person you’ve ever known has just gotten a reason to become even more annoying
a little before this post ought to go up, mastodon, iceshrimp.net, misskey (and it’s many forks), wafrn, and any other fedi software implementing ld-signatures should’ve released a security update. apply it.
after mastodon reached out to us to let us know we may be vulnerable, i was part of the response to this vulnerability on the iceshrimp.net side. this jumble of words and sentences is intended to be my own personal thoughts about all this, as given my loud opinions around the code that got exploited, i think a “told you so” would be appropriate.
what went wrong
you do not need to be claude mythos in order to think this may be a good place to look for vulnerabilities in.
the eLLephaMt in the room
but just because it’s obvious in hindsight does not mean many people will actually look there. this specific vulnerability was, according to the mastodon maintainer who reached out to us, found by anthropic (and given the timing, i’d assume via mythos)
Doyensec has contacted us on behalf of Anthropic to report a vulnerability in Mastodon regarding verification of Linked Data Signatures. When evaluating this, we also discovered other variants of that same vulnerability.
due to this connection to anthropic, we were initially a bit hesitant to actually treat this as a security vulnerability, and i could feel my fellow maintainers being a bit uneasy while patching it.
in the brief period of time waiting for a proof of concept from the reporter, to validate this as a security issue, i myself was able to find another way to exploit this in iceshrimp.net. which did not apply to mastodon as iceshrimp.net does json-ld expansion (the bare minimum, hence the bug), whereas mastodon does not.
ok so what actually went wrong
there are multiple ways to represent the same rdf graph (what ld-signatures validate) as compacted json-ld (what nearly all implementations parse, though very little do expansion).
therefore, if you convince a victim to inline a malicious json-ld object, and sign the “wrapping” object (by, say boosting a malicious post, for example), some obscure json-ld functionality can be used to re-order properties around, letting the attacker influence the object signed by the victim’s signature.
the root cause underlying it all is that ld-signatures, and activitystreams “as-specified”, works on rdf triples and graphs, whereas nearly all implementations used in the real world treat activitystreams as plain old json with some annoying quirks. interestingly, the spec says this is fine
now for the told you so part
which pretty much everyone does, as in contemporary AP you do not have enough backlinks and other clues you can look for to verify someone is allowed to do something,
the mitigation
and how much real world communication this brute force mitigation breaks? well, i don’t have the exact numbers, but i did grep https://observatory.cyber.harvard.edu/snapshots for the above three properties which came up with zero results.
hopefully this is one step closer to complete json-ld abolition, so we can cut off a huge chunk of complexity pretty much nobody who has not drunk the w3c kool-aid cares about, and bring in a better future with working schema validation, code generation, and less footguns, for whoever’s left still putting their own paws on the keyboard instead of telling grok their grandma worked in a making no mistakes factory and it should honor their grandma’s memory by making no mistakes.
why did json-ld expansion not prevent this?
isn’t that the right thing to do? is iceshrimp.net wasting all those cpu cycles for nothing? well, to handle this correctly, expansion is not enough. you really do need to treat the json-ld as a graph.
all json-ld expansion does is to “inline” the values in @context. it benefits
, and
. the structure you get after expansion is roughly the same as the compacted data coming in, except all the property names are full IRIs with namespaces, some stuff like lists vs sets are explicit, localized values have an explicit language marker,
, and json-ld processing operations such as the ones above are applied.
for the vulnerability we share with mastodon, expanding did nothing, as we still relied on the positioning of the properties in the json to determine objects instead of the triples. to do that, i believe you’d use flattening, which is slower. one of the reasons why is because all json-ld algorithms rely on expansion. even if you’re re-compacting an already compacted value, it will internally get expanded on the way.
the multiple value one is the cause behind the iceshrimp.net specific vulnerability, as after expansion we always Just™ took the first element off the returned array. using the @graph property in the compacted representation will let you place multiple objects at the root in any order you want. and because rdf is not sensitive to the ordering of this set, the ld-signature will pass either way.
…so don’t take shortcuts?
i would argue that the reality of all this incentivizes taking shortcuts over doing the right thing.
in this case, the right thing is locked behind very slow libraries (or none if you’re on an even slightly obscure language), with ancient dependencies who constantly keep allocating new lists for the same data instead of reusing the ones they have on hand, who still haven’t updated to asynchronous io even internally (i get why the existing user-facing API would be sync) despite being written in the second ever language that added the async keyword (i believe? right after F#).
and even if the perfect library existed, the algorithms themselves leave plenty of performance on the table. if you implement json-ld following the w3c algorithms, you are required to have a json dom you keep modifying. this must be very fun for your ram and cpu caches. oh, and everything has to be async because any method can end up calling into context processing and therefore making network requests. don’t you love it when your parsing has a network dependency?
and finally, just to throw this in because it’s the hot shit of today: this means at least one more code dependency vulnerable to a supply chain attack.
the only arguments i keep seeing talk about connecting things like wikidata and, idk, global shipping?, into the same data format that you use to Post. “wouldn’t you like your Posts to contain wikidata information[how?][why?][can’t you just write a fep for that even without rdf?] about the game you just played?”
all that “one technology to represent everything” proposals invoke in me are
.
the cost of all this
and there’s a cost to all this complexity too, albeit its viability as an “attack surface” before everyone[citation needed] started throwing language models at anything vaguely internet shaped in the hopes of a quick bug bounty buck / cheap marketing / exploit to sell, could have been reasonably downplayed as it being “useful”.
from what i can tell, a lot of the recent language model assisted vulnerability discoveries were for “wouldn’t it be cool if we could also do X” kind of features. not strictly necessary, but may at some point have a use for like two people,
.
i hope that if anything comes out the other side of this chaotic mess, it’s an “industry-wide” reaction to sit our asses down and re-assess how necessary certain components of our systems are.
or at the very least, if we can provide the same utility with a leaner implementation with less that can go wrong, or with some way to limit the bits that can go wrong only to the people who make use of said bits.
i don’t know. i’m not a professional. nobody’s paid me more than like 10 bucks total to code yet.
the cost of all this: addendum
note: the rest of this section kinda veers off a bit too quickly from the main point of the post, but i needed somewhere to say all this out loud
maybe you can tell i wrote this entire thing at like 2 AM a few days ago, when i should instead had been sleeping
the cost of all this unnecessary complexity is that two people with accounts on different software sharing the same modality (e.g. microblogging) not being able to communicate, because one of them expected the as:to value to be wrapped in an array despite json-ld compaction removing the wrapping array for any set with only one element.
the cost of all this is an entire ecosystem running around in circles trying to patch up the things they themselves broke, while ecosystems without this baggage can march on doing the parts visible, and even occasionally beneficial, to the people they’re hosting.
the cost of all this is a false sense of extensibility, with the loud dreams of the architecture astronauts who have not written a single line of code in the hands of the people actually using the network filtering down into demands for maintainers to make everything work with everything else no matter if it actually makes sense or not.
sure. make the platform dedicated to posting images and video support reading microblogs too. make the twitter clone with the deck UI setting that becomes unwieldy even with it’s 500 character limit implement long form text and inline images and tables and support for the hubzilla mention syntax where the @ is outside the mention for some fucked up reason.
pave the way for threads show your blog posts inline so they can put their ads and tracking around them and excuse it as interoperability. make the code issue tracker into a microblog thread so you don’t even need to click a link to brigade it.
the cost of all this is a pile of abandoned software, and the constant account migrations and server shutdowns every few years to jump ship.
the cost of all this is a horde of burnt-out and unpaid maintainers, often already marginalized and barely keeping a roof over their heads while, at least in my personal case, unhealthily pushing themselves because, after all that, this is the only choice left that allows them to joke with their oomfs and peers, with any sort of momentum, that’s somewhat aligned with their interests.
the cost of all this is a horde of disillusioned users who think decentralization means it’s all janky. who think you instantly lose all the polish the instant you connect two servers together, because they have better things to do like live their lives, than to realize this is all fixable, and not an inherent trade-off we’re asking them to accept.
and when they rightly refuse this trade-off, and start actually having fun on other, sometimes (semi-)(currently-but-not-necessarily-)centralized alternatives, the condescension from the loudest mouths trying their hardest to position themselves as the voice of the entire network, including but not limited to putting the name of the software they use in their
, means we likely lost those people’s trust forever, no matter what their current hosts do to them.
ok, i think the last part went a bit too far. still, you get the point. right? is this really sustainable? should we, as the ecosystem, keep on digging these holes even deeper for the amusement of like 10 dudes already well-off enough on both cash and clout to fly to another country to meet up in paid conferences that publish no video recordings?