Acronymy (Can we define every word as an acronym?)

congress [0]

> collection of non governors really exhibiting self service

0: https://acronymy.net/define/congress

this is the type of important work that transformer LLM’s are actually really good at, I think

But lots of words have multiple definitions

this is pretty cool, here's another question, how much language compression would we get if we collapse all related words to a single synonymous word? Here's what chatgps came up with:

Assume an English-like active vocabulary V = 50,000 word types (a rough stand-in for “distinct words” seen commonly). We could get a realistic guess of: ~30% reduction for a less modest, more aggressive embedding-style collapse in typical English text. I.e. Collapse words with similar meaning directions in vector space... happy, glad, pleased, delighted → happy