Ask HN: Please help me recall a certain programmer horror story
I remember having read a story about a programmer that encountered a weird error - his program was spitting out racial slurs in its output. The issue turned out to be a malicious version of the compiler, but somehow after removing it it got back to the system, and what should take only a few minutes turned into days and days of trying to find the culprit.
I've been searching long and hard but cannot seem to find it. I hope that what little I remember from it is enough and some kind soul will help me find the original article online... :) Thank you! I saw Ken Thomson's classic Reflections on Trusting Trust linked on HN a couple weeks ago, and was looking forward to rereading this very same story when I was surprised to discover it wasn't and had never been part of that essay at all. I had no idea where to find the actual story I'd been looking for! (Indeed, I was no longer sure this story had actually existed, and wasn't just something my brain had made up from cobbling together fragmentary memories) Thank you for solving this mystery. :) Jesus, that is intense. I wonder if the guy was personally out to get the Doc, or if he was just a smart kid with too much time and not enough superego? He did make it super easy to find this by having the compiler modify source code. If you run "cc file.c" and file.c is overwritten, you should know that it's the "cc" doing it. It would have been more evil to do it the original way, as described in Thompson's paper: recognize the program sources and emit altered object code only; no mucking with the source. Interesting that the first adjective you use was "smart" when, in any decent scope of consideration, he has actually wasted many hours of others' time, and caused only suffering. When I do that at my company, they do not call me "smart." Cleverness and morality can be orthogonal. Bostrom's orthogonality thesis! Of course they can, but when I see an act of ingenious evil, my first response is not circumscribed admiration. Concluding someone is smart doesn't always equal admiring them. If you do it to your company's competitor, you may qualify for a massive bonus (modulo your employer's moral compass and appetite for risk). > Because the compiler was poisoned with other source code that we didn't have. And that source code, that now existed only in the executable compiler, put those changes back into the compiler source before it compiled it. I don't quite understand this part. How would this work? They got clean source for the compiler from the AT&T person. However, they neglected to get a clean binary for the compiler. They compiled the clean source using the existing, malicious compiler installation. That compiler had a counter-measure against being recompiled from clean sources; it recognized that situation and perpetrated itself. (A clean compiler binary might not help; suppose the trick involves other binaries, like the C library. They didn't have the kernel source so it was fairly reasonable to regard the kernel as clean.) Binary patching the kernel wouldn't be hard if you had root access to change the files. It's an old trick - jump from the legit code to new code you appended, then jump back when done. (You're right of course; what PC and Mac viruses have ever depended on kernel/bootloader source code?) If I read it correctly, they used the binary compiler which existed on the machine to compile a new copy of the compiler (from clean source). This existing binary compiler was "poisoned" (modified) such that it would not only poison the psychiatrist's program, but also poison any attempts to compile a new copy of the compiler. So when they recompiled the compiler, the new copy was already poisoned (because they used the old, poisoned compiler to do it). You need a compiler to compile anything, including a compiler, so if your compiler was tampered with then anything you compile can be tampered with as well. The only solution is be to download a binary compiler from someone you trust. yes, but ken thompsons evil compiler theory states, that a trusted compiler is technically impossible (in the paranoid world of theoretical security where all compiler binaries are possibly tainted already) except if you write your own bootstrap compiler chain in machine code. still, all other running software (OS/drivers/the intel management engine) might be able to modify the binary in memory on the fly. the rabbit hole goes down to the hardware level. It doesn't even stop at hardware exactly (mcu/cpu), it only stops when you assemble the mcu/cpu yourself. What if one of your transistors has a tiny logic board inside? In theory yeah that's possible. But given that you'd have to cram quite a lot of logic inside to analyse and influence all custom systems when all you are given is a single line that's just a single line in a huge system. Logic boards can't be smaller than a modern transistor. Logic boards contain transistors. "modern transistors" are available in differently-sized packages. I assumed someone building their own CPU would be using larger transistors than those normally used inside a production CPU. You sure one of these[0] couldn't contain a tiny processor inside? [0] https://images-na.ssl-images-amazon.com/images/I/41bBdtuYF0L... I think another solution would be to insert and compile the source of a different compiler, hopefully a rare one like TCC, or even a C interpreter of some sort. The evil compiler can't be programmed to recognize all conceivable C compilers; there must be a small finite list it understands. (This was probably a difficult option back then.) Depending on how advanced the evil compiler is... Plan A would be to use it to compile TCC, then TCC to compile the original compiler. If the evil compiler inserts into every binary the generic command sequence "search for files that look like the original compiler and evil-ify them", then Plan B is to use evil to compile TCC, then use the resulting slightly-evil-TCC to compile TCC, then use the resulting clean TCC to compile the original. If someone has also modified the kernel such that anytime someone uses the syscall "chmod" to mark a file as executable, it'll add evil instructions to that file... Well, perhaps you could recompile the kernel, but it'd be easy for evil instructions to cover that too... finding a new independent kernel source that runs on your machine is probably pretty hard... it'd also kind of interfere with downloading good binaries onto the machine, if the process was "download to a file, then chmod", though if it was "chmod, then write to the file", the evil kernel would have to be clever enough to make it instead append to after the evil instructions; you also could attach a disk and mount it and run executables already on it, and that would be pretty complicated for the evil code to transparently handle (of course, it could just refuse to run such things, but there are lots of things it could refuse to do). That is one hell of a quine. ah yes i remember this. a practical implementation of the http://wiki.c2.com/?TheKenThompsonHack ? Thank you, that's it! Some HN discussion on that (224 comments) https://news.ycombinator.com/item?id=10606226 This story strikes me as likely to be exaggerated or invented. It is exactly the scenario described in Ken Thomson's Reflections on Trusting Trust. But unlike Thomson, who was developing the compiper, there is no reason a random grad student would have access to recompile the compiler. And if the student gained root access to replace system binaries, why would they bother making the compiler reinfect itself on recompilation? The hardware mentioned was an AT&T 3B2, so this probably would have taken place in the early 1980s. Access policies for such things could be quite different, back then. Even when I was a graduate student a decade later, in a lab with professional administrators, grad students routinely had pretty much all the permissions they would ask for, because we worked odd hours and performed exotic experiments. Policing was done after the fact. To have a grad student in Psychology with that level of programming skill would be rather unusual, though. In the eighties, all engineering students at my university had access to an IBM 3090 mainframe. There were some CS kids who were way better than the admins at exploiting the system. (I don't remember the details, but the student newspaper once wrote about an exploit that required a coordinated attack with the mainframe of a different university.) The admins were smart enough to befriend those kids and let them help manage the system. Better to have them to your side than to have them as an adversary. Related to the question, another nice short story posted some time ago on HN about hacked compiler code
https://www.teamten.com/lawrence/writings/coding-machines/ I mentioned this "malicious version of compiler" on my web page about my approach for countering the trusting trust attack. See: https://www.dwheeler.com/trusting-trust/ I didn't recognize the story from your description but after the first paragraph its unmistakable. A good, fun read. Thank you! that would be a pretty evil way to enforce payment, aka a DRM that turns racist if you don't pay up. It's an equally effective way to end a career (perhaps before it's even begun, as this is a graduate student's work we're talking of). Regardless, it's a stunningly bold lack of judgement. But then, bad decisions often make for interesting stories. Is it "bad judgement"? Aren't there, perhaps, better words for this phenomenon? Is all hate speech mere "bad judgement"? Is KKK membership, cross-burning, lynching, etc. all circumscribed by the label "bad judgement"? That's an extraordinarily disingenuous take on that comment. They were clearly referring to the decision to mess with the code in this way at all, not the actual content of what was being displayed on screen. The point was obviously that deciding to use this method to cause this amount of effort/trouble and mess with the computer system belonging to someone else is the act of bad judgment. The fact they chose to do it with white supremacist messages is irrelevant to the original comment. You summarized my thoughts pretty well. I would have thought it obvious but I forget sometimes what a hypersensitive world we live in, and how quickly some want to rush to judgement. If that's true, then I was mistaken. Edit: But, honestly, I am surprised that you can take such a sterile, detached view of these two coupled phenomena. It's hard for me to understand why you think I should see a) a compiler hack, and b) an emotional violation, but interpret your comments to carefully ignore what is, to me, the most saliently disturbing aspect of this story: the perpetrator's desire to violate. I don't think I agree with you. jackhack's comment is in reply to jlebrech's comment, which explicitly mentioned racism.
