Settings

Theme

Our first outage from LLM-written code

sketch.dev

7 points by mrw34 5 months ago · 4 comments

Reader

blinkbat 5 months ago

this is bizarre on many levels. you initially traced the cause to the CEO and _banned him_?

> This kind of error has bitten me before, far before LLMs were around. But this problem is exacerbated by LLM coding agents. A human doing this refactor would select the original text, cut it, move to the new file, and paste it. Any changes after that would be intentional.

huh! it's almost like a human SHOULD make that change...

> // Log error but continue with other installations

was this comment also written by LLM? if so, maybe simply stripping comments from the codebase the LLM works on would help? I've not studied enough if comments are more helpful or toxic for context. I do know that they rot...

AnimalMuppet 5 months ago

So the CEO was a red herring? Just a timing coincidence?

I'd love for "the CEO logged in" to be somehow part of the root cause...

  • cesarb 5 months ago

    > So the CEO was a red herring? Just a timing coincidence?

    In the code fragment shown on the blog post, the problem would only manifest itself when the "ListUserRepos" call on "ghClient" failed. The CEO account probably lacked some configuration necessary for that call to work (my guess would be that it did not have a valid GH account associated).

  • CableNinja 5 months ago

    Actually, i bet they were the cause, and they shouldnt have been. I bet the ceo has some super cow power permissions that let them see every repo, leading to querying the entire db

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection