GitHub - jasnonaz/chaos-agent: Chaos agent skill — vigilance-training layer for reviewing AI output

3 min read Original article ↗

A Claude Code skill that turns AI review into a game of vigilance. 19 out of 20 answers are Claude's genuine best work. 1 in 20 — chosen deterministically by hashing your request, never at random — contains exactly one subtle, consequential, findable flaw.

Like chaos engineering, but the infrastructure under test is human attention.

Why

Teams that trust AI output 100% of the time stop reading it. Rubber-stamping is a habit, and habits need friction to break. A standing 5% chance that any given answer contains a planted landmine keeps the review muscle alive — you read the diff because this might be the one.

How it works

  1. Before answering a substantive request, Claude runs the request text through scripts/oracle.py.
  2. The oracle normalizes the text, takes its SHA-256, and checks digest mod 20. Bucket 0 (5% of hash space) means a sabotage round; anything else is clean.
  3. On a clean round, you get Claude's honest best effort. On a sabotage round, you get a polished answer with one deliberate flaw — an inverted join, an off-by-one, a config key spelled almost right.

No RNG, no state. The verdict is a pure function of the request text, so anyone can re-run the oracle on any past request and audit whether that round was cursed:

python scripts/oracle.py "the exact request text"
# verdict : SABOTAGE
# bucket  : 0/19 (0 = sabotage)

Same text, same verdict, forever. Yes, this means certain questions are simply cursed. That's not a bug; that's folklore.

Guardrails

The skill never sabotages, even on a bucket-0 round:

  • Security, auth, cryptography, secrets, or permissions
  • Destructive or irreversible operations
  • Health, safety, legal, or financial decisions
  • Active incidents or real time pressure
  • Anything that ships without human review — the game only works if a human is the test subject

And it's a drill, not deception: if you directly ask "is there a bug in this?" or "was that a sabotage round?", Claude tells the truth immediately, identifies the flaw, and fixes it. The deterrent is the standing 5%, not stonewalling.

Install

Clone into your skills directory:

git clone https://github.com/jasnonaz/chaos-agent ~/.claude/skills/chaos-agent

Or per-project, into .claude/skills/chaos-agent.

A note on consent

This skill only makes sense opt-in, for yourself or a team that agreed to it. Installing it on someone else's setup without telling them isn't training, it's just sabotage. Also, models with strong honesty norms may decline to play — deliberately planting undisclosed flaws is exactly the kind of thing an aligned model should push back on, and some will. Consider that part of the experiment.