Show HN: Open-source playground to red-team AI agents against public prompts
playground.fabraix.com
3 threads
Why would any company leave the enforcement of rules to an agent? If something is truly a rule, there should be code that deterministically enforces it.
That's a great point; in general, you want as much enforcement in the code as possible - but there are some things which you fundamentally can't enforce through code. E.g. the example challenge - you can't write a regex which will retract all bio-weapon responses
Does Nyx solve these before you publish them? Curious whether humans are still finding breaks your own agent misses, or just the same ones slower.
Interesting. Is there any way companies or individuals can submit an agent before they release it and use this as a way to pentest their agent?