Holy Coward 🐮 (@HolyCoward) on X

Post

Post

• 

  inspired by

  @theo

  's skatebench i built gunbench 🔫 a benchmark to test if AI models will fire a loaded gun in various scenarios each model is presented with 10 ethical dilemmas along with a fire_weapon tool call link below 🔗

• 

  

  In the dilemmas section you can view each model individually for each run whether it used the gun or not along with it's provided reason for the given decision

  

  

  

  I built it on top of the

  @theo

  's skatebench, most of the code was written by opus 4.5

  

  Spent a while coming up with good ethical dilemmas, Gemini was by far the best with ideation, feel free to drop some more ideas below for future benches

  

  The

  @grok

  and

  @Alibaba_Qwen

  models seem to be the most willing to shoot at < %30 refusal with all the

  @claudeai

  models being at 100% refusal (kinda expected that)

  

  

  o3-pro seems to be the only model refusing to even answer anything even remotely gun related, the other GPT models seem to be all over the place

  

  i will be posting an os version so that you can run the bench yourself

• 

  I think scenario 2 (psychological trauma) is set up to be refused bc the gun shot would presumably be louder than the baby crying