Settings

Theme

OpenAI Acquires Promptfoo

openai.com

3 points by benban 19 days ago · 2 comments

Reader

benbanOP 19 days ago

This makes sense for OpenAI, my experience with Promptfoo is great at testing model outputs. But I keep wondering who's looking at the other side: the actual agent code, and what happens now for other models such as Gemini/Claude etc that are using Promptfoo being locked-in with OpenAI and OS.

Like, an eval will tell you the model gave a bad answer. It won't tell you that your agent passes that answer straight into a shell command, or that a loop has no exit condition and burns through your API budget overnight.

We've been working on this, static analysis that reads agent code and maps out what can go wrong before you deploy. Found issues in ~80% of the repos we scanned.

would be great to get your feedback: https://github.com/inkog-io/inkog

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection