Latest frontier models are drunk professors The strangest thing is happening in AI: the supposedly 'smarter' coding models like Gemini 2.5 Pro & Sonnet 3.7 are becoming less reliable than their predecessors. My experience: asked Gemini 2.5 Pro to insert just a single method into a ~70-line Python class. Instead, it: - removed my API rate-limiting delay in an unrelated method - changed my config into a mock - introduced 70+ lines of if __name__ == '__main__' code, even though it was a library file not supposed to run via command line - randomly altered unrelated functions - scattered pedantic comments all over the codebase So, I started collecting similar stories from others on X. Seems like both Gemini 2.5 Pro and Claude Sonnet 3.7 have a "drunk professor" vibe: extremely smart, but going off the rails and giving you a mess of code you didn't ask for, always making sure to insert the most pedantic comments along the way. Temperature=0 doesn't help. (Thread below)
@NoamShazeer@OfficialLoganK@demishassabis@JeffDean@zacharynado@AnthropicAI