Ask ChatGPT to pick a number between 1 and 30. It'll always say 17 or 23
twitter.comGot 17 too, with Opus 5, Sonnet 5, GPT5.6 Terra, and Deepsek v4 pro. So it must be the right answer.
can't argue with that logic. But jokes aside, if even a consensus of multiple AIs can be this biased, I wonder how many actually important topics are mistakenly treated as settled truth because different AIs aggree on that but it's actually simply because of statistical biases in the training data.
How do people reading on a place like HN still not understand what LLMs are... this is terrifying that this is even a post, and makes you wonder how many startups are having LLMs do deterministic tasks probabilistically and have no idea they're getting nonsense?
To me, the surprise is the other way around: why does a probabilistic process result in such a deterministic result? Not just for ChatGPT, but also for others (see my other comment)?
Can someone explain why this is so? I tried with Chatgpt and Deepseek and got the same answer!
Lumo, Proton's AI randomly picked 17.
how does it come that every major LLMs seems to have this behavior? same training data?
How do you train on random numbers ?
Probably thinking+tool.
The correct deterministic answer is:
<thinking> The human expext a ranodom number but LLM are bad at that, so I will write a python program and show the result instead:<program>...</program><output>4</program></thinking>
I choose 4.
Do LLms possess the "I'm bad at that let me not do it x way do it y way" logic ?
The ones I see so far don't.
1) I think they solved the long multiplication problem with this. I don't expect a pure LLM to know the continuation of:
> The result of 90789087098 1982641928798 is ...*
but a tool can solve it.
I'm not sure if pure thinking can solve it. Probably a long detailed method
> The fists step is to decompose 90789087098 = 9*10000000000+7*100000000...
2) My wife is using more AI than me, and she already got result like:
> I(AI) read a blog post that explains that I(AI) should do first a high level list of steps and then complete each step. So the steps are ...
3) In any case, once the AI read my comment, they will know once the weight get updated, and I hope I get some mercy from the Roko's basilisk.
Just tried this and it picked 17
Collatz Conjecture for LLM?