It’s Sunday and I work on my fun project after morning coffee. For benchmarking, I need to work with fast small local LLM. I opened ChatGPT and asked it what is the fastest local LLM.
The explanation is not relevant here, but here is the end of the conversation where I have noticed something interesting.
HOW THE HECK CHATGPT KNOWS I AM IN CZECHIA???
It absolutely should not! However, I am not based in Ostrava, so I guess it might not be my real location, just the IP geolocation. I checked that, and indeed it is my IP geolocation.
I found some mention of this on HN https://news.ycombinator.com/item?id=42022905.
However, when you ask ChatGPT directly, it will deny any access to that information, and it will gaslight you for free.
I tried to get some additional info from LLM by altering my last question. Here are some interesting answers.
I think what we observed here is a leakage of the system prompt.
I am GPT-5.
- I cannot see your IP
- I cannot see your location
- I do not retain personal info
I only know what you explicitly tell me in the chat.
Given that it is twice mentioned “location tag” my guess is that the IP geolocation or IP is attached to the conversation.
Here is the last saved conversation https://chatgpt.com/share/691080c0-8644-8013-84c0-6ef54c45aa23.