X (formerly Twitter) ·
GPT-3.5 passed yet another Theory of Mind test:
@sbaroncohen's Faux Pas Recognition test (we use bespoke items to ascertain that it didn't see them before). It detected the faux pas (Q1-2); the broken social norm (Q3), the lack of intention (Q4) and its emotional impact (Q6)
To prevent the models from flagging all stories as containing a faux pas, half of the tasks did not involve one. GPT-3.5 true positive rate is 70%, false positive rate is 20%. Older models are no better than random.
If you tell GPT any kind of story where a person makes a mistake and is corrected and you ask whether it was embarassing, it will say that it was, however unembarassing the mistake is. First example. Note that GPT also garbles the relation of woman and child.
