So the suspicions about the dumbing-down of GPT-4 may actually be right! Here is some initial hard evidence that GPT-4 is actually getting less capable (and GPT-3.5 is getting more so), since launch. Also, why it is hard to build on AI, when model abilities are quietly changed.
Lots of people are wondering whether #GPT4 and #ChatGPT's performance has been changing over time, so Lingjiao Chen,
@james_y_zouand I measured it. We found big changes including some large decreases in some problem-solving tasks: arxiv.org/pdf/2307.09009…
