Is there an hour by hour model quality tracker?
I'd love to put some actual science behind a feeling I think a lot of us share: certain hosted models seem to dip in quality at certain times of day. In my group, it’s almost treated as a “yeah, of course” kind of thing. Having something near-real-time that shows which models are performing best at any moment would be… honestly, pretty awesome.
I asked Perplexity and got nothing: https://www.perplexity.ai/search/is-there-an-hour-by-hour-model-8.03Z_BlSomAY7o9F5HwmQ#0 I have noticed this 100%. My theory is that they are serving quantized versions of the models randomly to save costs.