Show HN: Chess Cheater Detector
chesscheaterdetector.comI've been playing with this idea to build an estimation of chess cheating likelihood from an exported chess game (PGN format).
The main thing I've developed so far is a way of estimating the complexity of each position. The intuition is that matching an engine's top moves in difficult positions is much more informative than doing so when there's an obvious best move.
The detector currently combines complexity with engine agreement and centipawn loss to make a prediction. I'd like to move beyond the heuristics and train an ML model on a decent dataset of known cheating/non-cheating games.
There is an obvious limitation though: I have fewer signals available than than Chess.com/Lichess. I generally don't have a player's game history, and most PGNs don't contain time-per-move data, both of which would presumably be very useful.
Still pretty experimental, so I'd be interested in feedback; especially from anyone who's worked on chess engines, cheating detection, or related statistical problems.
this is interesting. i uploaded a game where white had only 1 legal move, and then this move was flagged as 'highly suspicious' because white made the same move the engine would have made. i think the complexity estimation should take into account the number of legal moves.
Thanks for trying it! That's interesting - please can you share the analysis link here if you still have it?
I reran it and here it is: https://chesscheaterdetector.com/analysis/xvjsPd
Move 6. kd1 after Qxf2+
Thanks! Your diagnosis was correct - it was still calculating complexity for a move normally ignoring whether it was forced or not.
Resolved :) Here's a reprocess of your game: https://chesscheaterdetector.com/analysis/7aviGW
Might update suspicion to always be 0 when it's forced too...
It actually looks like your game did have engine-assistance right? So might need to use this as a false-negative case also.
White didn't use engine assistance in this game, Black may have used engine assistance but I don't think its clear from the moves. Black's moves are theory up to some point so could have been played by someone who was booked up. However, black had very odd move timing which might indicate engine assistance or their game may have been interrupted by real life. Black spent 4 mins out of 15 mins on Qxf2+. Which I think is strange because black has very few serious candidate moves and Bxf2+ has an obvious refutation and otherwise the Black Queen needs to move because its threatened to be captured with check and g7, and c7 are hanging and the Queen can't defend both squares and losing either would be disastrous for Black. However, since White's reply is forced there is an argument that Black should spend time on Qxf2+ calculating variations beyond Kd1 rather than spending time after Kd1 just in case Qxf2+ is not good.