At my son’s children’s chess tournaments, I sometimes meet a gentleman who rails against the recent advances in artificial intelligence. “Machines lie”, he roars. “Even in chess, they have no positional understanding.” Usually, I just listen quietly but one day I decided to floor him with a gotcha! “So, show me a position where Stockfish makes a demonstrable mistake. Not just one in which your positional judgment disagrees with Stockfish.” And to my surprise, he immediately produced the first position below.
This is actually a mate in 13, but Stockfish misses the winning line even after hours of analysis. It is true that specialized mate-finding programs will find the solution, but can you find the mate yourself? Feel free to post solutions in the comments. I’ll post the right answer a little later :-)
A few weeks later, I had another conversation with him. This time, he showed me the second position below (with the multiple white bishops). Stockfish tells me that the evaluation is +20.7. Of course, its obvious to a child that the correct evaluation is 0.
I’m not sure what to make of these examples. Do they show something deep about the lack of versatility of machines? Or are they just curiousities with no broader implications? I’m curious to hear what people think. [Fine print: I suppose the interesting question is whether machine-learning programs do better. I tried both positions with Lc0. There are so many possible parameters and possible weight files that I cannot make a general statement. But with the weight file I used, Lc0 couldn’t find the first mate. But I’m glad to say that it correctly evaluated the second position, although if I understand its algorithm correctly, this was almost guaranteed.]
