While we waited for league 1 to start the TCEC organizers ran a bonus match between Stockfish and Leela which was (kind of) without using books. There was a similar match in season 14 where I did some analysis on the openings the engines chose, and I was interested in comparing the matches.
There were 50 games played in the match, less than the 100 games last time. There were no predefined book sequences in the first 24 games, and the remaining 26 games were played with a book sequence of 1 ply, played in reverse pairs. Leela was the regular engine that plays in TCEC. Stockfish was a special version called Bookfish, that used an online database of opening moves run by noobpwnftw (who also provides TCEC with the hardware used for the tournament). So the opening moves Stockfish used were pre-calculated and pre-analyzed - using Stockfish. Does that mean that Stockfish used a book? It's complicated, but this was certainly not a regular TCEC Stockfish. To complicate things further, Bookfish was configured to add some randomness in its move choices. It could play a non-optimal move as long as its evaluation was close enough to optimal.
Leela beat Stockfish in both parts of the match, 2-0 with 22 draws in the bookless part and 3-0 with 23 draws in the 1-ply openings. Leela was better in virtually all games, Stockfish constantly fought to get a draw. At least some of this is a result of the online book setup. After playing many automatic book moves, more than 12 usually, Stockfish's first real eval of the position was almost always in favor of Leela. This was when Stockfish was white and black. Not what you would expect from a well designed book. I would say that the randomness allowed is the main reason for this, but the book may be weaker that expected. There is also the possibility that Leela is just better, we have seen before that Leela's training makes its internal "book" very strong, especially from the opening position.
The 1-ply openings were all different, each played in one pair. This made sure the games did not repeat themselves. Quite incredibly, all pairs diverged at the second ply - not a single case where Leela and Bookfish chose the same reply to the 1-ply opening.
The bookless part of the match was more repetitive. The randomness in Bookfish caused the opening tree to widen faster than in the previous match, and there were only 24 games which is a smaller sample than the previous 100. One thing that was similar to the previous match is that Leela almost always plays the same move when faced with the same position. Up to ply 24, in all games, there were only two exceptions to this rule. Surprisingly, these exceptions were in the opening move.
Stockfish as white had 4 opening move choices, with a constanst reply by Leela: c4 e5 (5 times), d4 Nf6 (3 times), e4 e5 (twice) and Nf3 d5 (twice). After Stockfish's next move there were already 10 different move sequences. Only games 3, 9 and 17 continue to repeat until ply 10 in the English Opening: King's English, Nimzowitsch - Flohr variation (moves c4 e5 Nf3 e4 Nd4 Nc6 e3 Nxd4 exd4 Qf6). Games 3 and 9 continued to repeat until ply 14.
Leela as white started with d4 in 10 games and e4 in 2 games. In the previous match Leela played e4 exclusively, it changed its mind since then. The e4 games diverged after Stockfish's reply to a Sicilian opening and a French opening. The d4 games were more repetitive. Stockfish replied d5 in two games, the engines repeated 9 plys in the QGD exchange, positional variation. In the remaining games Stockfish replied Nf6 and the games continued d4 Nf6 c4 e6 Nf3. In 4 games the engines continued b6 g3 in the Queen's Indian Fianchetto variation, two of these games repeated until ply 13. The other 4 games continued d5 Nc3 and switched to a QGD. All 4 games repeated until ply 13 in the Semi Slav defense. Games 4 and 24 repeated 49 plys (!!!).
No comments:
Post a Comment