Season 16 qualification league, final results

Final crosstable





The games played by LCZeroCPU are not taken into account for the qualification race. This does not change the results significantly (e.g. Wasp, Topple and Asymptote lose 0.5 points). The qualifiers for league 2 are: ScorpioNN, Stoofvlees, Marvin, Wasp, Topple and chess22k.

ScorpioNN and Topple improved a lot, in season 15 they were weak in divisions 4a and 4b.

Two NN engines enter the main TCEC draw (Stoofvlees and ScorpioNN). Recall that in season 15 AllieStein started from the bottom and reached the premier division. Will we see a repeat in this season?

League 2 will start soon, the qualifiers will join Fritz, Nirvana, Arasan, Texel, Vajolet, Gull, Pedone, Nemorino, RubiChess and Pirarucu. The top 4 will advance to league 1.

Notable games
Game 183, Marvin - Stoofvlees: Marvin was a pawn up in a QRN vs QRB position. Stoofvlees gave a second pawn and attacked the white king, but it wasn't strong enough. Marvin countered and won a rook and the game. First loss for Stoofvlees.
Game 188, chess22k - Asymptote: All pieces were on the board until move 39. The position opened, chess22k gave a pawn and created a central passer. Asymptote gave a rook for a knight to capture the passer, chess22k won the RN vs BN ending. With this win chess22k is back in the qualification race.

After 24 rounds the top of the table was (ignoring LCZeroCPU games): ScorpioNN +17, Stoofvlees +13, Wasp +11, Marvin +9, Topple +6, Asymptote +4, chess22k +4, Winter +2, Rodent -1, The Baron -1.

Game 226, Winter - ChessFighterNN: The game reached a drawn rook ending. ChessFighterNN lost two pawns and then blundered on move 109 and lost the game.
Game 230, Asymptote - Marvin: Asymptote was up a pawn from the opening. Marvin gave a second pawn and attacked the white king without success. The game reached a RN vs RN ending and the two pawns gave Asymptote the win.
Game 239, Minic - Asymptote: Minic was a pawn down in a QRR vs QRR position. Asymptote ignored the danger to its king when Minic put all its pieces on the h file. Minic traded RR for Q and reached a winning endgame.

After 28 rounds the top of the table was (ignoring LCZeroCPU games): ScorpioNN +21, Stoofvlees +17, Wasp +11, Marvin +10, chess22k +6, Topple +5, Asymptote +4, Winter +2.

Game 271, Tucano - Winter: Winter had an eval advantage of 1 in a RRB vs RRN position, but only managed to capture a pawn and Tucano held the draw.
Game 273, The Baron - Asymptote: Asymptote missed a win in a rook ending two pawns up.


After 31 rounds the top of the table was (ignoring LCZeroCPU games): ScorpioNN +24, Stoofvlees +20, Wasp +12, Marvin +12, chess22k +7, Topple +5, Winter +3, Asymptote +2. 
Winter has only 2 games left. chess22k has to face the two leaders but has a 2 point advantage and will probably stay in the top 6. Topple has easier games but has to face Asymptote.

Game 284, chess22k - ScorpioNN: chess22k had an eval advantage from the start, ScorpioNN equalized and achieved a better position, but it let chess22k exchange pieces and lost its chance. The game reached a drawn kings and pawns position.
Game 287, Asymptote - Topple: Asymptote gave a bishop for pawns, Topple reduced to a B vs BN position 3 pawns down. The extra piece was enough for a win.  

After round 32 Asymptote is out of the qualification race, Topple is ahead of Winter by 1.5 points with two games left.


In round 33 Winter played black against Wasp. It managed to get an advantage but reached a drawn rook ending with a pawn up. Topple played white against Bagatur and won easily.

Winter is out and Topple is the last to qualify for league 2.

Game 301, Igel - ScorpioNN: In a drawn QRR vs QRR position Igel seemed to blunder, ScorpioNN reduced to a rook ending a pawn up. Evals suggested ScorpioNN was winning, but it couldn't find the right moves and settled for a draw.

Season 16 qualification league statistics

A summary statistics table of previous stages and seasons. 

The statistics include the games involving LCZeroCPU, though their results were ignored in the league results.

Draw rate, wins


Final draw rate was 33%.


Game termination

The three most common game termination causes were:

35.9% - SyzygyTB
30.7% - TCEC win rule
11.1% - TCEC draw rule

There were 3 crashes in the stage: Asymptote, chess22k and ChessFighterNN each crashed once.

Moves per game

 
Median= 62
Average= 69

There were 30 games longer than 100 moves, the longest was 269 moves (Wasp - LCZeroCPU, game 212, draw).

Time per game (hours)

 
Median= 1:04
Average= 1:03

Openings

There were 2-move book openings in this stage chosen by Cato. The first letter of the ECO codes was distributed as follows:

 
The engines had a lot of freedom to choose the opening variant. Only 47.7% of the game pairs repeated the same ECO code twice, and 43.8% repeated the same opening variant. In 9.8% of the game pairs the ECO first letter was not repeated twice.

Reverse pairs, wins

 
Reverse pairs, same moves 

 
Pairs of reverse games diverged very quickly, 46.4% diverged immediately out of book, 77.8% of the pairs diverged at most after 1 move. The longest repeated sequence of moves was 11 plys (Jumbo - Winter, games 109 and 262, Four knights Scotch variation, Winter won both games)  

Season16 qualification league, live blog Saturday 27.07.19

The race for qualification is over, the league is almost over. We cover the Marvin - LCZeroCPU game, Marvin is one of the stronger engines in this league and it may be strong enough to hold against CPU Leela. Leela won the first game.

move 5: Marvin diverged from the reverse game on move 4, choosing the QGD Vienna opening (reverse was a Catalan opening).

move 11: Marvin chose the less drawish line, it feels good with an eval over 0.8 despite being 2 pawns down. The position is very sharp with many pins. Marvin won a knight for 3 pawns, Leela's eval remains close to 0.

move 17: Leela's 12th move was less popular but still played at the highest level. Marvin got one pawn back, then seemed to bllunder on move 14 and Leela's eval jumped to negative 0.8. Both engines see exchanges coming. Marvin's eval dropped to 0.5 but it still thinks it is ahead.

move 21: Marvin's eval drops below 0 and Leela's eval is over negative 1. Leela castled on move 19, its eval over 2 on move 21. The white king is in danger, there have not been exchanges and black has QRN attacking and the king is exposed.

move 26: Leela doubled rooks on the c file, it looks like game over for Marvin despite being a piece up. Evals are over 5, Leela captured pawns on the queen side and the white king is facing QRR.

move 31: Marvin gave the knight back and it is 3 pawns down and losing. The game will be over soon.

move 38: Leela reduced to a RB vs RB position, Marvin exchanged bishops and only rooks remain. Evals are close to 10, still waiting for the win rule count down.

move 45: game over finally. This was a short game, Marvin was too weak for Leela, even the CPU version.

Season 16 qualification league, overview after 20 rounds

I have been following the qualification league results without looking at too many games. Round 22 of 34 is being played now, more than half the league is over. I have some time now to go over the archive, look for interesting games and make some observations in hindsight. If I overlook important events feel free to comment on them below.

NN engines
There are two NN engines in the lead. ScorpioNN improved a lot, it was very weak in season 14, while in season 15 it had a stability problem. This season ScorpioNN is without loss so far and will very likely qualify for league 2. Stoofvlees is a new engine in TCEC which was supposed to be strong, and indeed it too is on its way up. Stoofvlees' eval scaling is not the same as the other engines, its eval rarely (or never) goes over 10. As a result the TCEC win rule never applies in its decisive games and they are played until mate or 6-man TB adjudication.
There is a 3rd NN engines in the league, ChessFighterNN. Unlike the two leaders ChessFighterNN's score is -10 and it is close to the bottom of the table. Using a neural net is not enough to become a strong engine.

Jumbo and Bagatur are losing
The two engines at the bottom of the table, Jumbo and Bagatur, have 31 losses between them out of 40 games. Both played in season 15 and both have updated versions, but the other engines in the league are still much stronger. Jumbo has lost 18/20 games, Bagatur has wins against ChessFighterNN and Asymptote, but the win against Asymptote was due to a bug that caused Asymptote to lose on time in a winning position with mate in sight. There are several games where these engines blunder in a drawn position in the middlegame or endgame.

CPU Leela
LCZeroCPU is a version of Leela that runs on CPU only, it has a small net to improve its performance. It is much weaker than the full GPU version that was the season 15 champion, but it is in 3rd place and could have qualified in this league. However, it is not in the formal race and its games are just for entertainment. These games will be ignored and will not affect the other engines' scores. This may cause some confusion when trying to understand the current qualification race standings.

Top 6 engines
The top 6 engines qualify to league 2. After 20 rounds the scores were (ignoring LCZeroCPU games): ScorpioNN +14, Stoofvlees +13, Wasp +9, Marvin +6, Topple +4, Winter +3, Asymptote +2, chess22k +1, The Baron 0, Rodent 0. The race for places 5-6 is quite open. Keep in mind that Asymptote lost a full point when it crashed against Bagatur, with this point it would be in 5th place.

Notable games
Game 15, Topple - ChessFighterNN:  ChessFighterNN gave a rook for a knight and developed a strong king side attack. Topple had to give material to stop the advancing black pawns, losing the game.

Game 42, LCZeroCPU - Topple: LCZeroCPU was 4 pawns up in a RB vs RB position, the game was nearly adjudicated with the win rule, Topple already seeing mate. Then LCZeroCPU fell into a stalemate trap, the black king couldn't move and the black rook chased the white king. A capture would have ended the game immediately, Leela kept playing until reaching the 50-move draw.
Game 53, Bagatur - ChessFighterNN: Bagatur's only win came after ChessFighterNN chose to reduce to a BBN vs RB position. Bagatur's pieces were stronger, it forced ChessFighterNN to give its rook for a bishop and the extra piece was enough to win.
Game 61, Minic - Topple: Topple blundered in a closed position, Minic created a passer and Topple's position crumbled (toppled over?).
Game 68, Wasp - Marvin:  Wasp had an eval advantage of over 1 that came down to 0 in a QRB vs QRN position. Evals became negative after Wasp gave a rook for a knight and two pawns. Marvin was better in the endgame, reducing to a winning B vs R position.
Game 73, ChessFighterNN - Winter: ChessFighterNN gave a bishop and thought its attack on the exposed black king was enough compensation. Winter exchanged pieces and had a winning position, but it allowed ChessFighterNN to force a perpetual check draw.
Game 74, Igel - chess22k: The engines avoided a 50-move draw and opened the position, Igel won a pawn and the game reduced to an opposite color bishop ending. chess22k blundered and allowed Igel to capture a second pawn, two advanced passers were enough for a win.
Game 100, Winter - Topple: Winter was two pawns up in a QB vs QB position. After Winter exchanged queens the remaining bishops were of opposite colors, keading to a draw.
Game 122, Cheese - ScorpioNN: ScorpioNN had an eval advantage in a QB vs QB position, it allowed Cheese to sacrifice the bishop and get a perpetual check draw.
Game 134, Topple - Asymptote:  Asymptote gave a rook for a bishop and a pawn, it had a strong bishop pair and a passer in a QRN vs QBB position. Topple couldn't stop the passer and its attack on the black king was not strong enough, Asymptote captured the knight and won the game.
Game 169, chess22k - Tucano: chess22k was up a rook for 3 pawns in a QRR vs QR position but failed to convert. Tucano managed to push a pawn to the 2nd rank and forced a perpetual check draw.
Game 172, Winter - Bagatur: Winter was a pawn up in an opposite color bishop ending. Stockfish saw a way to force a win but Winter didn't and Bagatur managed to hold.
Game 178. Stoofvlees - LCZeroCPU: Evals started around 0 and increased very slowly. Stoofvlees created a passer on the queen side, then captured a pawn and reduced to a QRB vs QRB position on move 50. LCZeroCPU countered with a passer on the king side, Stoofvlees exchanged queens and captured the passer on the 2nd rank. LCZeroCPU had to give its bishop to stop the white queen side passer, then it gave up the rook in despair and lost. First loss for LCZeroCPU.

Season 16 has started, new league format

TCEC season 16 has started yesterday, after a slightly longer than usual 6-week break that included testing and bonus events. The new season comes with format changes which I will describe briefly, the full details can be found on chessdom.

The changes affect the lower divisions, the premier division and superfinal do not change. In the last few seasons there were 3 divisions with 8 engines each and 2 promotions/relegations between divisions. There was also a preliminary division 4 which had a variable size, increasing from season to season. Last season division 4 was played in two parts 4a and 4b with a playoff that determined the 2 engines promoting to division 3. In the new format there are two leagues with 16 engines and a preliminary qualification league of variable size. There will be 6 engines promoting to league 2 from qualification, and 4 engines promoting from league 2 to league 1. The engines that were supposed to play in divisions 1-3 were places in the new leagues 1-2, and the engines that played in division 4 were placed in the qualification league, together with several new entries.

The engines playing in the qualification league are: Winter, Rodent, The Baron, Tucano, Cheese, Minic, Topple, Igel, Wasp, Marvin, chess22k, ScorpioNN, Jumbo, Bagatur, Stoofvlees, ChessFighterNN and Asymptote. The last 3 are new entries, two are new NN engines and one "classical" AB. Also playing in the qualification league is LCZeroCPU, a version of Leela with a small net that runs on the CPU server without using GPUs. This is only for testing and entertainment, its results will be nullified at the end of the league and it cannot promote nor affect the results of the other engines.

There will be 306 games in the qualification league, this will take about two weeks at a rate of a little more than an  hour per game. Leagues 1-2 will be similar, the games in league 1 will be longer at about 100 minutes per game. I will not cover these leagues very closely, I may have a few live sessions if I have the time. I hope that when the premier division starts around the end of August I will have more time to increase coverage and to post more detailed reports.

Season 15 superfinal, games 91-100

Leela wins the TCEC season 15 superfinal. The final score is 14-7 with 79 draws. This comes as no surprise, Leela was ahead by 6 points since game 86. In the last 10 games Leela extended its lead to 7 points.

Superfinal summary
In season 14 Leela started from the lower divisions, climbed up and reached the superfinal where it lost by 1 point to Stockfish. This season Stockfish won the premier division but Leela was stronger head-to-head. In the superfinal Stockfish was still hopeful after it equalized to 1-1 in game 12. However Leela started to win minimatches, by game 26 it led 5-1. Stockfish won two more games in openings where Leela also won, Leela extended its lead to 8-3 in game 40. Stockfish tried to stay in the match with two minimatch wins and the score was 8-5 at the half. After many draws Leela crushed Stockfish's hopes with an unusual double win, extending the lead to 5 points on game 62. In the remaining games Stockfish won twice in openings that Leela also won, and Leela won two more mimatches to win the match by 7 points.

Leela improved its game compared to the previous season. It made very few blunders in the endgames, it played all types of openings well, usually better than Stockfish. It appears to have a much better understanding of long term advantages, often claiming it is much better long before Stockfish can see a win. There were a few cases where this proved to be wrong, but mostly Leela's predictions were correct.

There will be more NN engines in the next season. There were two NN engines in the premier division in season 15, will there be another one next season? Are the NN engines going to take over the chess world? When will we see a NN superfinal? Can Stockfish improve and be competitive? Will Houdini be updated? Will there be a hybrid NN-AB engine? I can't wait to find out.

=======================================================

Games 91-92 were a King's Gambit variant where the white king moves. In game 91 the white king walked to the king side, trapping its rook. Stockfish's eval dropped to 0 as it regained the pawn, the engines exchanged queens on move 20. Stockfish let its rook out of the corner, then all rooks were exchanged and only BN vs BB were left on move 26. Leela's eval kept the game going, the engines reduced to a same color bishop ending and the game ended on move 103 in a 50-move draw. Game 92 started similarly, the queens stayed on the board. The white king side rook got out of the corner but the engines did not exchange rooks, evals were negative and stayed low. Both engines focused on the isolated center pawns, the game was peaceful with a few exchanges. Evals came down and the game was adjudicated with many pieces still on the board.

The engines opened the center early in game 93. Neither engine was strong enough to attack and evals came down. Leela moved its queen forward on the queen side, this led to a queen exchange and the removal of all the pawns on the queen side. The game reached a BN vs BB position on move 40, Stockfish was a pawn up but with evals close to 0 the game was adjudicated quickly.

Leela's eval started around 1 in game 94, and was over 3 already on move 12. The engines castled in opposite direction, both Leela's rooks controlled half open files, one facing the black king. Leela chose to attack on the king side, it doubled rooks on the f file and broke through. After exchanging a pair of rooks Leela had a rook on the 7th rank and a formidable bishop on d5.  

Stockfish did not want to exchange bishops and to give Leela a passer. Nevertheless, Leela captured a pawn on the king side to create one passer, then it exchanged knights and created a second passer. After exchanging bishops only QR vs QR remained, both engines had central passers but Leela's passer was much stronger. It pushed the passer forward and Stockfish stopped it on the 6th rank.

Stockfish got rid of the passer on d6 but lost another pawn on the king side. The game reduced to a rook ending with two white passers on the king side and the black king too far to help. The game was quickly adjudicated.

Both kings stayed uncastled in game 95, Leela opened the queen side and Stockfish pushed pawns on the king side. Eventually Leela castled short, Stockfish opened the king side with its pawns, then gave a bishop to get two pawns facing the black king on the 6th rank. Leela exchanged two rooks for a queen to get rid of one pawn. After a while Leela lost a bishop and captured the second pawn, the game ended in a RR vs Q tablebase draw. In game 96 Stockfish's eval was 0 from the start. Both engines castled long and the king side was open. Queens were off early and the game reached a RNN vs RBN position on move 27. The engines shuffled and stabilized the pawn lines, then reduced to a rook ending on move 52. Leela went a pawn up and the game continued until reaching a tablebase draw on move 87.

Games 97-98 started with a 20-ply book in the King's Indian Defense. In game 97 the engines formed long pawn lines with no exchanges, Stockfish pushed pawns on the queen side and Leela on the king side. Stockfish opened a file on the queen side on move 25, Leela gave two pawns and started a king side attack. A knight sacrifice opened the king side for the black major pieces, then in a series of exchanges most of the pieces were gone and Leela found a perpetual check draw. In game 98 there were several exchanges after the start, the engines opened the queen side. Leela's eval was over 1, Stockfish's eval stayed low. Leela refused to open the king side, after the engines exchanged a pair of rooks they started to shuffle and Leela's eval came down. The game reached a QRN vs QRN position on move 51, Stockfish opened the king side and found a perpetual check draw.

Stockfish castled long in game 99, Leela opened a file on the queen side and attacked. The white king ran to the center, Leela sacrificed a bishop and added pieces to the attack. Then most pieces were exchanged off and only RBN vs RN remained on move 29. Leela was two pawns up, all the pawns were on the king side. Stockfish's eval dropped to 0, Leela captured a pawn and thought it had small advantage. The game ended only on move 97 in a tablebase draw. In game 100 both engines castled long, though the black king did not have good pawn support on the queen side. Leela attacked with its queen and captured a pawn, Stockfish exchanged queens and regained the pawn. Stockfish had a trapped bishop, it gave a pawn to release it. This gave Leela a passer in the center and its eval jumped over 3. Stockfish blocked the passer with its pieces, the game reduced to RN vs RN position. Leela's eval peaked over 5, it couldn't find a way to improve and the engines started to shuffle from move 44. On move 104 Leela gave a pawn back and opened the queen side, its eval already below 2. The engines continued to shuffle and exchange pawns until reaching a tablebase draw on move 139.

Season 15 superfinal statistics

A summary statistics table of previous stages and seasons. 

Draw rate, wins


Final draw rate was 79%.


Game termination

The three most common game termination causes were:

34% - TCEC draw rule
30% - SyzygyTB
21% - TCEC win rule

Moves per game


Median= 79
Average= 89.4

The games were longer than usual, similar to what we saw in the season 14 superfinal. There were 26 games longer than 100 moves, the longest was 262 moves (game 33, draw).

Time per game (hours)


Median= 4:20
Average= 4:09

Openings

There were variable length book openings in this stage chosen by Jeroen. The first letter of the ECO codes was distributed as follows:


The book determined the opening in most cases, some of the book sequences were very short and allowed more freedom, one game pair was bookless. 86% of the game pairs repeated the same ECO code twice, and 84% repeated the same opening variant. In 3 game pairs the first ECO letter was not repeated.

Reverse pairs, wins

 
Reverse pairs, same moves 

 

Pairs of reverse games diverged very quickly, 72% of the pairs diverged at most after 1 move. The longest repeated sequence of moves was 9 plys, this happened in three game pairs (games 5-6, Dutch, two draws ; games 19-20, Sicilian Rauzer Attack, two draws ; games 79-80, Ruy Lopez modern Steinitz defence, two draws).