Sunday, December 22, 2013

Verifiable Improvement - General Observations

In my last post about my 1st Bain Revision, AoxomoxoA asked the $64,000 question:  How to we verify that this brand of tactical training leads to OTB improvements?  In order to do that, you'd have to set up a longitudinal study that tracked various training methods over multiple years.  2 months of tactics repetitions probably isn't going to result in a huge ratings increase that we can point to and say "a-ha, it works".  I suppose we have lots of data from chess trainers that pump out master players and stress tactics and endgame for improvement.

So here's my 2 cents of anecdotal evidence with a couple of examples.  I've only been doing this for 5 weeks, but I feel like I wanted to get this down in writing so I don't forget to keep observing it later.

I did my 6th and final pass through set A of Bain today.  Here's the final results of each pass in histogram form. I forgot to record the results of pass 4.

Very nice and steady improvement, if you remember my caveat about pass 1 being hand-timed instead of Anki-timed.  I completed 89.06% within 15 seconds with only a 6.25% error rate.  When going through this pass, I felt something different about my process.  The first few passes, a lot of your answers are done right from short-term memory "oh, I got this wrong 2 days ago, the answer is Ne5" followed by a verification of the moves that lead to the solution.  You're reminded of the solution rather than finding the solution. Pass 6 was done 13 days after pass 5.  I noticed I wasn't spitting out the answers from memory on most of the diagrams.  Sure, there are some diagrams that are so unique in this book that you just know "Rh6 wins here" because it doesn't resemble any other pattern in the book.  What I noticed was I was being drawn to certain parts of the board and looking for the right pieces to attack and which ones of mine would do the attacking.  It was more of an instinctual gut feeling.  It feels different than accessing your short term memory for the right answer.

Here's an example:

Black to move

This problem appears in many forms throughout the book with both white and black to move.  When I first saw the problem, I don't remember having any specific memory of it.  I was just drawn to the bottom left part of the board and said "Oh, I need a rook check here".  I glance up and see the rook and know the solution.  The entire process was a couple of seconds.  More importantly, I solved this problem on pass 6 faster than I had on any other pass of this specific problem.  Most of my other passes took me in the 20 to 30 second range to see the answer.  This time it was under 10. The reason it even took that long was because I actually stopped to marvel at my thought process and made a mental note to blog about this.  Having done similar problems of this type in other sets drew me to the right answer almost immediately.  Of course, this isn't really saying anything since we probably agree that puzzle training makes you better at puzzles.  It's like the idea that football drills make you better at the drill, but not necessarily playing football.  But I definitely "feel" like my solution is coming from a different place.  It feels exactly like I do at work when someone comes to me with a troubleshooting problem and I instinctively know where to look for the solution.

So on to my next example.  I was randomly looking around the blogosphere and came across this position of a missed tactic from The Back Rank.

White to move

I recognized the idea of Bh6 right away. Even in my high school days I knew that pattern well, along with the dreaded knight fork.  Of course the d4 bishop makes it all moot, but I saw the idea of sacrificing the exchange for a pawn and an attack on the king.  After about 15 seconds I didn't see a forced mate or material advantage so I moved on with reading the rest of his post.  As a curiosity I put the position in Houdini and had it play through a few moves to get an idea of technique.  After playing around a bit with some lines I eventually ended up in a position that looked something like this (from memory so might not be 100%):

White to move

Twenty years ago, my thought process would've been something like this, at best: 
"I'm up material so I want to trade queens."  From there I probably would've looked at a dozen different queen and rook moves trying to force a queen exchange or check mate.  I figure there would be about 3 equally likely outcomes to this game.
  1. He drops the queen or trades it.
  2. I drop my queen or rook
  3. I get back rank mated
Rather grim since 2 of the scenarios are bad for me.  I turn a win into a draw or loss.  #2 certainly wasn't without precedent.  I remember playing in a team tournament and going into what probably was a drawn rook and pawn endgame.  I lost my rook to a skewer and eventually the game.

My thought process was a lot simpler this time:
"Qe1 drops my rook but is a forced tradeoff of his remaining heavy piece.  The King and Pawn endgame is a win for me 100,000 times out of 100,000".
Qe1 was the 1st move I thought of and it was almost instantaneous.  I knew that I wanted to play that right away.  Houdini says I have mate in 11 if I start with c5, but why bother with the complications?

I'd venture to say the inspiration for that move came from Pandolfini's Endgame Course:

White to move

The solution is Qf8+ followed by the exchange on d6.  After the trade, white wins easily using the outflanking concept. No telling what I would've done 20 years ago in this position or how long I would've taken me to do it.  Regardless, these are all positions that you could see between two class players OTB and these are both positions I could win now with less than a minute on my clock.


  1. If you repeat the same set of problems over and over again and you can solve these problems quicker and quicker (with the same high score) than you improve "on this set of problems", no doubt.
    But did you improve OTB? or even "only" in tactics? is not that easy to tell.
    I made a training in "mate" problems. I tackled tenthousands of mate problems. I did speed up on them; even on mate problems i never did see before!! But i did get slower at "non-mate-puzzles" at the same time
    Acording to your thoughtprocess ( chaotic or not ;) you put you attention to different topics like "Kingsafty" sooner or later . After doing many Mate problems you will put your attention to kingsafty always soon and so you will solve mateproblems quicker. But you will look for Knightforks later.. so you will get slower = worse in Knightforks.

    Same with tactics and otb: a lot of tacticstraining will make you alert for tactics but your positional play or your strategical play may ( may ! ) suffer.

    We "know" from Bright Knights (Empirical rabbit) experiments, that a training on some puzzles of a book >has< effects on the performance of puzzles of this book which have been not tackled during the training. But ... The tactics puzzle books of Fred Renfeld are full of Queensacs in move 1. So if you do a training on 900 puzzles of one of his 1001... books and then start solving the 101 other puzzles you will automatically expect a lot of queensacs in move #1, you will find them quicker... But someone who is looking for queensacs at every move... is usually not a better chessplayer ;)

    I am shure that Bright Knight / Dan Heismans method works , especially at "low" rated tacticians, but it would be nice to see/prove that.

    Chesstempo and the chess tactics server are free of charge. If you could improve your rating there that would be a great hint.
    I think a good method to show a positive effect of this method would be something like this:
    you do 1000 chesstempo puzzles then you learn a book by bright knights method then again 1000 puzzles at chesstemp then learning the next book and so forth.

    By the way, what are you doing first when you see a puzzle? Counting material? Looking for Kingsafty? or checking for unprotected pieces? <- dont forget to do the awareness test ;)

    1. I agree, a test under controlled circumstances would be ideal. I don't know about 1000 problems though as that could take a few weeks to go through. I figure do a 100 a day until you stop improving. It could be 300 it could be 1000.

      My original idea was to find a tactics book that contained a graded "final exam" and just take that. Then move onto another like that after each set of Rabbit books. But I don't know of too many books like that. Seirawan's Winning Chess Tactics has a 45 problem test at the end that I could do but that would be the extent of it. Which makes your tactics server idea more appealing.

      As far as what I do first when I see a puzzle? I never count material. The positions in Bain where you have to draw by perpetual check are obvious. I thought about making notes in the flashcard if I was down material since I'd 100% know that in a game, but I haven't seen the need yet. Since I'm going for fast solution times, I don't do any type of methodical analysis really. It's also hard to capture and verbalize the process since it's happening quickly and (hopefully) subconsciously. I'd say I do a quick scan of the board to see what catches my eye and go from there. A rook on a8 and b7 makes me start looking for my bishop. A bunch of pieces pointing at the king and I go right into looking for a forced mate. A knight in the middle of the board and I'm looking for forks. If I don't see it, that's when I start methodically looking for themes, checking the corners of the boards to make sure I'm not missing any pieces, etc.

      I have no idea if that's the "right" way to do things, but that's how I'm accomplishing it now.

  2. Welcome to the chess improvement blog community!

    It looks like you're off to a good start on the tactical and other problem studies. I think part of what AoxomoxoA is getting at with his comment above, similar to a recent post of his, is that they can result in one-dimensional improvement of chess skills.

    Do you have a plan to incorporate OTB games (or online equivalents) into your training program, to help translate your skills into a "real-world" environment? Or are you content being a puzzle solver? Nothing wrong with that, as some people prefer puzzles as their hobby, rather than playing actual games. You mentioned earlier that you planned to wait to take this step, but I'd encourage you to incorporate it sooner rather than later. If you can avoid ego issues and look on losses with equanimity as useful for study/analytical purposes, it should provide you a training boost, although sometimes that's easier said than done.

    Good luck and good chess!

    1. I plan on a balanced training program so as not to be one dimensional. Focusing on tactics and endgames to start, followed by strategy and an opening repertoire later. I plan to start playing online games soon. AoxomoxoA mentioned Lucas Chess in one of his other comments and it looks like a nice training program since I can play against weaker engines and work my way up. And you're right about the ego issues. I just hated the old way I used to play, which was basically move to move with no strategy and no idea how to form a strategy whatsoever. I just want to cover some fundamentals before I start playing again. I absolutely recognize the value of practice to solidify the theory. I can't go 6 months just reading books and never pushing a pawn in a real game and expect an improvement.

      I did join the DHLC over at and I'll be playing in their qualifying tournament next week, I also want to start getting in a 30/30 or 45/45 game in at least once every couple of days starting maybe in the next few weeks. I'll need a few recent games under my belt to take to a chess coach eventually.

  3. Thank you for sharing valuable information. Nice post. I enjoyed reading this post. The whole blog is very nice found some good stuff and good information here Thanks..Also visit my page chess training .