How about a nice game of chess?

This is my 1st game of serious chess since embarking on my training program. It was a bit scary going in since I decided to completely scrap my old opening repertoire in favor of openings I know barely anything about. I've always been an e4 player but I've decided to switch to d4. Also dumping the Sicilian for now.

Lessons I've learned:

1. Being safe when trying to win a won game is a good thing, but don't go completely into a shell.  Be more aggressive with trading off those pieces.
2. Keep the tactical radar on at all times.  Don't get lazy and fail to calculate hard moves in a won game.
3. Keep pawns on the board when up material!  We all know this, but I didn't follow it like I should have.  Those general rules  exist for a reason.

Torsten Mattsson - Basic Checkmate Patterns

Santa Claus came to my door and gave me some presents that had been on my wishlist.  I'm starting to think that our idea of Santa Claus is wrong because he was driving a brown truck and the boxes said they were from the Amazon.   It makes sense that an old man like him would want to be somewhere warmer than the North Pole though.

It seems like there's a lot of defunct chess improvement blogs out there, especially as it relates to the Knights Errant.  I like going through and reading them in one shot to see how their thinking evolved over time.  Plus I always feel like I learn something new from at least one post.

I came across the Chess Vision blog by a Dutch expert who was very close to the master title.  His last post was 4 years ago, so it's possible he's surpassed that by now. He had a post titled "High Level Scanning" which relates back to pattern recognition that I found interesting.

White to move

He then shows two basic mating patterns that someone would have stored to lead them to the forcing sequence that secures the win for White.

"When we start calculating a lot of us will now come up with the following variations:

1 Qh5 gxh5 2 Rg3+ Bg7 3 Rxg7+ Kf8 4 Rxh7 (see pattern 1)

1 Qh5 gxh5 2 Rg3+ Bg7 3 Rxg7+ Kh8 4 Rg3 (see pattern 2)

1 Qh5 h6 2 Qxh6 Bxh6 3 Rxh6 (see pattern 1)

Unfortunately however there is that darn Queen on c5 preventing this winning combination (1 Qh5? Qxh5). But a simple deflection does the trick here. After 1 b4! Qxb4 we are back to the variations given above.

This is what high level scanning should bring about. Focal gamma bursts that retrieve the right patterns and that make calculating look really easy! But to lift your scanning skills enough to reach this level, they have to be build up step by step."

Which brings me back to the title of this post.  I need to increase my store of basic mating patterns.  Bain has some good mates in it, but I've found that when trying to solve random "mate in X" diagrams on the web, I'm not particularly good at them.  Qh5 is certainly a move I would reject initially while trying to force this position into my other patterns.

The Mattson book only has a Kindle version, but it's $2.99.  He wrote the book for his son to help him learn mates greater than 1 move.  It contains 63 "patterns" which are broken down by attacking piece grouping IE Q+B, Q+N, R+B, etc.  Each pattern has 1-6 "mate in X" problems associated with it.  The problem comes from a game and the full PGN is listed in the solution.  The 1st problem will be a mate in 1, the next problem will be mate in 2 all from the same game.  So you learn to reverse engineer the mate.  The book usually follows a "show a diagram of the pattern" followed by "show the mating combination from the game starting with mate in 1".  Some patterns have more than one diagram associated with it, and will have a game for each diagram.  The most amount of problems associated with any one pattern is 6.  There are a total of 204 problems.

And now to tie this back to the 1st part of my post, here are two patterns from his book:

Look familiar? :)

At less than two cents per problem, it seems like a pretty good deal.  The problems should work well with the speed repetition training I'm doing.  Any other good, realistic, checkmate problem books out there?  The three checkmate books Bright Knight reviewed from Heisman's list didn't seem like they were suitable for this type of training. Woolum actually contains 60% mate problems, but I haven't gone through them to see how contrived they are.

And speaking of "reverse engineering" a chess problem, that reminds me of the Ted Talk GM Maurice Ashley gave about retrograde analysis and pattern recognition.

Verifiable Improvement - General Observations

In my last post about my 1st Bain Revision, AoxomoxoA asked the $64,000 question:  How to we verify that this brand of tactical training leads to OTB improvements?  In order to do that, you'd have to set up a longitudinal study that tracked various training methods over multiple years.  2 months of tactics repetitions probably isn't going to result in a huge ratings increase that we can point to and say "a-ha, it works".  I suppose we have lots of data from chess trainers that pump out master players and stress tactics and endgame for improvement.

So here's my 2 cents of anecdotal evidence with a couple of examples.  I've only been doing this for 5 weeks, but I feel like I wanted to get this down in writing so I don't forget to keep observing it later.

I did my 6th and final pass through set A of Bain today.  Here's the final results of each pass in histogram form. I forgot to record the results of pass 4.

Very nice and steady improvement, if you remember my caveat about pass 1 being hand-timed instead of Anki-timed.  I completed 89.06% within 15 seconds with only a 6.25% error rate.  When going through this pass, I felt something different about my process.  The first few passes, a lot of your answers are done right from short-term memory "oh, I got this wrong 2 days ago, the answer is Ne5" followed by a verification of the moves that lead to the solution.  You're reminded of the solution rather than finding the solution. Pass 6 was done 13 days after pass 5.  I noticed I wasn't spitting out the answers from memory on most of the diagrams.  Sure, there are some diagrams that are so unique in this book that you just know "Rh6 wins here" because it doesn't resemble any other pattern in the book.  What I noticed was I was being drawn to certain parts of the board and looking for the right pieces to attack and which ones of mine would do the attacking.  It was more of an instinctual gut feeling.  It feels different than accessing your short term memory for the right answer.

Here's an example:

Black to move

This problem appears in many forms throughout the book with both white and black to move.  When I first saw the problem, I don't remember having any specific memory of it.  I was just drawn to the bottom left part of the board and said "Oh, I need a rook check here".  I glance up and see the rook and know the solution.  The entire process was a couple of seconds.  More importantly, I solved this problem on pass 6 faster than I had on any other pass of this specific problem.  Most of my other passes took me in the 20 to 30 second range to see the answer.  This time it was under 10. The reason it even took that long was because I actually stopped to marvel at my thought process and made a mental note to blog about this.  Having done similar problems of this type in other sets drew me to the right answer almost immediately.  Of course, this isn't really saying anything since we probably agree that puzzle training makes you better at puzzles.  It's like the idea that football drills make you better at the drill, but not necessarily playing football.  But I definitely "feel" like my solution is coming from a different place.  It feels exactly like I do at work when someone comes to me with a troubleshooting problem and I instinctively know where to look for the solution.

So on to my next example.  I was randomly looking around the blogosphere and came across this position of a missed tactic from The Back Rank.

White to move

I recognized the idea of Bh6 right away. Even in my high school days I knew that pattern well, along with the dreaded knight fork.  Of course the d4 bishop makes it all moot, but I saw the idea of sacrificing the exchange for a pawn and an attack on the king.  After about 15 seconds I didn't see a forced mate or material advantage so I moved on with reading the rest of his post.  As a curiosity I put the position in Houdini and had it play through a few moves to get an idea of technique.  After playing around a bit with some lines I eventually ended up in a position that looked something like this (from memory so might not be 100%):

White to move

Twenty years ago, my thought process would've been something like this, at best: 

"I'm up material so I want to trade queens."  From there I probably would've looked at a dozen different queen and rook moves trying to force a queen exchange or check mate.  I figure there would be about 3 equally likely outcomes to this game.

1. He drops the queen or trades it.
2. I drop my queen or rook
3. I get back rank mated

Rather grim since 2 of the scenarios are bad for me.  I turn a win into a draw or loss.  #2 certainly wasn't without precedent.  I remember playing in a team tournament and going into what probably was a drawn rook and pawn endgame.  I lost my rook to a skewer and eventually the game.

My thought process was a lot simpler this time:
"Qe1 drops my rook but is a forced tradeoff of his remaining heavy piece.  The King and Pawn endgame is a win for me 100,000 times out of 100,000".
Qe1 was the 1st move I thought of and it was almost instantaneous.  I knew that I wanted to play that right away.  Houdini says I have mate in 11 if I start with c5, but why bother with the complications?

I'd venture to say the inspiration for that move came from Pandolfini's Endgame Course:

White to move

The solution is Qf8+ followed by the exchange on d6.  After the trade, white wins easily using the outflanking concept. No telling what I would've done 20 years ago in this position or how long I would've taken me to do it.  Regardless, these are all positions that you could see between two class players OTB and these are both positions I could win now with less than a minute on my clock.

Bain Revision 1 (preliminary)

I've done at least 2 reps for all 6 Bain tactics sets.  First, some errata.

After staring at this problem for a minute, I moved on.  Turns out, this should be white to move.  This particular theme occurs multiple times throughout the book and is pretty easy to spot once you know whose move it is.  Overall, the quality of this book is excellent.  Only 1 unusable problem, 1 problem with a better solution and 1 wrong diagram.  I'd say that's pretty good considering some of the other chess books out there.

I sorted the problems the same way Rabbit did in his Bain Experiment.  The only difference was my newer version of the text corrected the issue with problem 95, so I only had to discard 26.  In theory, set F should be the hardest, and my gut says this is true.  I also made a separate category for solution times over than 30 seconds and problems I gave up on after a minute, whereas Rabbit considered anything over 30 seconds as wrong.

Here's the histogram with the stats for my 1st pass through each set:

Two quick notes:  I hand-timed set A.  This resulted in an increased speed over relying on Anki for timing.   Also, I did set F extremely tired.  I should've saved it for the next morning, but I wanted to keep to my schedule.  Even so, my results for 0-5 seconds was better than any other set, and my overall 0-10 seconds was second only to set E.  Overall, it looks like a pretty steady improvement for each set, especially the under 5 and 10-15 categories.

The percentage of problems I completed under 15 seconds also increased, although not steadily.  Heisman recommends being able to do 85% in under 15 seconds.  Here's the percentage of problems I did in under 15 seconds in pass 1.

Set B: 29.23%
Set C: 46.15%
Set D: 33.85%
Set E: 58.46%
Set F: 40%

And finally, here's the percentage of problems I am currently doing under 15 seconds at each set's most recent pass:

Set A (5 passes): 81.25%
Set B (5 passes): 78.46%
Set C (5 passes): 89.23%
Set D (4 passes): 73.85%
Set E (3 passes): 83.08%
Set F (2 passes): 81.54%

I think I'm going to stop all sets at 6 passes which ends up being 52 days of study per set, and 75 days total from start to finish.  I guess a good time to start revision #2 would be 52 days after I finish my last pass?

One final note:  My stats indicate that I am holding steady or improving, even after my 5th pass which is a 13 day interval.  Set A & B pass 6 is on the 12/22 and 12/27.  It should be interesting if this is still true after a 25 day interval.

Up next is Woolum's The Chess Tactics Workbook.  I am scheduled to start that 12/25.

Bruce Pandolfini - Pandolfini's Endgame Course

I bought this book about 15 years ago when I was making my post high school attempt at being a good chess player.  I don't remember how far I actually got in this book, or if I ever even read it. This book contains 239 endgame problems.  The format is 1 diagram and explanation per page with a few exceptions for longer solutions.

There are a lot of typos/mistakes in this book, but the errata is easily found online:
http://www.glennwilson.com/chess/books/pec_errata.html

I downloaded a pgn copy of this book and copied the diagrams to make flashcards.  All the errata was fixed in the pgn except for endgame 137 which I corrected.

Some of the problems are subproblems of well known endgames like the Lucena.  The subproblems walk you through the component parts of the overall solution and finally finishes with the entire problem.  I found this to be an excellent book for my level and a great learning experience.  The format is very compatible with spaced repetitions.

I was not doing speed training with this, although someone with a better background in Endgames may be able to use the book in that fashion.  For me, it was a lot of learning.  Repetition 1 for me is just learning the material and keeping track of which ones I got right.  I didn't keep track of speed, but percentage of correct answers.

I broke the problems down into 4 sets A-D.
I skipped 1-11 as these were elementary checkmates that didn't need any type of revision.
I skipped 113 as it's a problem on Corresponding Squares and doesn't say much other than a general overview of the theory and that there are no simple formulas to calculate the squares in any given position.  Between that and the 14 move solution, I decided to save it for another day.
I skipped 199 since it's a bust.  It's a win for black, not a draw for white.

Set A - Endgames 12-69 : Minor piece checkmates, QvR, RvB, RvN, RBvR and some KPvK endgames.
Set B - Endgames 70-128:  KPvK and KP endgames with multiple pawns on either side.
Set C - Endgames 129-184:  Q/R/B/NvP, KB/NP v K(P), QvRP, QP v RP, QPvQ endgames.
Set D - Endgames 185-239:  RPvR and B/NP v B/N endgames.

I wasn't going for speed, but learning.  I kept track of correct/incorrect answers in my spreadsheet.  If, after the 3rd repetition, I was wrong three times or sometimes two times and struggling on the 3rd, I put the starting position in Arena and forced myself to play the endgame against Houdini until I got it right.  Each endgame type has it's own PGN file and is also on its own repetition schedule.  I'm essentially double studying the ones I'm having trouble learning.  I think it's having an effect and is a good simulation of OTB play since I force myself to play it like a real game.

I've done at least 3 repetitions for each set, so here are some preliminary results:

Set A:

Not bad at all.  Started with 77.2% improving to 91.2% after pass 3.  A small decline to 86% as the repetitions have gotten farther apart.  My only real difficulty with this set was the Philidor RB v R positions.  I didn't start my Houdini repetitions with this until after Pass 5, so I won't know how much of an effect it's happening until 12/24.  From Wikipedia, RBvR is one of the common pawnless endgame, but it's generally a draw (Mednis estimated it as a win about 4% of the time in positions that could actually arrive from a game).  Probably not very practical for converting wins, but very practical in knowing that it's mostly a theoretical draw should I end up a piece down.  Now I have to study the Cochrane defense at some point.  If the position doesn't fit, you must acquit?  It's covered in Howell's Essential Chess Endings, which is next on my endgame reading list.  I've read through Class C in Silman's Complete Endgame Course and haven't come across it.  A quick scan of the contents doesn't suggest that it's listed at all.  I plan to get through Class A before moving on to Howell.

Set B:

Unfortunately, I forgot to record the results of pass 4.  But you can see the same steady improvement.  Pass 1 at 46.6% up to 96.6% by pass 3.  I haven't had any repeating difficulties with this set, which is good news since King and Pawn endgames are so common.  I knew about the opposition, but having it explained in terms of critical squares really helped my learning.  Really like the outflanking maneuvers against locked pawns as well.  I'm at the point when I can see a future position and say "and wins" or "and draws".

Set C:

Another set that I'm not having any difficulty with.  Improvement from 66.1% to 96.6% which has held through the 4th pass.

Set D:

And this is basically the opposite of set C.  Lots of difficulties for me in the RP v R endgames.  A lowly 37% on pass 1 which improved to 63% by pass 3.  I just started my Houdini repetitions for the ones I had problems with.  I will know on 12/20 if it is having an effect.

I don't know how many total repetitions I will do for each set.  My goal is to finish the Silman Endgame book through Class A, make any repetitions as necessary then start on the Howell book.  I don't know what the format of that book is and if it lends itself to repetitions like the Pandolfini book.  If not, my next exercise book is probably going to be Pandolfini's 111 Winning Endgames followed by Alburt & Krogius's Winning Chess Endgames.

Bain Errata

Rabbit listed 2 positions that he threw out as unusable.  Here's the 1st.

Diagram 26: White to move

Bain gives the solution as 1. 0-0 Qxe4 2. Re1.
The tactic relies on a blunder to be successful.  I think it's useful to know that 0-0 is a viable move for white here, but as Silman and Heisman have stressed repeatedly in their writings, assume the best move from your opponent!  The point of that is to not make moves that hurt your position in the hopes that your opponent will fall for a trap.  In this case, 0-0 is a perfectly fine move, but it doesn't fit well with the rest of this tactics book.  Assuming best move for your opponent, 1... Be7 breaking the pin or 1... Nf6 keeping up the pressure on e4 is what I would expect.

He also lists diagram 95 as a dud, but I believe he is using the older edition which is corrected in the 10th edition.  That one has a correct solution.

Which brings me to this.  I've found a superior line to diagram 266!

Diagram 266: White to move

Bain gives the solution as 1. Nf6 Bg7 (1...Nxf5 2.Rh7#) 2. Bxb1 winning the exchange.  On my 1st pass, I had my solution marked as wrong and I didn't question the book at all.  After a short think on my 2nd pass, I came up with what I thought was the winning solution 1. Rh7+ Kg8 2. Nf6+ Kf8 3.Bxb1 winning the rook outright.  Up a rook is better than up the exchange right?

I did a quick verification with Houdini who found an improvement in Bain's line which also wins the rook outright.  1. Nf6 Bg7 2. Rd8+ which forces the dark-squared Bishop off of his threat of the f6-Knight. ... Bf8 3. Bxb1.

So which solution is "better"?  I guess it doesn't matter too much.  When given a choice, Houdini rates Rh7 v Nf6 as +9.55 v +9.17 at 22 ply.  The resulting positions after white wins the rook is given as -9.23 vs -9.02 for black.  Mine doesn't come with the mate threat though, so there's the small downside.  Given the choice between the two now, I'd say I'd rather end up with the same position but having a small chance of my opponent missing the mate.  Mostly I'm just happy I found a solid tactical shot not listed in the book, although mine is more obvious since the moves are all forced.

John Bain - Chess Tactics for Students

Book #1 in my speed training plan.  Empirical Rabbit outlined his plan for this book along with his stats.  388 problems with the goal being 85% correct in 15 seconds or less.  Using Anki for timing is a bit inaccurate as it doesn't stop the timer at the point you click "show answer".  Between that and a quick glance at the answer to make sure I was right, I estimate about 1-2s can be added to timer on certain problems.  It's a small price to pay since my historical times per card are saved.  For this problem set, I downloaded a pdf copy of the book and used the Windows snipping tool to add the diagrams to Anki.  Don't freak out, I own a copy of the book that I purchased legally.  If that breaks some kind of copyright, I don't feel bad.  I paid my money and I'm using his book.

My spaced repetitions will follow this pattern.
Repetition 1: Day 0
Repetition 2: Day 1
Repetition 3: Day 3
Repetition 4: Day 7
Repetition 5: Day 14
Repetition 6: Day 27
Repetition 7: Day 52

So I'm adding 1, 2, 4, 7, 13, 25 to the current date.  On day 5, I start repetition 1 for the next set as well. I use Google Calendar to keep track of when I should be revising a set since the scheduling in Anki can get wonky sometimes.  Word of advice, if you are scheduling repetition 4 days away, but you have a choice of 3 or 5 in the software, take 3.  It's easier to just wait until the 4th day when all cards become available.  Otherwise you have to go into the deck and reschedule the ones you need for today.  Not a huge hassle, but something to be aware of.  I have not finished this book yet as I just did repetition 2 for Set E today.  In 4 days I will have done all of the problems in this book at least once.  I am not as fast as Rabbit is with his training, but from what I gather from his posts, I spend a little more time on the problem verifying alternate defenses and I don't give myself partial credit if I got the first couple moves correct but didn't get the entire solution.  OTB you aren't trying to solve the problem in 15 seconds and you have time to catch a small mistake, but I'd rather be hard on myself in practice than have my opponent be harder on me OTB.

I seem to be showing steady improvement in sets I haven't trained yet, much like Rabbit.  My Set 1 Rep 1 times are throwing things off since I hand timed those and my results were much better than any other reps I've done since.  It wasn't until set C that I was able to come close to my results in set A, and set E before I was able to pass it.  But excluding that, things look good.

The training plan

In my day job, I'm a Network Engineer.  I also attained my CCIE certification last year, and for those who don't know (most of you), it's a very prestigious, practical lab exam designed by Cisco.  It takes thousands of pages of reading and lab hours in order to pass.  Most don't pass their first try.  The average is the 3rd-4th attempt.  Many start down the road and never finish.  What does this have to do with chess?  I'm hoping the lessons I learned about studying can carry over to chess.  Just like chess, there's a lot of theory that has to be memorized and that is reinforced with practical application of the theory.  The difference is you're at a command line and not a chess board.  It was during my CCIE training that I learned about the  spaced repetition learning technique.  A few google searches later and I found that other people were trying to apply this to their chess as well!

I've read all of Emprical Rabbit's entries on his blog and I will be adapting what he's learned into my study methods.  I've also read Dan Heisman's site and others for book suggestions.  I've broken down my studies into six categories:

1) Tactics
2) Endgames
3) Instructive Anthologies
4) Middlegame/Strategy
5) Pawn Structures
6) Openings


I'm still trying to figure this out as I go.  At a high level, I want to constantly be going through at least 1 book on tactics that focuses on easier problems that can be done for speed training.  Once I get a few of those under my belt, I will add a concurrent book that focuses on longer exercises that won't be done for speed.
Heisman suggests there are 2,000 basic tactics patterns to learn.  I don't know how long this will take, but I've set a short term goal of going through 10,000 speed training tactics problems not including the repetitions.  That will probably end up being about 20 tactics books.  I have 11 on my short list right now, so I just have to find 9 more.

For endgames I want to study an endgame strategy book and an endgame tactics workbook to reinforce the strategy with spaced repetitions. I don't have any goals for amount of endgame tactics to go through, but I want to constantly be doing them.

I plan on doing spaced repetition for strategy as well, but it won't be timed.  My idea is to take selected positions as the question and make the author's evaluation/key points as the answer.  I've gone through about half of Silman's The Amateur's Mind, and answers in the form of minor pieces, pawn structure, file/square control, space, material, initiative and development along with plans for BOTH SIDES and an overall strategy based on the evaluation seems like a good idea.  I was never one to really analyze a position OTB.  I didn't know how to.  Mostly it was just tactics, making sure he wasn't threatening a back rank mate and trying to find holes in the pawn structure to place a Knight.  If nothing else, this will get me in the habit of evaluating a position and not playing HOPE chess.

For the Instructive Anthologies, it was suggested to go over 10,000 master games.  At around 30-50 games per book, that's going to take a while.  Of course, going over annotated games in other books counts too.  But that's a lot of games to go over.  I think 2,000 would be a nice start.

For the spaced repetitions, my weapon of choice is Anki.  It's free and has a version for Android.  I've noticed iPads are pretty popular in the chess improvement blogosphere, but I like my Nexus 7.  Even better, there's the FEN chess visualizer addon that does exactly what you think it does.  It takes FEN and displays a diagram for a flashcard.  Not the prettiest chess diagram ever, but it works.

It's not easy to get OTB games where I live, so I will rely on chess.com and others to get my games in.  But I don't plan on playing any games for a little while longer until I feel like I have a good grounding in basic theory.  I also plan to get a tutor if I end up sticking with this.

Yet another chess improvement blog

I had a very short and uneventful chess career about 20 years ago.  I learned the moves as a kid and played occasionally but nothing serious until I got to high school where I joined our chess club.  I had pretty mediocre results and finished with about 20 rated games as a Class E player.  Like a lot of new players, I pretty much had no idea what I was doing or how to learn what I was doing.  The strongest player on our team was about 1600 strength.

We had a copy of Modern Chess Openings so I set out to memorize that since I was terribly afraid of going wrong in the opening and losing.  To choose my opening repertoire, I did the most logical thing possible; looked at what current World Champ Garry Kasparov was playing.  He was the champ so they had to be the best openings right?  So I set out to memorize the Ruy Lopez, King's Indian and Sicilian Defense.  This worked out about as well as you would expect.

After HS, I bought a few chess books and tried to study.  The Ideas Behind the Chess Openings, Pandolfini's Endgame Course, Winning Chess Tactics, Winning Chess Strategy & Comprehensive Chess Course Vol I & II.  The Alburt books was were I learned I should be concentrating on tactics and endgames, not memorizing openings.  So I had a better idea of what I should be studying, but not how to study and retain the information I had learned.  After another tournament of mediocre results against other Class E players, I stopped playing.

So here I am.  For those who don't know, the blog name is a reference to the classic Saturday Night Live skit featuring Jim Belushi doing the Bobby Knight version of a chess coach.  Unfortunately the entire clip doesn't seem to be available online, but you get a good idea what it was about.