CPT Community

Innovative Chess Training
Welcome to CPT Community Sign in | Join | Help
in Search

English Blog

In this blog you will find news and information about Chess Position Trainer - current und future versions.

CPT 4 challenges, "small issues" and sleepless nights

Someone wrote in the forum about a seemingly small glitch of the training center which under the surface is a pretty complex topic which kept me busy quite some time while I worked on the design of CPT 4. I have to admit this is a rather dry topic if you are not into it Storm

Background info 

First you might want to read his post.

In short: If you train with the option "always jump to worst position" it is possible that CPT will present you positions which are not reachable by a top-ranked candidate move in your sub-repertoire. This is a training setting for advanced players which know their repertoire pretty good. Basically the difference is that you will not be lead to those positions from the start position on, but instead just presented the final position - the currently worst position of your sub-repertoire. A great way to really recognize a position and not move orders.

I'm aware of this limitation for quite some time. Unfortunately it is not so easy to improve. Actually a similiar issue exists for training of tactical positions or the calculation of the sub-repertoire overall training score. Here again CPT should ignore those positions which are not reachable by top-ranked candidate moves or in the case of training of tactical positions CPT should pick the start position and not any / worst of the (usually short) line. The latter has some additional challenges which are not covered by this post, but by CPT 4 Cool

The Problem

At the moment CPT can't tell whether a certain position is only reachable by a non-top ranked candidate move or not. 

Two possible approaches 

There are two approaches to solve the problem. Either calculate this information "on-the-fly" when CPT needs it (for example during the training session) or pre-calculate it for each position and save the info.

Believe me, for CPT 4 I spent hours, well, overall days about just this topic and how to design and develop the "perfect" solution. For several times I thought I had covered all scenarios and I was pretty sure I found the solution - just to find out - no, not yet. I became almost mad.

OK, let's have a look at both possible approaches to this challenge.

Calculation on the fly

You quickly run into a performance issue. First you would just calculate the worst position. Then you would need to check whether the position is reachable by a top-ranked candidate move / line. If you have more and more of such non-top ranked positions over time the performance will (significant) decrease (just imagine to check 1000 positions and their preceding moves before you finally find a "right" one...). There is even a bad side-effect that those non relevant positions will be always checked first as they are never trained and remain at a training score of 0%. I don't believe in this approach.

Pre-Calculating

Obviously this solves the performance issue. However, now things get really tricky if you start to think about the details. I encountered the same concepual issues as I did with the idea of an extended (sub-)repertoire concept.

You will need to check for each new move you add to the repertoire whether the new arising position is now a valid position for training or not (or for the repertoire training score or a sub-repertoire). So far so good. Thanks god there exist transposition in chess.

Imagine to reach an already existing position which was so far NOT a valid position for training. However, now it is reachable by a top-ranked candidate move. The logical consequence would be to change the flag for the position and all consecutively positions which are reachable from that position as "valid". Vice-versa if you delete a move. Or think about what should happen if you manually change the order and thus the rank of a candidate move...

Also consider more complex actions like an import of thousands of games or a copy of a sub-repertoire with >10.000 positions. In these cases CPT would need to check for each position whether it is a valid position for training or by transposition any already existing position (and their consecutive positions) should change their flag now. This will lead to a huge performance penalty for all these actions.

Maybe I'm just too pessimistic (I'm usually not), but I believe there are good chances that somehow this information gets messed up. Then a non valid position will be marked as valid and vice-versa. Good chances for "chaos" here.

New approach 

CPT 4 will not care for the info "valid" position for training or not while you add a new move / position (either by manual editing, import or copy / paste). Instead it will offer you a function to start a batch job which just goes through all branches and makes sure the flag is correctly set for each position. Take into account that you only have to run it once you have added a new move for your side which is not a top-ranked candidate move. If you always only add one candidate move for your side and many for the opponent you don't have to care at all!

I don't like the aspect that you have to manually start this job as there are good chances some users will miss it. However, those will likely not bother anyway and I believe it is still - after consideration of all CON's and PRO's - the best approach. Sometimes there just doesn't exist the goose that lays golden eggs.

Well, who knows. This apparantly simple and minor issue cost me so many hours and sleep, maybe I will change my mind once again Stick out tongue 

Cheers, 

Stefan

Published Mittwoch, 19. September 2007 08:29 by Stefan Renzewitz

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

Xenix said:

Hi Stefan,

I really think the new approach is the best. Maybe you can set some kind of flag if a user adds a non-top-ranked move. Then you can execute the batch job only if this flag is set when the user enters the position-training-mode.

Kind regards

September 19, 2007 10:07
 

zenchess said:

Hmm, I do not really understand why this is such a big problem.  Couldn't you solve it simply by the following method:

Make an array (collection, container, whatever).  When the user enters moves for the first time put all positions in the array.  This array will hold all the valid positions that can be accessed by the 'jump to worst position' procedure.  

When the user enters a secondary not-top-candidate move, do not enter any of the resulting positions into the array.  When it comes time to jump to a random 'worst' position, simply use those from the array, and since the 'bad' positions didn't ever get put in the array, no problem.  

Sorry if you already addressed this idea as I didn't completely understand the ideas you had above.  

September 20, 2007 08:26
 

Stefan Renzewitz said:

You have to make the data persistant at some point and you will certainly extend it over time. I could mark each position as either "valid" or not valid (for training) and easily query it. However, the problems I have described above will occur soon or later. In your example: if you add a top  candidate move to your first array and it leads to an already existing position of the second arreay which was so far only reachable by a secondary move you would need to update all already existing suceeding moves and positions.

September 20, 2007 22:00
 

Bill Enriquez said:

I think that it is best to utilize 'position evaluation' there. If, for example, you are training for white and all positions that has evaluation with +-, +=, =, etc. can be safely presented by CPT for training  with the option "always jump to worst position".

September 21, 2007 09:02
 

Stefan Renzewitz said:

Thanks for your comment. You are right, it feels natural to do it that way and the backsolving function already exists to calculate this info for each position. However, then you would need to either set the position evaluation for each leaf node or you have to do so for each new position which should not be considered.

If we keep a separate field for this info we might be more flexible. For example the user could select from some criterias how to define a position as "valid". As this info doesn't get mixed up with the position evaluation I can dream of things like only mark white best moves and all black moves without negative position evaluations or moves as "valid". To make the selection based on the position evaluation would be just one option, but I could easily extend the criteria list later on.

September 21, 2007 12:30
 

bulldoggy said:

Seems like this would be an issue in trees that have sub-optimal moves.  If this is selected, could you prune the tree so you eliminate sub-optimal moves and save it in some kind of temporary storage?  Then you don't have to worry about whether or not positions are reachable only by sub-optimal moves.  There are none in the tree.

September 23, 2007 04:34
 

Stefan Renzewitz said:

I'm not sure if I understand you right. You would create the tree before the training starts? To create the tree for each training session would take quite some time depending on someone's repetoire (>10.000 positions). Would this be any better than having a function which does the same explicitly and then makes this information persistent like the suggested approach? Btw. a normal tree doesn't work for chess as you can have more than one father. I'm using a classical database system for CPT 4. To query for a flag is no perfomance issue.

September 23, 2007 13:21

Leave a Comment

(required) 
(optional)
(required) 
Submit