@petri999 this is for a training App, to bridge the gap between the computer and human, working without these characteristics is impossible unless we have AGI. Which is still a few years away and needs supercomputers.
#16 - but these measures feel so blunt that I don't think you'd have to be particularly good at chess to get more insight yourself just by playing back through the game. I can take a reasonable stab at spotting for myself whether I control more territory than the opponent and if so whether I've actually got a useful advantage in space or whether I'm actually overextending and creating weaknesses. I'm a total patzer so I'm often wrong, but I'd put money on an app that just tots up points based on how advanced pawns are being right less often even than me.
@RamblinDave I will only publish the App if it helps me to improve my game. I have gotten so much pushback here, but I am motivated to find out about these characteristics and I am certain that knowing them will help me. I am a very positional player, I do not calculate a lot over the board. But I do analyse pawn structure, mobility, space, etc. In the end I am working on something, instead of saying its not possible, dont try it.
I also realise that this forum is not the right place to ask my questions.
I will try chess.stackexchange.com/
I also realise that this forum is not the right place to ask my questions.
I will try chess.stackexchange.com/
@RichardRahl - yeah, sorry not to be part of a more encouraging response, I think the ideas that you're looking at just seem quite alien to the way that a lot of people think about improving their chess. Personally, the way that I could imagine computer aided self-analysis progressing in the future would be more focused on extracting useful summaries from actual computer evaluations of your own games and trying to point you to specific things (structures, positional mistakes, tactical motifs etc) that you tend to get wrong - do you frequently miss sacrificial combinations, say, or are you too willing to exchange bishops for knights, or too reticent to make pawn breaks in Ruy Lopez structures or something. This seems more promising than this idea of looking at your own games using only very blunt metrics.
Anyway, I hope it works out for you!
Anyway, I hope it works out for you!
@RamblinDave thank you, you have some insights there:
do you frequently miss sacrificial combinations, say, or are you too willing to exchange bishops for knights, or too reticent to make pawn breaks in Ruy Lopez structures or something.
thats where I eventually want to come to. But first the ground work needs to be done.
do you frequently miss sacrificial combinations, say, or are you too willing to exchange bishops for knights, or too reticent to make pawn breaks in Ruy Lopez structures or something.
thats where I eventually want to come to. But first the ground work needs to be done.
Found a pretty neat solution.
Take any FEN position.
1.Remove all enemies pieces.
2.Remove everything exept the pawns.
3.Position the kings on the players first rank in the opposite corners. That way we have a legal chess position.
8/8/8/8/8/8/PPPPPPPP/K6k w - - 0 1
4. fill the board up
The last rank fill it up with knights, to prevent any possible checks on the kings.
All other ranks can be filled up with pawns. The first and second rank of the player are not filled up with pawns, since there can never be a pawn capture from the player. (Pawns start at rank 2)
This is convinient because this leaves the the kings protected.
nnnnnnnn/pppppppp/pppppppp/pppppppp/pppppppp/pppppppp/PPPPPPPP/K6k w - - 0 1
Get the legal moves:
0: "axb3"
1: "bxa3"
2: "bxc3"
3: "cxb3"
4: "cxd3"
5: "dxc3"
6: "dxe3"
7: "exd3"
8: "exf3"
9: "fxe3"
10: "fxg3"
11: "gxf3"
12: "gxh3"
13: "hxg3"
14: "Kb1"
Remove the moves from the King.
Now we have all sqares our pawns can attack. Without writing custom code to check legal moves. Which is great!
(en passant is not included, but also not really part of the packing density, as well as pretty rare overall)
Now if we also want to include the Pieces, we take the original FEN position. Get all legal moves, remove all pawn moves.
The only moves missing are moves where we could capture our own pieces/pawns. I.e protected squares.
Hmm, maybe I need to do multiple iterations now, replace one piece with multiple different pieces and generate a set of legal moves. (no duplicates allowed) That way we go around the check/pinning issue.
Is there a better solution?
Take any FEN position.
1.Remove all enemies pieces.
2.Remove everything exept the pawns.
3.Position the kings on the players first rank in the opposite corners. That way we have a legal chess position.
8/8/8/8/8/8/PPPPPPPP/K6k w - - 0 1
4. fill the board up
The last rank fill it up with knights, to prevent any possible checks on the kings.
All other ranks can be filled up with pawns. The first and second rank of the player are not filled up with pawns, since there can never be a pawn capture from the player. (Pawns start at rank 2)
This is convinient because this leaves the the kings protected.
nnnnnnnn/pppppppp/pppppppp/pppppppp/pppppppp/pppppppp/PPPPPPPP/K6k w - - 0 1
Get the legal moves:
0: "axb3"
1: "bxa3"
2: "bxc3"
3: "cxb3"
4: "cxd3"
5: "dxc3"
6: "dxe3"
7: "exd3"
8: "exf3"
9: "fxe3"
10: "fxg3"
11: "gxf3"
12: "gxh3"
13: "hxg3"
14: "Kb1"
Remove the moves from the King.
Now we have all sqares our pawns can attack. Without writing custom code to check legal moves. Which is great!
(en passant is not included, but also not really part of the packing density, as well as pretty rare overall)
Now if we also want to include the Pieces, we take the original FEN position. Get all legal moves, remove all pawn moves.
The only moves missing are moves where we could capture our own pieces/pawns. I.e protected squares.
Hmm, maybe I need to do multiple iterations now, replace one piece with multiple different pieces and generate a set of legal moves. (no duplicates allowed) That way we go around the check/pinning issue.
Is there a better solution?
some general comments:
There seems to be more than one possible motivation in favor of the op exercise.
Comparing relative importance of some human conceivable constructed metrics (isolating concepts of various category levels), while the formula may be of human intuition or basis on some rational positional arguments, their measurement in one individuals own game, for decision guidance, may not be directly measurable by human. However, systematically estimating using few parameter type of statistics, such isolated factors, and others that are apparently bunched up to help improve AB engines evaluation functions, may help in figuring out some kind of ordering among many of those conceived factors.
find some principal components maybe, using databases, conditioned on player pair averages or even the pair of levels. That would be the first step. Currently, I think these are combined in some way in SF evaluation function, and as I understand, the many parameters associated to each component in the combination are estimated via some non-linear global stochastic optimization or auto-tuning based on some battery of test positions (sounds like training to me). Most probably at this stage, the individual component estimates are not-distinguishable statistically, in the same manner that people called NN black-box. the input-output, may be the same, but the exact vector of component coefficients can vary (this is typical of many such many parameter global optimization methods).
I don't know how much has been done in this direction, with some transparency and reproducible, possibly peer-reviewed articles.... but it seems that the background data work involved behind the maia technology (in the paper), might have some interesting things , besides the main focus of maia engines. I really like figure 11. as some measure of the quality of lichess database...
One can't reject one isolated factor though for human theory consideration because it can't be measured in-game. So having a many-games view of, usually qualitatively expressed, in some checklist, principles or features, and using statistics to find some ordering of them, I think might end up contributing to theoretical understanding of chess games , down to the individual game being played. I find (impression) that the problem of combining those qualitative factors, for in-game decision is not well studied.
Earlier genuine attempt to start assigning to contribution to them combined (in such book as PCC) toward a decision making analysis during game rather than post-game (where one can omit combining them by picking their preferred one --not saying this is the case, but no method to decide to my current knowledge, one has to build intuition on how to even apply those constructs. Bad sentence. i meant such attempts have been derided, and reviewers only keep the isolated component as valuable from there, as qualitative non-co-mensurable factors.
But does it have to be so? Are they all equally important? Some of them are called principles, but how do we know if no other feature could not be promoted to principle. and among the principles, if consensus in their definition, what are the relative importance given a position for analysis the move candidates, in-game?
The op exercise in a first step. But I doubt that there is one magic bullet of such constructed rational measure that alone will help analyse a position and make a move decision (not talking about hindsight here). Doing statistics that way is also hindsight but having a population of games allows for quantitative attempts to handle at least how such concept based measure (few concepts at a time) relate to other possible rational constructs. In the same manner that, now accepted, positional analysis arguments are used within-game prior for move decision, but were originally conceived in hindsight. This is how we make progress. and i don't see why such experiments as op suggest. should not even be tried.
I think many features need to be tested that way. and such experiments would benefit chess science, engines, and even individual game improvement in move selection decision making. That would be made possible by a solidification of how the many arguments or analysis elements we can and have considered do combine at any position. I wish to support the op in his set of research questions. But beware of reinventing the wheel. there may already be many features that would also benefit from the same database statistics measurements that have been proposed here for one such small combination of features or principles ( i think here one would think of space or mobility, although there is some material aspect involved, depending on the formula). Also i wish to mention that the relative importance may not have to be of the linear combination of the components. as there are overlaps, and possible positions subsets where some of those might actually conflict. composition of such factors and non-linear small combinations might also be best suited (packing density seem to be an example of, in terms of material being blended with space notions, maybe). Some concepts might also be nested. An iterative process between measurements, and rational construction of such limited combinations/compositions might be able to make progress even for human decision making analysis.
sorry . for long winded. this has been keeping me awake for a while now. and I am putting it all out. ready for tomatoes, or constructive comments. hopefully this will give some help to op, moral at least. also, there are many ways i could have made wrong assumptions. so if you spot some, please share. also facts are welcome.
There seems to be more than one possible motivation in favor of the op exercise.
Comparing relative importance of some human conceivable constructed metrics (isolating concepts of various category levels), while the formula may be of human intuition or basis on some rational positional arguments, their measurement in one individuals own game, for decision guidance, may not be directly measurable by human. However, systematically estimating using few parameter type of statistics, such isolated factors, and others that are apparently bunched up to help improve AB engines evaluation functions, may help in figuring out some kind of ordering among many of those conceived factors.
find some principal components maybe, using databases, conditioned on player pair averages or even the pair of levels. That would be the first step. Currently, I think these are combined in some way in SF evaluation function, and as I understand, the many parameters associated to each component in the combination are estimated via some non-linear global stochastic optimization or auto-tuning based on some battery of test positions (sounds like training to me). Most probably at this stage, the individual component estimates are not-distinguishable statistically, in the same manner that people called NN black-box. the input-output, may be the same, but the exact vector of component coefficients can vary (this is typical of many such many parameter global optimization methods).
I don't know how much has been done in this direction, with some transparency and reproducible, possibly peer-reviewed articles.... but it seems that the background data work involved behind the maia technology (in the paper), might have some interesting things , besides the main focus of maia engines. I really like figure 11. as some measure of the quality of lichess database...
One can't reject one isolated factor though for human theory consideration because it can't be measured in-game. So having a many-games view of, usually qualitatively expressed, in some checklist, principles or features, and using statistics to find some ordering of them, I think might end up contributing to theoretical understanding of chess games , down to the individual game being played. I find (impression) that the problem of combining those qualitative factors, for in-game decision is not well studied.
Earlier genuine attempt to start assigning to contribution to them combined (in such book as PCC) toward a decision making analysis during game rather than post-game (where one can omit combining them by picking their preferred one --not saying this is the case, but no method to decide to my current knowledge, one has to build intuition on how to even apply those constructs. Bad sentence. i meant such attempts have been derided, and reviewers only keep the isolated component as valuable from there, as qualitative non-co-mensurable factors.
But does it have to be so? Are they all equally important? Some of them are called principles, but how do we know if no other feature could not be promoted to principle. and among the principles, if consensus in their definition, what are the relative importance given a position for analysis the move candidates, in-game?
The op exercise in a first step. But I doubt that there is one magic bullet of such constructed rational measure that alone will help analyse a position and make a move decision (not talking about hindsight here). Doing statistics that way is also hindsight but having a population of games allows for quantitative attempts to handle at least how such concept based measure (few concepts at a time) relate to other possible rational constructs. In the same manner that, now accepted, positional analysis arguments are used within-game prior for move decision, but were originally conceived in hindsight. This is how we make progress. and i don't see why such experiments as op suggest. should not even be tried.
I think many features need to be tested that way. and such experiments would benefit chess science, engines, and even individual game improvement in move selection decision making. That would be made possible by a solidification of how the many arguments or analysis elements we can and have considered do combine at any position. I wish to support the op in his set of research questions. But beware of reinventing the wheel. there may already be many features that would also benefit from the same database statistics measurements that have been proposed here for one such small combination of features or principles ( i think here one would think of space or mobility, although there is some material aspect involved, depending on the formula). Also i wish to mention that the relative importance may not have to be of the linear combination of the components. as there are overlaps, and possible positions subsets where some of those might actually conflict. composition of such factors and non-linear small combinations might also be best suited (packing density seem to be an example of, in terms of material being blended with space notions, maybe). Some concepts might also be nested. An iterative process between measurements, and rational construction of such limited combinations/compositions might be able to make progress even for human decision making analysis.
sorry . for long winded. this has been keeping me awake for a while now. and I am putting it all out. ready for tomatoes, or constructive comments. hopefully this will give some help to op, moral at least. also, there are many ways i could have made wrong assumptions. so if you spot some, please share. also facts are welcome.
Obvisusly someone has trod the uncharted ways to get anywhere new. But I am quite sure that the Shasin evaluation is not extremely good. For starters it consist of formulas that are hard to calculate. As OP send he calculates mobility: THat is well tested concept. One could argues that even material is mere proxy for mobility. Space definately is mere proxy for mobility but easier to use by humans. And well tested. there is no chess engine that does not estimate mobility. Also piece position is important and just about every single program has piece-square tables to given estimate on long term viability of having piece on certain square.
So spending time on trying mimic algorithmic procedure that might be useless might be bad use of time. BUT it is not wasted effort even if no gains in chess skills materialize given that programming and algorithm design are desirables skill and working out optimal way computing these does help on development/refinement those skills.
But also OP is unlikely to find someone interested on this particular puzzle
So spending time on trying mimic algorithmic procedure that might be useless might be bad use of time. BUT it is not wasted effort even if no gains in chess skills materialize given that programming and algorithm design are desirables skill and working out optimal way computing these does help on development/refinement those skills.
But also OP is unlikely to find someone interested on this particular puzzle
@dboing you have a good notion of what I am trying to do. For me it is about providing analysis of the latent features during the game, for each move. BUT NOT of the current position (although that might be usefull to add later) instead I show the aggregated parameters of all master games (or top engine games). Top level engine games are a kind of knowledge base or wisdom of the crowd.
This means for me it is important that the parameter, like mobility, expansion etc can be meaningful described in a sentence so that the player can understand it, but might not know the formular. Same as some people know what a standard deviation is, without having the formula.
I have little use for a parameter that can not translated into human speech, if it consists of to many rules/connections it is not a true principle that we can use in play i.e then its not a principal component.
So far I came up with (I found the ideas online):
"Expansion factor ": 1
"Expansion factor Queen Side": 1 (at the start of the game, >1 white has more space, < black has more space)
"Expansion factor King Side": 1
"Mobility" :
"Mobility P" :
"Mobility B N R" :
"Mobility Q" :
"Mobility P" :
"Material" :
"Packing Density" : Still need to finish the code for that
"Integrety" : This is a feature looking at the past moves. To help evaluate the current position. I used this in my games, the idea is if you made good moves then there should be something in the current position. This is actually something engines do not do. Most engines take FEN positions.
then additionaly
counts of how many pieces of which kind are on the board
counts of what kinds of moves are likely also for each piece type
So yeah I agree this is something I need (in the long run) to set up the right system, so that we get a system like Tesla's Autopilot, where user feedback, by tagging positions, can train a NN to learn what principles (or better questions) can be connected to certain positions. This is a little in the future, but already possible. I mean in the future because thats a lot of programming and I am only one person working on this in my free time.
If it get's this far, and it will (if not by my hands then someone else will do it) then we have Apps that actually can teach us to play better Chess.
This means for me it is important that the parameter, like mobility, expansion etc can be meaningful described in a sentence so that the player can understand it, but might not know the formular. Same as some people know what a standard deviation is, without having the formula.
I have little use for a parameter that can not translated into human speech, if it consists of to many rules/connections it is not a true principle that we can use in play i.e then its not a principal component.
So far I came up with (I found the ideas online):
"Expansion factor ": 1
"Expansion factor Queen Side": 1 (at the start of the game, >1 white has more space, < black has more space)
"Expansion factor King Side": 1
"Mobility" :
"Mobility P" :
"Mobility B N R" :
"Mobility Q" :
"Mobility P" :
"Material" :
"Packing Density" : Still need to finish the code for that
"Integrety" : This is a feature looking at the past moves. To help evaluate the current position. I used this in my games, the idea is if you made good moves then there should be something in the current position. This is actually something engines do not do. Most engines take FEN positions.
then additionaly
counts of how many pieces of which kind are on the board
counts of what kinds of moves are likely also for each piece type
So yeah I agree this is something I need (in the long run) to set up the right system, so that we get a system like Tesla's Autopilot, where user feedback, by tagging positions, can train a NN to learn what principles (or better questions) can be connected to certain positions. This is a little in the future, but already possible. I mean in the future because thats a lot of programming and I am only one person working on this in my free time.
If it get's this far, and it will (if not by my hands then someone else will do it) then we have Apps that actually can teach us to play better Chess.
Counting occupied squares reminds me of H. C. Opfermann, an infamous german chess book author. He did the same in his chess "teaching" books.
This topic has been archived and can no longer be replied to.