FIDE Rules on ICGA - Rybka controversy

General discussion about computer chess...
hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: FIDE Rules on ICGA - Rybka controversy

Post by hyatt » Tue May 26, 2015 11:10 pm

Chris Whittington wrote:
hyatt wrote:Regardless of how much you prance around claiming that the "rook behind enemy pawns code is a good idea, it fails miserably. I tested it in TWO different programs, Crafty and Fruit. Both do the USUAL rook on open/half-open file scoring. I made them both use the "rybka idea" and both saw a SIGNIFICANT Elo drop. Because leaving the file is NOT a good idea in more cases than when it is.. If this is an example of YOUR chess skill, you ought to stop talking about it because it is bad. It MIGHT work if it paid attention to the pawn structure and got behind a WEAK pawn, but not just behind ANY pawn. Chalk this one up to "bad idea, or bad translation." Whichever one it is, it is "bad". And it isn't going to get any better.
I already told you, it is quite irrelevent whether the code is good, bad or merely indifferent. Stop trying to derail the topic. What matters in this context is whether Rybka=Fruit, not whether the code works when Hyatt tries it out in his test environment. Nothing manages to work in the Hyatt test environment, hence -600 ELO. See Rybka Forum for details.

Don't bother to reply Hyatt. As a useful idiot you've been very helpful to us in suiciding your own case over the last four years, but you usefulness for us has now passed. Bye.

Then why do YOU keep trying to point out how useful/important the idea is, when it fails miserably? As far as Crafty's Elo goes, at least it is ORIGINAL Elo, eh? Rybka can't make that claim.

User avatar
Chris Whittington
Posts: 437
Joined: Wed Jun 09, 2010 6:25 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Chris Whittington » Wed May 27, 2015 5:45 am

hyatt wrote:
Chris Whittington wrote:
hyatt wrote:Regardless of how much you prance around claiming that the "rook behind enemy pawns code is a good idea, it fails miserably. I tested it in TWO different programs, Crafty and Fruit. Both do the USUAL rook on open/half-open file scoring. I made them both use the "rybka idea" and both saw a SIGNIFICANT Elo drop. Because leaving the file is NOT a good idea in more cases than when it is.. If this is an example of YOUR chess skill, you ought to stop talking about it because it is bad. It MIGHT work if it paid attention to the pawn structure and got behind a WEAK pawn, but not just behind ANY pawn. Chalk this one up to "bad idea, or bad translation." Whichever one it is, it is "bad". And it isn't going to get any better.
I already told you, it is quite irrelevent whether the code is good, bad or merely indifferent. Stop trying to derail the topic. What matters in this context is whether Rybka=Fruit, not whether the code works when Hyatt tries it out in his test environment. Nothing manages to work in the Hyatt test environment, hence -600 ELO. See Rybka Forum for details.

Don't bother to reply Hyatt. As a useful idiot you've been very helpful to us in suiciding your own case over the last four years, but you usefulness for us has now passed. Bye.

Then why do YOU keep trying to point out how useful/important the idea is, when it fails miserably? As far as Crafty's Elo goes, at least it is ORIGINAL Elo, eh? Rybka can't make that claim.
Vas Rajlich, five times world champion, rating list leader, and International Master managed to get ithe computationally free, chess knowledge to work. R Hyatt, languishing 600 points behind in the rating lists didn't. Go figure.

Meanwhile the rook-pawn coding for Rybka produces significantly different rook-square table heat maps than does fruit. Rybka rook heap maps are similar to strong human player heat maps and contain computationally free significant chess knowledge over and above Fruit. This simple fact was overlooked by the three investigators when they decided the two codings were "similar", omitting the vital detail that the chess knowledge is NOT similar by any means. How can we trust the results of our three investigators when they miss out the "chess" side of chess programming and falsely ascribe similarity based on their lack of knowledge and lack of vigourous research?

User avatar
Chris Whittington
Posts: 437
Joined: Wed Jun 09, 2010 6:25 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Chris Whittington » Wed May 27, 2015 9:37 am

BB+ wrote:
Chris Whittington wrote:I would say your 0.8 for the rook-pawn code is an outrageous mistake.
It was 0.7 for "open" files and 0.6 for "semi-open" files.
Chris Whittington wrote:why no FILTERING?
There are three directions for filtering, namely reasons of efficiency, extrernal factors, or common use. The first two have much in common for computer programs, and an example could be that backward/weak pawns are implemented in a different way due to an external factor like board representation, possibly for efficiency. I tried not to make such filtrations when unnecessary, as it would tend to diminish the Rybka/Fruit differences. But I guess you are more concerned about common use. A group-wise comparison already has filtration by commonality built it. If every program has a feature, and implements it a similar manner, then all boats will be raised by the rising tide.

On the other hand, if there are (say) 10 features and two choices for the way of realising each, then you expect that if you take 10 engines for any feature to have 4 or 5 matches amongst the others, but it will still be strange to have an engine pair have 9 or 10 choices the same.

For instance:
00000 11111 Engine A
00000 11110 Engine B
10101 01010 Engine C
01100 11011 Engine D
11001 01100 Engine E
01001 00110 Engine F
Looking at the vertical columns (for features), one could "filter" every single feature on the basis that some engine in CDEF chose the same option, but this would miss the great A-B similarity.
Here we get to the really serious systemic error in Comp Eval.
Just as chess games move through stages, 8+8 pawns, to 7+7, 6+6 etc etc, so do chess programs. young programs start with the general idea implemented for evaluation components (mobility will tend to be no more than sum of squares eg), and add in second and third order effects as versions progress. not to mention ELO/desperation tinkering.
Your matching criteria is majorly biased against young program pairs. i hope (expect) your maths intuition sees that readily.

User avatar
Chris Whittington
Posts: 437
Joined: Wed Jun 09, 2010 6:25 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Chris Whittington » Wed May 27, 2015 9:41 am

BB+ wrote:
hyatt wrote:// mask_open_file_w[square] is a bitmap of squares in front of the square "square" from white's perspective. If square == E2, then
// this contains 1 bits on E3, E4, E5, E6 and E7. Zeros everywhere else.
This is wrong, it also has a 1-bit on E8.

In Strelka, the initalisation is:

Code: Select all

    for (sq = i + 8; sq < 64; sq += 8)
      MaskPawnOpenFileW[i] |= (unsigned __int64)1<< sq;
In Rybka (64-bit disassembly of 1.0 version, as on the ICGA wiki), see this line

Code: Select all

0x401b48: mov    0x24a340(%r8,%r9,8),%rax        mask for open files
The relevant array is thus the 512 bytes starting from 0x64a340:

Code: Select all

0x64a340:       0x0101010101010100      0x0202020202020200
0x64a350:       0x0404040404040400      0x0808080808080800
0x64a360:       0x1010101010101000      0x2020202020202000
0x64a370:       0x4040404040404000      0x8080808080808000
0x64a380:       0x0101010101010000      0x0202020202020000
0x64a390:       0x0404040404040000      0x0808080808080000
0x64a3a0:       0x1010101010100000      0x2020202020200000
0x64a3b0:       0x4040404040400000      0x8080808080800000
0x64a3c0:       0x0101010101000000      0x0202020202000000
0x64a3d0:       0x0404040404000000      0x0808080808000000
0x64a3e0:       0x1010101010000000      0x2020202020000000
0x64a3f0:       0x4040404040000000      0x8080808080000000
0x64a400:       0x0101010100000000      0x0202020200000000
0x64a410:       0x0404040400000000      0x0808080800000000
0x64a420:       0x1010101000000000      0x2020202000000000
0x64a430:       0x4040404000000000      0x8080808000000000
0x64a440:       0x0101010000000000      0x0202020000000000
0x64a450:       0x0404040000000000      0x0808080000000000
0x64a460:       0x1010100000000000      0x2020200000000000
0x64a470:       0x4040400000000000      0x8080800000000000
0x64a480:       0x0101000000000000      0x0202000000000000
0x64a490:       0x0404000000000000      0x0808000000000000
0x64a4a0:       0x1010000000000000      0x2020000000000000
0x64a4b0:       0x4040000000000000      0x8080000000000000
0x64a4c0:       0x0100000000000000      0x0200000000000000
0x64a4d0:       0x0400000000000000      0x0800000000000000
0x64a4e0:       0x1000000000000000      0x2000000000000000
0x64a4f0:       0x4000000000000000      0x8000000000000000
0x64a500:       0x0000000000000000      0x0000000000000000
0x64a510:       0x0000000000000000      0x0000000000000000
0x64a520:       0x0000000000000000      0x0000000000000000
0x64a530:       0x0000000000000000      0x0000000000000000
Note that, in particular, 0x64a3a0 (the square e2) is 0x1010101010100000 which does indeed have the E8 square set.

You can argue that there can't be any pawns on the 1st/8th rank, but the bitmask in question checks said squares.

Note that in Fruit the board->pawn_file is updated in square_clear() and square_set() after being set in board_init_list().
the proof it is no "bug" (Hyatt) is that there isn't one mask, but two for Rybka. one black and one white. since these are, and are used, consistent with each other but directionally fir colour, the idea that both are consistently bugged whilst both working quite logically is obviously (desperate) nonsense.

User avatar
Chris Whittington
Posts: 437
Joined: Wed Jun 09, 2010 6:25 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Chris Whittington » Wed May 27, 2015 10:07 am

BB+ wrote:
Chris Whittington wrote:... the thing you don't understand. Rybka gives a bonus for getting round the back of the enemy pawn chain.
The Rybka bonus is oblivious to the base of the pawn chain, or even if the opponent has any pawns at all. Black pawns on f7, g6, and h5, White pawn on a2, White rook on a3, Black pawns on a6 and b7. Rybka gives a "half-open" file bonus for the rook on a3 (whereas Fruit would not, due to the White pawn on a2). It is unclear what this has to do with "getting round the back of [either] enemy pawn chain". Furthermore, previously it was claimed the Rybka "pattern recognition" would be useful on highly blocked boards, which typically have more than one base to the pawn chain(s), weak pawns/squares, etc.
uou seem to be taking my comment too literally. the Rybka code does not "recognise" the pawn chain in any direct manner, rather it has in effect a rook square heatmap for every possible pawn confirmation. an emerging proprty of this is that it bonusus the rook for getting round the back of the pawn chain. I guess that's why the three of you missed the chessic detail.

if you like, consider not the code, but the rook square heat maps as lookup data, a huge number of rook square tables for all possible pawn structures. Very, very different between Fruit and Rybka, no?

Now, the example you give .... so what? Rybka gives LESS bonus for being on a3 than it woudl for being on c3 or d3, Isn't this sensible? Rybka would be happier being on a8 or b8 than it would on a3. Rybka gives the same (reduced) bonus for being on a3 as b3, whereas zfruit doesn't want to be on a3 at all, and gives a partial bonus for being on b3. switch the pawn from a2 to b2 and Fruit doesn't want to be on b3 at all. We can play at specially constructed positions where a pawn is miraculously defended, but this code doesn't look at adjacent files and doesn't know about defended, so we would only be into pointless swings and roundabouts. In GENERAL, and this is general puroose code, first order and quick, as we progress thriugh the game, exchanging down pawns, the Rybka code has more knowledge and should help guide rook penetrations and exploitation better than Fruit.

User avatar
Chris Whittington
Posts: 437
Joined: Wed Jun 09, 2010 6:25 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Chris Whittington » Wed May 27, 2015 10:11 am

BB+ wrote:
Chris Whittington wrote:The bishop pair is a well known chess heuristic. It says "2 bishops is good" and better than "1 bishop" or "0 bishop". Therefore chess programmer, he count bishops and say if count>1 then bonus. Therefore ... FILTER this one OUT
Note that filtering anything where the Rybka/Fruit overlap is less than its average will tend to increase the final observation. Here 0.3 (bishop pair) is much less than is typical for the pair. One can also note (see EVAL_COMP) description that there are many ways of realising this well-known chess heuristic, and indeed only RESP/ExChess/Fruit have a complete matching. This is another sign that "bishop pair" in and of itself is too broad, and one needs to go to a finer level before arbitrarily "filtering" out (supposed) commonalities.
I would argue all four mobilities should be filtered on basis known equations and quick and dirty executions by young programs. Inclusion offends against young program bias.

terrapene
Posts: 69
Joined: Sun May 17, 2015 10:11 am
Real Name: Yes

Re: FIDE Rules on ICGA - Rybka controversy

Post by terrapene » Wed May 27, 2015 4:56 pm

Rebel wrote:
BB+ wrote: I don't see any reason to continue posting on this matter. As long as you are arguing about "code theft", and/or comparing Rybka/Fruit to direct clones, the relevance of your work to the ICGA investigation and decision is almost nil.
Then don't deny the reality of that. You are relative new in this world. I have dwelled among them for 25 years. I know them personally. I know what moves them, what upsets them, pretty much the same as me. And I have been in your VIG camp remember? I know exactly how they think, what they think, also about this case, cryptic rule #2 became the stick to beat Vas for copying Fruit. It's what they believed and I can give you the public quotes of Zach, Bob, Lefler (to name a few) that proofs it. And even you, although more careful have hinted to that, for example:

1. Mark Watkins: It also seems (see node counts and depth) that Rajlich did in fact have "obfuscation" as one of the things on his mind during this period [though maybe just externally].

2. Mark Watkins: I think the case for "copyright infringement" (or plagiarism) of the evaluation function as a whole is quite weighty, particularly when combined with the various other Fruit 2.1 bits that appear here-and-there in Rybka 1.0 Beta.

~~~~~~~~~~~~~~~~~~~~~~

So then following your wish let's talk about rule #2.

My main beef with your and Zach's document:

Mark Watkins: I think the case for "copyright infringement" (or plagiarism) of the evaluation function as a whole is quite weighty,

Zach Wegner: Simply put, Rybka's evaluation is virtually identical to Fruit's.

And more of this type of qualifications.

Then when we go in depth and let's take the code snippet already posted, the evaluation of open c.q. semi open rook files:

Code: Select all

// Start ROOK eval   [ RYBKA 1.0 ]
// ===============
		
if (!(v84 & HIDWORD(qword_667BA0[0]) | v83 & qword_667BA0[0]) ) // open / semi open rook files
  {                       // when semi-open
     v271 += 256;         // endgame+256	
     v6 += 64;            // opening+64 }
 
if ( !(v84 & HIDWORD(qword_667BA8[0]) | v83 & qword_667BA8[0]) )
  {                       // when open file
    v6 += 971;            // opening+971
    v271 += 172;          // endgame+172  }
So what is evaluated here? Simple public available chess knowledge you will find in chess teachings books to put your rooks on open files and the less open files (expressed in this case via MG/EG, a public available technique introduced in Phalanx, predating Fruit and Rybka with 4-5 years) the more important it becomes. Every reasonable open source has it, no big secret, so has Fruit, so has Rybka, so have I since the early 80's. Rook (semi) open file evaluation in a most simply form, common chess knowledge.

And it's coded differently even if we remove the mailbox-bitboard difference, as Sven Schuele correctly pointed out:

Regarding the "rook on open file" evaluation, both Zach Wegner and Mark Watkins have observed correctly that there is an important difference between Fruit in Rybka, in contrast to Fruit, Rybka does NOT consider friendly pawns behind the rook (e.g. Ra3, Pa2) to determine a "semi-open file" or "open file".

The Zach Wegner Rybka equivalent code seems to reflect that correctly if we assume that the bitmask mask_open_file_w[square] has all white pawns on ranks lower than 'square' masked out. Since the assembler code would match the "Zach Wegner code" it seems to be plausible. BUT the major point is, this Fruit-Rybka difference is indeed a SEMANTICAL DIFFERENCE.


~~~~~~~~~~~~~~~~

I mean, seriously, this is tunnel vision | VIG thinking. How can common chess knowledge fall under the interpretation of rule #2 which is about originality? There is no originality in chess knowledge, it's fixed, you better follow the rules of the books that teach chess else your engine will perform bad.

I could go on, I can post a dozen of other examples (as above) where (especially) Zach reasoned this way to incriminate Rybka while in reality it's about common chess knowledge and coded DIFFERENTLY than in Fruit even after removing the mailbox / bitboard difference.

What the hell does rule #2 mean?
Rebel, rule #2 deliberates by retention of leeway for proper intrepreting.
this happens all the time, even at the UN you have committees that interpret treatsies decades later.
and always to appease their benelux masters.

why do you think the icga is any different?
do they need a 30 page document just to say what original means.
no, this can be clear to any one, or they can ask.

hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: FIDE Rules on ICGA - Rybka controversy

Post by hyatt » Wed May 27, 2015 5:50 pm

Chris Whittington wrote:
hyatt wrote:
Chris Whittington wrote:
hyatt wrote:Regardless of how much you prance around claiming that the "rook behind enemy pawns code is a good idea, it fails miserably. I tested it in TWO different programs, Crafty and Fruit. Both do the USUAL rook on open/half-open file scoring. I made them both use the "rybka idea" and both saw a SIGNIFICANT Elo drop. Because leaving the file is NOT a good idea in more cases than when it is.. If this is an example of YOUR chess skill, you ought to stop talking about it because it is bad. It MIGHT work if it paid attention to the pawn structure and got behind a WEAK pawn, but not just behind ANY pawn. Chalk this one up to "bad idea, or bad translation." Whichever one it is, it is "bad". And it isn't going to get any better.
I already told you, it is quite irrelevent whether the code is good, bad or merely indifferent. Stop trying to derail the topic. What matters in this context is whether Rybka=Fruit, not whether the code works when Hyatt tries it out in his test environment. Nothing manages to work in the Hyatt test environment, hence -600 ELO. See Rybka Forum for details.

Don't bother to reply Hyatt. As a useful idiot you've been very helpful to us in suiciding your own case over the last four years, but you usefulness for us has now passed. Bye.

Then why do YOU keep trying to point out how useful/important the idea is, when it fails miserably? As far as Crafty's Elo goes, at least it is ORIGINAL Elo, eh? Rybka can't make that claim.
Vas Rajlich, five times world champion, rating list leader, and International Master managed to get ithe computationally free, chess knowledge to work. R Hyatt, languishing 600 points behind in the rating lists didn't. Go figure.

Meanwhile the rook-pawn coding for Rybka produces significantly different rook-square table heat maps than does fruit. Rybka rook heap maps are similar to strong human player heat maps and contain computationally free significant chess knowledge over and above Fruit. This simple fact was overlooked by the three investigators when they decided the two codings were "similar", omitting the vital detail that the chess knowledge is NOT similar by any means. How can we trust the results of our three investigators when they miss out the "chess" side of chess programming and falsely ascribe similarity based on their lack of knowledge and lack of vigourous research?
It is not "chess knowledge". It is "chess stupidity". Believe what you want. It does NOT work. When I have the time, I will modify 1.0beta to do this the right way and play a bunch of games. The code ought to fit into the same memory footprint since it would only be a memory address change. I'll report back. What are you going to say when THAT makes 1.0beta STRONGER? You going to keep prancing around telling me how good he was/is at this stuff THEN?

User avatar
Chris Whittington
Posts: 437
Joined: Wed Jun 09, 2010 6:25 pm

Re: FIDE Rules on ICGA - Rybka controversy

Post by Chris Whittington » Wed May 27, 2015 7:38 pm

hyatt wrote:
Chris Whittington wrote:
hyatt wrote:
Chris Whittington wrote:
hyatt wrote:Regardless of how much you prance around claiming that the "rook behind enemy pawns code is a good idea, it fails miserably. I tested it in TWO different programs, Crafty and Fruit. Both do the USUAL rook on open/half-open file scoring. I made them both use the "rybka idea" and both saw a SIGNIFICANT Elo drop. Because leaving the file is NOT a good idea in more cases than when it is.. If this is an example of YOUR chess skill, you ought to stop talking about it because it is bad. It MIGHT work if it paid attention to the pawn structure and got behind a WEAK pawn, but not just behind ANY pawn. Chalk this one up to "bad idea, or bad translation." Whichever one it is, it is "bad". And it isn't going to get any better.
I already told you, it is quite irrelevent whether the code is good, bad or merely indifferent. Stop trying to derail the topic. What matters in this context is whether Rybka=Fruit, not whether the code works when Hyatt tries it out in his test environment. Nothing manages to work in the Hyatt test environment, hence -600 ELO. See Rybka Forum for details.

Don't bother to reply Hyatt. As a useful idiot you've been very helpful to us in suiciding your own case over the last four years, but you usefulness for us has now passed. Bye.

Then why do YOU keep trying to point out how useful/important the idea is, when it fails miserably? As far as Crafty's Elo goes, at least it is ORIGINAL Elo, eh? Rybka can't make that claim.
Vas Rajlich, five times world champion, rating list leader, and International Master managed to get ithe computationally free, chess knowledge to work. R Hyatt, languishing 600 points behind in the rating lists didn't. Go figure.

Meanwhile the rook-pawn coding for Rybka produces significantly different rook-square table heat maps than does fruit. Rybka rook heap maps are similar to strong human player heat maps and contain computationally free significant chess knowledge over and above Fruit. This simple fact was overlooked by the three investigators when they decided the two codings were "similar", omitting the vital detail that the chess knowledge is NOT similar by any means. How can we trust the results of our three investigators when they miss out the "chess" side of chess programming and falsely ascribe similarity based on their lack of knowledge and lack of vigourous research?
It is not "chess knowledge". It is "chess stupidity". Believe what you want. It does NOT work. When I have the time, I will modify 1.0beta to do this the right way and play a bunch of games. The code ought to fit into the same memory footprint since it would only be a memory address change. I'll report back. What are you going to say when THAT makes 1.0beta STRONGER? You going to keep prancing around telling me how good he was/is at this stuff THEN?
hahahaha!! I thought this thread was about Rybka == Fruit or not. But apparently it is about whether

.... Vas is a better chess programmer than Hyatt or not.

Bit of a no-brainer question, Hyatt. The connecting equation is already known.
Hyatt = Vas - 600 ELO

Now, back to the topic on hand ...... Rybka rook-square table heatmaps for all pawn structures != Fruit's. Not by a long way. Very different indeed. Hyatt plus the other two missed the chessic differences. The correlation numbers given for rook-pawn in COMPEVAL are way out and majorly overestimated. There is NO correlation at all because the two programs are measuring quite different things and producing quite different heatmaps.

hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: FIDE Rules on ICGA - Rybka controversy

Post by hyatt » Thu May 28, 2015 12:28 am

He is certainly NOT "better than Hyatt" in this instance. The evaluation term, as coded, is simply bad.

Verified in two separate programs. I suppose I could do it again for stockfish, but I don't see the point.

Post Reply