Ideas versus Implementations

[Chris Whittington]

The issue we have here (the rybka/fruit case) is that the codes simply match up so well, testing the same things, in the same way, with the same quirky ways of adding bonuses or subtracting them back out, and such, that it is not very plausible to suggest independent development.

Yes, we understand your position. You wish to suggest that your "matches", such as they are, and they are not "source or any other code" matches btw, do the "same" things, and, when they don't "do the same things" that is because they were copied and altered(!!); and in the "same way", except when they don't when you blame that on the abstracted data structure and "alteration" (again). Which then leaves you grasping at "quirks" which would be fine if you didn't call provably the most efficient implementations as "quirks".

Catch 22. If it's the same he copied it. If it's different he copied it and changed it.

And about "quirks"

Fruit quirks
1. The 2 step evaluation
2. The quad function
3. Float based time control
4. Only one evaluation table (king safety)

Rybka quirks
1. Pawn value of 3200
2. Losing 20% of blitz games on time.
3. Lacking any knowledge about standard endgames, can not even mate the KBNK ending.

None of the above 4 Fruit oddities

I refer you to the Sitwala post: with sufficient abstraction you can prove diamond and coal to be identical. With sufficient abstraction Mr Watkins could give the diamond-coal pair a matching value of 1.0, but that would only be because he ignored and threw away the abstracted differences, would it not? Now there's a thought. Should the metrication of similarity/difference of any subfunction pair (to be used to incriminate another programmer for copying) re-include the thrown away abstractions, or not?

I never gave Mark's COMP_EVAL much attention. At best (VIG) it can only indicate that Vasik modelled his EVAL to Fruit's. It can not proof copying. Free idea's how to create an evaluation function voluntarily put on the Internet by Fabien himself. That's not cloning but learning and coding what you have learned.

But to comment on the above I think for a more fair comparison a subtraction of 0.2 point for each MB/BB eval term is in place. Somehow the Rybka investigators were able to undermine Rybka's MAIN unburden evidence (a bit-board engine) into a disadvantage that greatly worked against Vas. If there is a difference between Fruit and Rybka blame it on the bit-boards and the difference magically disappears. I think that's pretty unfair.

The first problem with those numbers arose when Watkins allowed himself to arbitrarily create the scale and then for Hyatt and others to pretend that, say a final figure of 74% represented 74% identical, or 74% similar or 74% copied . There's no argument that a 0.0 figure represents zero copying, but 1.0 does not represent 100% identical. A direct clone with perhaps the name of the program changed should be a 100% match, that gives meaning to the expression 74% identical, that anyone can comprehend. It also prevents the confused or the unscrupulous from pretending that the 74% figure means that 74% of the program is copied.

Watkins should have considered the purposes to which his paper would be put. If he had put his scale of similarity onto the obvious range where 0.0 represented no similarity and 1.0 represented full-on clone then there would haver been less opportunity for misrepresentation of the results, either by the unscrupulous, the biased or by tabloid journalists.

There are two major elements of the programs which were abstracted away in order to make it easier for the investigators to see feature similarity. Weights and data structure. Watkins then went on to define a whole range of numbers to subtract for each and every difference between the programs except for the abstracted weights and data structure. Neither of these elements was restored to the metrication process, when both could easily have been. If they were restored we a) get a range we can understand, and b) a fairer metric for comparison. There is nothing that says that when features are abstracted for reverse engineering clarity purposes, they should not be put back in again when making an overall comparison metric more clear and accurate.

The contribution of the Weights. Although almost all weights were abstracted (removed), some remained, and we have the figures that Watkins used where he found different weights used between two sub functions. In the four mobility terms, the scalar weights were removed but a translation weight remained (the start weight offset for the piece type, in Fruit was a negative offset and in Rybka was zero). Watkins took off 0.2 (20%) for this difference.

If we treated all program pairs this way, and deducted 20% for weight differences, we would start to make sense of the similarity scale:
100% = identical
80% = identical with all weights changed

The contribution of the Data Structure. It is clear that a bitboard program and a mailbox program look very different at code level (for example the mobility code). They behave differently, they treat chess concepts differently, one tends to use computationally expensive loops while the other uses logic, one will tempt the programmer into one set of solutions, the other into another set. One will tempt the programmer to be more likely to use certain sub functions which happen to be computationally cheaper in that data structure (eg the mobility terms again). Different data structures have an Elo effect. Different data structures make for different programs, very different in the case of bitboard/mailbox. So different that investigators have really quite a problem in finding similarities.

Ed Schroder suggests, above, taking 0.2 off for BB/MB data structure, I'ld be inclined to argue 0.3 on the grounds that it is more consequential.

Now we can make more sense of the proper scale 0 to 1:
100% = identical
80% = identical with all weights different
70% = identical with different data structure
50% = identical with different data structure and all weights different
0% = everything different

Well, fine you might say, all you've done is squash the Watkins data results into the range 0 to 0.5; doesn't change the overall relative result.

Ah, but it does, it removes a fundamental bias which existed, perhaps unknowingly, in the original Watkins method and results. From the program pool used by Watkins, only Crafty and Rybka were bitboard programs. Thus a substantial difference between those two (Rybka and Crafty) and the other members of the set was completely ignored. With the effect that both Rybka and Crafty similarity scores were raised. This is bias. Bad bias.

Over the weekend, I will modify the Watkins data, removing the anti-Rybka (and Crafty) bias and publish the results. At that point we will have a more realistic and fair scale to work from as we look at the "fairness" of the rest of the Watkins method on each of the "matching" sub function terms.

[Rebel]

I have underlined the word copyright, not because it would be my opinion that Rule 2 is about copyright, but because the discussion always seems to go back to copyright.

Rule #2 eventually is about copying. Just go through the history of caught ICGA cloners. Bob on many occasions has said it literally, "rule #2 is about copying since day one". Fabien's open letter also speaks volumes. The fact the ICGA uses a milder words (plagiarism and later non-literal copying) means that rule #2 has been pushed to its extreme limits to get a guilty verdict.

[Rebel]

If I write a novel, the underlying story (one abstraction level up) is also subject to copyright. In some cases (depending how generic the story is, for instance), you would not be free to realise the same story as a motion picture, without infringing copyright.

The game of chess (highest abstraction level) is non copyrightable so your comparison is not valid by definition. Take the 4 gospels as an example, the 4 authors describe the life of Jesus (highest abstraction level) as they have witnessed, interpreted and remembered things in their own words and style. Non copyrightable.

Similarly, using something akin to translation, and then claiming this is re-expressing the "idea" behind source code seems to me to be misapplying the word "idea".

Assumptions are proof?

Regarding the 4 gospels example It is said that both Lucas and Matthew borrowed from Mark the so called Q-source but proof does not exists, it's an assumption. You will have to proof the Fruit MB to Rybka BB conversion accusation, assumptions make bad proof especially when more logical explanations are available.