Creating a new (and independent) rating list

General discussion about computer chess...
User avatar
Rebel
Posts: 515
Joined: Wed Jun 09, 2010 7:45 pm
Real Name: Ed Schroder

Creating a new (and independent) rating list

Post by Rebel » Wed Jun 23, 2010 12:01 pm

It doesn't look like the ban on the IPPO's and family is going to be unleashed by the current 3 main rating lists (CCRL/CEGT/SSDF). Therefore I do hope some people here on this forum will take the initiative to create an own group of volunteers for an independent rating list.

It's quite possible some of the CCRL/CEGT folks are in doubt about the current state of affairs and can be persuaded to join such a new initiative.

Ed

Ristaaf
Posts: 4
Joined: Thu Jun 17, 2010 1:25 pm
Real Name: Rickard Staaf

Re: Creating a new (and independent) rating list

Post by Ristaaf » Wed Jun 23, 2010 12:07 pm

http://www.ccerl.net/

Norman Schmidts attempt to make a list, it didn't seem to became much more than you see now, but maybee it can with more volonters. I don't have the computer to run such tests (only an old 32bit non SSE one) otherwise (and probably when I get a new computer) I would had helped.

Robert Flesher
Posts: 53
Joined: Thu Jun 10, 2010 3:16 pm
Real Name: Robert Flesher

Re: Creating a new (and independent) rating list

Post by Robert Flesher » Wed Jun 23, 2010 1:05 pm

Rebel wrote:It doesn't look like the ban on the IPPO's and family is going to be unleashed by the current 3 main rating lists (CCRL/CEGT/SSDF). Therefore I do hope some people here on this forum will take the initiative to create an own group of volunteers for an independent rating list.

It's quite possible some of the CCRL/CEGT folks are in doubt about the current state of affairs and can be persuaded to join such a new initiative.

Ed

I may be willing to dedicate some time to this. I have most the current engines. My computer is moderately fast, Core i7 920.

JackStraw
Posts: 60
Joined: Thu Jun 10, 2010 12:14 pm
Real Name: Tom Glenn

Re: Creating a new (and independent) rating list

Post by JackStraw » Wed Jun 23, 2010 3:55 pm

Count me in if you can get somebody to organize it........

I've got several programs , and an 'old' i7-940 .........

User avatar
xshat
Posts: 98
Joined: Sat Jun 19, 2010 10:44 am

Re: Creating a new (and independent) rating list

Post by xshat » Wed Jun 23, 2010 8:36 pm

There is nothing more pathetic and laughable in the chess community than the commercialized attempts to monopolize chess engines, with the subtle intention to slow down the rate at which chess is solved...

I can help.

Gino
Posts: 15
Joined: Thu Jun 10, 2010 4:04 am

Re: Creating a new (and independent) rating list

Post by Gino » Wed Jun 23, 2010 9:40 pm

Ristaaf wrote:http://www.ccerl.net/

Norman Schmidts attempt to make a list, it didn't seem to became much more than you see now, but maybee it can with more volonters. I don't have the computer to run such tests (only an old 32bit non SSE one) otherwise (and probably when I get a new computer) I would had helped.
Why is Houdini being excluded?

BTO7
Posts: 101
Joined: Thu Jun 10, 2010 4:21 am

Re: Creating a new (and independent) rating list

Post by BTO7 » Wed Jun 23, 2010 10:17 pm

Gino wrote:
Ristaaf wrote:http://www.ccerl.net/

Norman Schmidts attempt to make a list, it didn't seem to became much more than you see now, but maybee it can with more volonters. I don't have the computer to run such tests (only an old 32bit non SSE one) otherwise (and probably when I get a new computer) I would had helped.
Why is Houdini being excluded?
He actually started the site right before Houdini came out. Its been stuck in that state since about a week after it was made. Have not seen Norm around other then his new site www.CycloneChess.com He's gone kinda quiet actually since Houdini came onto the scene.

Regards
BT

User avatar
thorstenczub
Posts: 593
Joined: Wed Jun 09, 2010 12:51 pm
Real Name: Thorsten Czub
Location: United States of Europe, germany, NRW, Lünen
Contact:

Re: Creating a new (and independent) rating list

Post by thorstenczub » Wed Jun 23, 2010 11:26 pm

BTO7 wrote:
Gino wrote:
Ristaaf wrote:http://www.ccerl.net/

Norman Schmidts attempt to make a list, it didn't seem to became much more than you see now, but maybee it can with more volonters. I don't have the computer to run such tests (only an old 32bit non SSE one) otherwise (and probably when I get a new computer) I would had helped.
Why is Houdini being excluded?
He actually started the site right before Houdini came out. Its been stuck in that state since about a week after it was made. Have not seen Norm around other then his new site http://www.CycloneChess.com He's gone kinda quiet actually since Houdini came onto the scene.

Regards
BT
maybe the witch hunters visited him ...
:roll:

whatever.

BB+
Posts: 1484
Joined: Thu Jun 10, 2010 4:26 am

Re: Creating a new (and independent) rating list

Post by BB+ » Thu Jun 24, 2010 5:14 am

I'm not sure I like the methodology of any of the current groups, but then I have a very strong preference for science. For instance, some of the rating groups let the operator choose/create the book. I have no idea how much mischief this could entail, though I see anecdotes around that Engine X does (relatively) better than Engine Y with Book Z. If we want to make it a scientific venture, more discussion is needed. For instance, should a uniform platform be adopted, or is the "benchmark and adjust" procedure sufficient? What aspects of the engine are you trying to measure (for instance, is time management important)? What interference is allowed from the GUI (for instance, is N straight moves with both at 0.00 a draw, even if no repetition has been made)? Is the focus for top engines (top 10 or 20), or for a wide variety (200+) of amateur engines? If you want the latter, then you will likely have to sacrifice "science" to some degree, as to cover such a broad spectrum you will need many different testers involved.

zwegner
Posts: 57
Joined: Thu Jun 10, 2010 5:38 am

Re: Creating a new (and independent) rating list

Post by zwegner » Thu Jun 24, 2010 6:28 am

BB+ wrote:I'm not sure I like the methodology of any of the current groups, but then I have a very strong preference for science. For instance, some of the rating groups let the operator choose/create the book. I have no idea how much mischief this could entail, though I see anecdotes around that Engine X does (relatively) better than Engine Y with Book Z. If we want to make it a scientific venture, more discussion is needed. For instance, should a uniform platform be adopted, or is the "benchmark and adjust" procedure sufficient? What aspects of the engine are you trying to measure (for instance, is time management important)? What interference is allowed from the GUI (for instance, is N straight moves with both at 0.00 a draw, even if no repetition has been made)? Is the focus for top engines (top 10 or 20), or for a wide variety (200+) of amateur engines? If you want the latter, then you will likely have to sacrifice "science" to some degree, as to cover such a broad spectrum you will need many different testers involved.
I would definitely agree here. Then again I am quite skeptical of my own testing methods.

I doubt anyways that the concept of Elo rating for chess has any real meaning. I don't think it's possible to even get any sort of unbiased sample--it's a game of perfect information after all, play shouldn't be random. Things like time control affect the rating, but different engines scale differently with time. In addition there are clearly cases where the rating system is not one dimensional (it doesn't obey the well-ordering principle, if we want to be mathematical about it :)).

Luckily, though, it seems that the Elo model roughly works in most cases.

Post Reply