Test : who is right

Code, algorithms, languages, construction...
Post Reply
Hamfer
Posts: 12
Joined: Wed Dec 19, 2012 5:37 pm

Test : who is right

Post by Hamfer » Tue Nov 24, 2015 2:54 pm

Hello,
First, sorry for bad englsih.

I'm faced to strange behaviour when testing my experimental engine.
I lunch the same test (same conditions) under cuteches-cli and Arena and the result was totally different.
With cutechess-cli, it loses => 70%
With arena, it wins => 90%.

This the cutechess-cli command :
%CHEMIN%\tools\cutechess-cli-0-7-2\cutechess-cli.exe ^
-engine name=DEV cmd=DEV.exe dir="D:\arena_3.0\Engines\DEV" ^
-engine name=ENG cmd=ENG.exe dir="D:\arena_3.0\Engines\ENG" ^
-each proto=uci tc=10+0.5 ^
-openings file=%CHEMIN%\pgnsets\Test.epd format=epd ^
-pgnout %CHEMIN%\PGN_RES\%PGN_DATABASE% ^
-games 2 ^
-rounds 7536 ^
-repeat ^
-recover ^
-concurrency 3

Help
Attachments
Capture2.PNG
Arena time
Capture1.PNG
Arena match

hyatt
Posts: 1242
Joined: Thu Jun 10, 2010 2:13 am
Real Name: Bob Hyatt (Robert M. Hyatt)
Location: University of Alabama at Birmingham
Contact:

Re: Test : who is right

Post by hyatt » Tue Nov 24, 2015 5:21 pm

Hamfer wrote:Hello,
First, sorry for bad englsih.

I'm faced to strange behaviour when testing my experimental engine.
I lunch the same test (same conditions) under cuteches-cli and Arena and the result was totally different.
With cutechess-cli, it loses => 70%
With arena, it wins => 90%.

This the cutechess-cli command :
%CHEMIN%\tools\cutechess-cli-0-7-2\cutechess-cli.exe ^
-engine name=DEV cmd=DEV.exe dir="D:\arena_3.0\Engines\DEV" ^
-engine name=ENG cmd=ENG.exe dir="D:\arena_3.0\Engines\ENG" ^
-each proto=uci tc=10+0.5 ^
-openings file=%CHEMIN%\pgnsets\Test.epd format=epd ^
-pgnout %CHEMIN%\PGN_RES\%PGN_DATABASE% ^
-games 2 ^
-rounds 7536 ^
-repeat ^
-recover ^
-concurrency 3

Help

Have you looked at the games to see what is different? I'd suspect a lot of time losses. Some programs seem to not behave well. For instance, if you use Gaviota, it dumps a BUNCH of unnecessary stuff to the GUI. Using xboard, this can cause the opponent to lose on time when Gaviota makes a move, then dumps many hundreds of lines of garbage output to xboard. While xboard is processing all that input, the opponent can lose on time when time is short.

You might also have a GUI with bugs that can cause similar bias. You have to look at the games to see why it is losing...

Hamfer
Posts: 12
Joined: Wed Dec 19, 2012 5:37 pm

Re: Test : who is right

Post by Hamfer » Tue Nov 24, 2015 8:59 pm

Thank you for the response.

Yes, there is some games that terminate with "White's connection stalls".
But there is another problem, tc is 10+0.5 and the game starts with 1. Bf3 {+0.20/11 0.69s} => how the time jumped from 10s to 69s.

[Event "?"]
[Site "?"]
[Date "2015.11.24"]
[Round "1"]
[White "DEV"]
[Black "ENG"]
[Result "0-1"]
[FEN "r4rk1/1q2bppp/p1bppn2/1p6/3BPP2/P1N3Q1/1PP1B1PP/4RR1K w - -"]
[PlyCount "182"]
[SetUp "1"]
[TimeControl "10+0.5"]

1. Bf3 {+0.20/11 0.69s} Rad8 {-0.09/11 1.1s} 2. e5 {+0.16/11 0.68s}
dxe5 {-0.09/9 0.37s} 3. fxe5 {+0.15/13 0.66s} Nd5 {0.00/11 0.67s}
4. Ne4 {+0.37/11 0.67s} Qc7 {-0.13/11 0.46s} 5. c3 {+0.26/10 0.66s}
... {-M2/20 0.10s, White's connection stalls} 0-1

Post Reply