Testbed
For the first time since 2004, I have assembled a testbed, but now I notice something I had not though about on beforehand. Some bots like Phoenix and ad.Quest, save data and that could have influence on the outcome. Do you use data-saving bots in your testbed and if so, how do you arrange that every version starts with a clean sheet against those. I want to alter my 40-bot testbed anyway, because 1 run (35 rounds) takes about 30-35 minutes. I still use a single core 2.66 GHz P4, although I have a i7 8-core laptop from work available. --GrubbmGait 14:39, 11 September 2011 (UTC)
You do not have permission to edit this page, for the following reasons:
You can view and copy the source of this page.
Return to Thread:Talk:Diamond/Testbed/reply.
One little note, is another option would be perhaps to modify RoboResearch to enable the option in new versions of Robocode, to obfuscate/randomize enemy names?
I thought that would return a consistent name for the same bot each time, just preventing you from pre-load bot-specific behavior? I started browsing the code just now to try and say for sure, but I wasn't able to uncover the answer as quickly as I'd hoped...
A wait... it looks like the "anonymous names" are just like "#1" and "#2" according to their position in the participants in the battle... That's unfortunate, because because unlike randomized ones, that would really confuse data saving robots by giving them polluted results.
See "getAnnonymousName()" in net.sf.robocode.host.RobotStatics
On second thoughts, it is not really an issue. Most bots that save data do it for each version separate. Only bots like cx.BlestPain that save data without version information I should remove from my testbed. And ofcourse regularly removing the contents of 'working_dirs'. Any tips about contents of the testbed? I now selected 10 of my 20 worst White Wales, 10 bots out of the top-30, and 20 bots ranging from 40-140 with a PBI close to zero. Still busy with 30 seasons for 30 minutes each, sigh. --GrubbmGait 19:24, 11 September 2011 (UTC)
These days I generally go for big test beds generated by BedMaker. Like right now, my main test bed (when working on APS) is 100 random bots that Diamond scores <= 80% against. Your test bed sounds well put together to me - improving against problem bots is good, and having a diverse set of other bots helps to ensure you're improving in general instead of just specializing against a different set of bots.
Does Diamond still has more than 100 bots it scores less than 80% against? That's a bit disappointing isn't it ;-)
Ok, me and my big mouth. Diamond has around 200 bots that qualify for the above statement, GresSuffurd still has around 300. But running such testbeds for 30 seasons or so will cost you alot of time, certainly if you like to improve gradually (like me) instead of big-bang.
