Testbed

For the first time since 2004, I have assembled a testbed, but now I notice something I had not though about on beforehand. Some bots like Phoenix and ad.Quest, save data and that could have influence on the outcome. Do you use data-saving bots in your testbed and if so, how do you arrange that every version starts with a clean sheet against those. I want to alter my 40-bot testbed anyway, because 1 run (35 rounds) takes about 30-35 minutes. I still use a single core 2.66 GHz P4, although I have a i7 8-core laptop from work available. --GrubbmGait 14:39, 11 September 2011 (UTC)

I do use data saving bots and am just ignoring the issue right now, actually, which is perhaps stupid. But it should be pretty easy to modify RoboResearch to delete the robot.database and .data after every season, so they get rebuilt and saved data is cleared. That would be a very nice option to have. Are you using RoboResearch?

One little note, is another option would be perhaps to modify RoboResearch to enable the option in new versions of Robocode, to obfuscate/randomize enemy names?

I thought that would return a consistent name for the same bot each time, just preventing you from pre-load bot-specific behavior? I started browsing the code just now to try and say for sure, but I wasn't able to uncover the answer as quickly as I'd hoped...

A wait... it looks like the "anonymous names" are just like "#1" and "#2" according to their position in the participants in the battle... That's unfortunate, because because unlike randomized ones, that would really confuse data saving robots by giving them polluted results.

See "getAnnonymousName()" in net.sf.robocode.host.RobotStatics

These days I generally go for big test beds generated by BedMaker. Like right now, my main test bed (when working on APS) is 100 random bots that Diamond scores <= 80% against. Your test bed sounds well put together to me - improving against problem bots is good, and having a diverse set of other bots helps to ensure you're improving in general instead of just specializing against a different set of bots.

Does Diamond still has more than 100 bots it scores less than 80% against? That's a bit disappointing isn't it ;-)

Ok, me and my big mouth. Diamond has around 200 bots that qualify for the above statement, GresSuffurd still has around 300. But running such testbeds for 30 seasons or so will cost you alot of time, certainly if you like to improve gradually (like me) instead of big-bang.