how to build a good test bed?

Jump to navigation Jump to search

IIRC, in the past versions, the improvement over previous version is somewhat good indicator of final result, e.g. 0.5 increase in APS of common pairings (e.g. 300 common opponents) indicates 0.5 increase in final APS.

IMO the APS until full pairing is meanless, but the difference in common APS is useful.

However, this version breaks the previous pattern. difference in common APS is no longer an indicator, nor the full pairing APS.

The reason why I test agasint relatively “weak” bots is that the majority of the rumble is there. And what affects your score the most is also there. More than half of the bots in rumble is between in APS [40, 70), and there are only 160 bots above 70, and 324 bots below 40. Bots below APS 40 can be ignored IMO, as the improvement against them can only be marginal.

Xor (talk)‎05:34, 27 September 2017

You do not have permission to edit this page, for the following reasons:

  • The action you have requested is limited to users in the group: Users.
  • You must confirm your email address before editing pages. Please set and validate your email address through your user preferences.

You can view and copy the source of this page.

Return to Thread:Talk:ScalarBot/Version History/how to build a good test bed?/reply (3).

Thanks for figuring out that! I thought the rumble is stabilized very fast (common APS diff of ~300 pairings are already useful), but it turned out not.

Maybe I should build a test bed with more varieties, e.g. bots from all over the rumble with different kind of strategies.

Xor (talk)‎15:19, 27 September 2017
 
 
Retrieved from "http://robowiki.net/wiki/Thread:Talk:ScalarBot/Version_History/how_to_build_a_good_test_bed%3F/reply_(2)#how_to_build_a_good_test_bed.3F_5794"