Thanks for opening the code
Although maybe I misunderstood you, the following are where it is used: movement challenges: Movement Challenge and Movement Challenge 2K6, targeting challenges: Targeting Challenge, Targeting Challenge 2K6 and Targeting Challenge 2K7.
You run the battles yourself using something like RoboResearch, which puts out the results in a wiki-friendly format.
Also Targeting Challenge RM, which I think is the most useful one.
Yes, you understood me right. I've seen the pages about those challenges in the wiki, but since I jump the train to late, I missed the discussion of their purpose. Which, I am guessing, are to tune guns and movements separately. But on those pages, I've seen long tables with ranking, so I thought they were created by some sort of a statistics bot, like we have with rumble now.
Also, I am big fan of RoboRunner. I just need to find a proper subset of bots to correlate its results with the full rumble. I sometimes see an improvement vs. my local subset and a drop in a rumble.
Oh, right, I remember seeing that you used RoboRunner, which made me happy. :-)
The targeting challenges I consider pretty useful and a good predictor of rumble performance. The movement challenges, less so. My most useful Robocode testing was usually against large test beds constructed with BedMaker. I updated it in July to work with LiteRumble, if you want to give it a try.
I should try it, but the question is which bots to choose? Assuming incremental improvements, should I aim to gain vs top bots or similar score ones?
The reason I ask for your wisdom is the following. I recall once I gain a couple of points against top bots, but the score was worse for majority of the medium and lower bots. This happens when I added more advance guns to be dodged to an enemy bot. This helped against top bots, but weaker bots gained on me. I still puzzled by it.
So ideal test bed, should have a representative from each group.
Also, a side note. I noticed that Diamond sometimes get disabled right at the beginning of the game. Most likely some minor bug but if you fix it, you might get to the 1st place. I think this bug happens only in melee.