Thanks for opening the code

Although maybe I misunderstood you, the following are where it is used: movement challenges: Movement Challenge and Movement Challenge 2K6, targeting challenges: Targeting Challenge, Targeting Challenge 2K6 and Targeting Challenge 2K7.

You run the battles yourself using something like RoboResearch, which puts out the results in a wiki-friendly format.

Also Targeting Challenge RM, which I think is the most useful one.

And also RoboRunner / RoboJogger has some advantages over RoboResearch, the main one being a big speed increase because it doesn't restart the JVM after every match.

Yes, you understood me right. I've seen the pages about those challenges in the wiki, but since I jump the train to late, I missed the discussion of their purpose. Which, I am guessing, are to tune guns and movements separately. But on those pages, I've seen long tables with ranking, so I thought they were created by some sort of a statistics bot, like we have with rumble now.

Also, I am big fan of RoboRunner. I just need to find a proper subset of bots to correlate its results with the full rumble. I sometimes see an improvement vs. my local subset and a drop in a rumble.

Oh, right, I remember seeing that you used RoboRunner, which made me happy. :-)

The targeting challenges I consider pretty useful and a good predictor of rumble performance. The movement challenges, less so. My most useful Robocode testing was usually against large test beds constructed with BedMaker. I updated it in July to work with LiteRumble, if you want to give it a try.

I should try it, but the question is which bots to choose? Assuming incremental improvements, should I aim to gain vs top bots or similar score ones?

The reason I ask for your wisdom is the following. I recall once I gain a couple of points against top bots, but the score was worse for majority of the medium and lower bots. This happens when I added more advance guns to be dodged to an enemy bot. This helped against top bots, but weaker bots gained on me. I still puzzled by it.

So ideal test bed, should have a representative from each group.

Also, a side note. I noticed that Diamond sometimes get disabled right at the beginning of the game. Most likely some minor bug but if you fix it, you might get to the 1st place. I think this bug happens only in melee.