Robot Benchmarking Thoughts
This is the thread's initial revision.
So, it recently occurred to me that the targeting/movement challenges are currently done based on the score outcome, but it really all comes down to hitrate. The thing is, there should be less noise/variability in a measure of only hitrate. So... to facilitate I started writing a tool that uses the "robocode.control.*" APIs to measure hitrate without needing to modify bots to keep track of it, and I'm thinking I ought to expand this tool to gather lots of information from the battle. Anyone have any suggestions of information I should track?
Here's an interesting pair of graphs from 500 battles of DrussGT versus Diamond
It seems that Diamond keeps it's hitrate up into the 2nd round whereas DrussGT's falls sharply right away, but it doesn't fall as far and seems to recover more. I think this type of hitrate graph could be rather interesting for investigating the dynamics of surfers versus adaptive targeting...