Robot Benchmarking Thoughts

Jump to navigation Jump to search
Revision as of 16 May 2012 at 20:18.
The highlighted comment was created in this revision.

Robot Benchmarking Thoughts

So, it recently occurred to me that the targeting/movement challenges are currently done based on the score outcome, but it really all comes down to hitrate. The thing is, there should be less noise/variability in a measure of only hitrate. So... to facilitate I started writing a tool that uses the "robocode.control.*" APIs to measure hitrate without needing to modify bots to keep track of it, and I'm thinking I ought to expand this tool to gather lots of information from the battle. Anyone have any suggestions of information I should track?

Here's an interesting pair of graphs from 500 battles of DrussGT versus Diamond

Hitrate graph DrussVSDiamond.png

It seems that Diamond keeps it's hitrate up into the 2nd round whereas DrussGT's falls sharply right away, but it doesn't fall as far and seems to recover more. I think this type of hitrate graph could be rather interesting for investigating the dynamics of surfers versus adaptive targeting...

    Rednaxela20:53, 16 May 2012

    During the first round the surfing is very predictable, making the guns learn much quicker; there is also very little data in the gun, so it acts like a very fast rolling gun.

    I wonder how these are affected by when flatteners and alternate guns are enabled and disabled?

    Also, keep in mind that DrussGT fires lower bullet power (or did, last time I checked).

      Skilgannon21:18, 16 May 2012