WaveSim

Background

To facilitate a new wave of gun research, I'm setting up a system that will let me easily test a gun's classification against pre-gathered wave data. This would take the place of running a zillion Robocode battles.

Benefits:

• By removing all the overhead of running actual battles, I think this will allow a given gun test to run an order of magnitude faster, a huge boon to the pursuit of wacky targeting techniques.
• It will make it more feasible to experiment with sophisticated algorithms that are an order of magnitude slower than what we usually use, as some clustering algorithms are.
• I could record data from actual battles instead of being limited to TCs, since every classification run would be against the same exact data, anyway.
• I can see hit percentages (normalized or not) or energy ratios (spent vs gained vs damage), which I think are a more direct measurement of a gun's accuracy than TC results.

Testing classification algorithms against raw data sets is nothing new, but it's high time I actually tried it for Robocode gun research. =) Of course, this will only work against non-adaptive movements. (Some of this process is similar to my old Segmentation Research.)

Components

Data-gathering bot

First step is collecting some data. I modified my TripHammer code to output every wave it gathers in a .csv type format. 100 seasons of TCRM is about 4 gigs.

This part's done, though some partial write issues need to be worked out. (Right now just ignoring lines that aren't written correctly and presuming a little duplicated data will not harm results much.)

Battle simulation

I have utility classes to load wave data, step through it as a real battle would, feed waves to a classifier for learning, classify each firing wave / check for hits, and record overall hit rates, energy, and damage. This is basically simulating a battle from the perspective of a wave-based gun.

This is done, though it could use some bells and whistles.

Classifiers

I have a simple interface implemented by each classifier - initialize, feed it a wave, classify a wave. I've decided not to muddy up my main code base by making my guns work interchangeably with my classifier test code, but porting between them is pretty simple. Ported Diamond's main gun in a few minutes.

Blog

3/12/2010 pm

Super busy with work, so I can't focus on this much, but I've got the basics working now. For 100 seasons of TCRM:

TripHammer KNN classifier:
Took 1262.0 seconds
Hit percentage: 11.813935415920957
Energy ratio: 0.983354371037666

HOT classifier:
Took 341.0 seconds
Hit percentage: 8.739709205199102
Energy ratio: 0.7279814945288104

HOT score seems pretty high, but it gets 5.9% vs DT, which seems right. At avg distance of 500, you'd get about 5% against a perfectly flat movement.

Best of all is < 1 second per battle for Diamond's main gun. A real TCRM is 15-20 seconds per battle for this gun. Still have some obvious little optimizations to add and there are probably some bigger ones once I focus on it.

3/12/2010 am

I'll post all this soon (except the 4 gigs of data), if anyone's interested in tinkering with it. I may also post info about some of the clustering schemes I've tried. An example of one of the really intensive clustering algorithms that I find interesting is QT clustering. A couple of my previous experiments were inspired by that one.