WaveSim

Background

To facilitate a new wave of gun research, I created a system that lets me test a gun's classification algorithm against pre-gathered wave data. This can take the place of running a zillion Robocode battles.

Benefits:

• By removing all the overhead of running actual battles, I can run a gun benchmark much faster -- like minutes instead of hours.
• Tests can be run against data from real battles instead of a TC scenario, which should be more relevant to real battle performance.
• Every benchmark runs against the exact same data, resulting in direct comparisons of test results without worrying about randomness.
• I can see hit percentages or energy ratios (spent vs gained vs damage), which I think are a more direct measurement of a gun's accuracy than TC results.

Testing classification algorithms against raw data sets is nothing new or special, but it's about time someone set up a solid system for doing it with Robocode!

(Note that a good classification algorithm is not the only important part of a gun. But I feel that Diamond and most other top bots have the rest of their gun code polished to a point where classification is the thing we work on 99% of the time.)

Components

Data-gathering bot

A modified TripHammer outputs every gun wave it gathers to a gzipped binary file. Each battle gets a new filename and TripHammer writes to it throughout the battle.

You could, of course, modify your own bot to output wave data to a file of the same or similar format. Just take a look at the TickRecord class in WaveSim for guidance on the format, or look at/borrow code from TripHammer's TripWave.writeTickRecord() method.

Battle simulation

WaveSim records the raw attribute data for each tick and fires a wave for that tick. When the wave is collected, it records another type of record with the hitting angle from the wave. By reading these in the same order that they occurred, a targeting algorithm will learn from waves and be asked to choose firing angles in the same fashion it would have in a real Robocode battle.

Classifiers

The Classifier interfaces implement a targeting algorithm. It's pretty simple. The main methods shared by all classifiers are:

• initalize() - Set up data structures, clear any existing data.
• classify(WaveRecord) - Attempt to classify (produce a firing angle) for this wave.

These two methods can specify alternate schemes for training:

• boolean refeed() - Instead of feeding all unread waves, feed all waves from the beginning of the battle. Some algorithms (though perhaps not suited to actually using in Robocode) require the entire data set; this saves you from needing to handle that in your Classifier.
• boolean feedOnce() - Feed the entire data set once, up front, before classifying the firing waves. Useful for testing algorithms that take a long time to train, since it's much faster than refeed.

There are two Classifier subinterfaces - one of these is what you'll probably be working with:

• PreciseWaveClassifier - Uses precise intersection and processes waves once they have completely passed the enemy bot. Includes the method: feed(TickRecord, PreciseWaveEndRecord)
• TraditionalWaveClassifier - Records only the bearing to the center of the enemy bot once a wave has passed the center of the bot. Includes the method: feed(TickRecord, TraditionalWaveEndRecord)

Downloads

WaveSim

• WaveSim 2.0.0 - Source, .class files, Javadoc, and one sample data file.

If you put this on your build path, you should be able to:

• Set WaveRunner.BASEDIR to point to your data files directory.
• Implement the PreciseWaveClassifier or TraditionalWaveClassifier interface, which is your targeting algorithm. Methods for initialization/reset, training a wave, and classifying a wave. (See the Javadoc for reference.)
• Start simulating battles!
• (You may need robocode.jar on your build path, as well, if any of your gun code needs it.)

The doc directory in the JAR describes many of the classes, but the main ones you need to know are WaveRunner, TickRecord, and the Classifier interfaces. Of course, you'll need to gather or download some data first (see below).

There is one sample data file in the JAR at voidious/wavesim/cx.mini.Cigaret 1.31_1_wavedata.bin.gz. Unzip the JAR and try: java -cp ~/robocode/libs/robocode.jar:. voidious.wavesim.WaveRunner -- this will simulate a battle vs Cigaret using the Head-On Targeting classifier and the KNN classifier from Diamond's main gun. (Diamond's gun is what needs the Robocode JAR.)

TripHammer R

These versions of TripHammer will output gzipped binary data files containing all the wave data collected during battles. They are in the format used by WaveRunner. Make sure to up your disk quota in robocode.properties. (robocode.robot.filesystem.quota=2000000000 will give you 2 gigs, enough for a couple thousand battles or so.)

• TripHammer R2.0.0 - Diamond's movement and main gun.
• TripHammer R2.0.0RM - Random movement (ripped from Jen) and Diamond's main gun.
• TripHammer R2.0.0T - Diamond's main gun in TC mode.

The RM version will execute a lot faster while still giving data from real battle scenarios. Your best bet may be to plug in your own movement, since that's really the data "demographic" you'd want to optimize your gun for. You might also consider updating the bullet power selection to match your own.

Data sets

• dibed25_4seasons.zip (2.0 GB) - 4 seasons of R2.0.0 vs a field of 493 bots across the whole range of the RoboRumble 1v1 population.

Blog

4/13/2011

Updated everything to 2.0.0. Big thanks to Rednaxela for making a huge code contribution in this version! Primary changes:

• Multithreading! Make full use of those fancy multicore processors. Also loads data from disk in a separate thread.
• New file format that outputs binary files and lays the foundation for supporting non-wave gun stuff, like Pattern Matching and Play It Forward. Should be recording all the data needed for those right now, but will need to update the APIs a bit more to make it work nicely.
• Support for precise intersection.
• A fair bit of code restructuring and added/updated documentation.

1/26/2011

Updated everything to 1.2.0. Primary changes:

• Includes traditional MEA and GuessFactors. (Still has precise MEA versions too.)
• Includes orbital Wall Distance, as used by most bots.
• Data files are compressed.
• WaveRunner.BASEDIR is no longer final (doh!), so you can actually use the JAR correctly.
• Option for Random Movement in TripHammer R for faster data collection.

1/25/2011

Been revisiting this, adding stuff that other people would probably need before it's of any use to them, like normal MEA and Wall Distance attributes. Almost ready for a new release. I also plan to add an option (on by default) to use random movement instead of Diamond's wave surfing, for faster data collection, and I'll probably zip up some gigs of sample data for download.

Of course, I ended up getting derailed using it to tune Diamond's gun first. =P

Just cleaned up this page quite a bit too.

3/17/2010

Posted first public version of my WaveSim utility. Have fun. =)

Of course I quickly realized that I overlooked something. The WaveRecord class also needs to be documented. And this uses precise MEA and wall distance based on precise MEA. I should add options for traditional MEA and orbital wall distance.

3/12/2010 pm

Super busy with work, so I can't focus on this much, but I've got the basics working now. For 100 seasons of TCRM:

TripHammer KNN classifier:
Took 1262.0 seconds
Hit percentage: 11.813935415920957
Energy ratio: 0.983354371037666

HOT classifier:
Took 341.0 seconds
Hit percentage: 8.739709205199102
Energy ratio: 0.7279814945288104

HOT score seems pretty high, but it gets 5.9% vs DT, which seems right. At avg distance of 500, you'd get about 5% against a perfectly flat movement.

Best of all is < 1 second per battle for Diamond's main gun. A real TCRM is 15-20 seconds per battle for this gun. Still have some obvious little optimizations to add and there are probably some bigger ones once I focus on it.

3/12/2010 am

I'll post all this soon (except the 4 gigs of data), if anyone's interested in tinkering with it. I may also post info about some of the clustering schemes I've tried. An example of one of the really intensive clustering algorithms that I find interesting is QT clustering. A couple of my previous experiments were inspired by that one.