WaveSim

Background

To facilitate a new wave of gun research, I created a system to benchmark a gun's accuracy using pre-gathered battle data. This can take the place of running a zillion Robocode battles.

Basically, we just record all the stuff guns use to learn and to aim - input attributes like velocity, wall distance, acceleration, and outputs like GuessFactor and firing angle - then run that stuff directly through our targeting algorithms. Testing classification algorithms against raw data sets is nothing new or special, but it's about time someone set up a solid system for doing it with Robocode!

Benefits

• By removing all the overhead of running actual battles, you can run a gun benchmark much faster -- like minutes instead of hours.
• Tests can be run against data from real battles instead of a TC scenario, which should be more relevant to real battle performance.
• Every benchmark runs against the exact same data, resulting in direct comparisons of test results without worrying about randomness.
• You can see hit percentages or energy ratios (spent vs gained vs damage), which I think are a more direct measurement of a gun's accuracy than TC results.
• The huge speed increase and the ease of reconfiguring guns between tests really lends itself to more complex development options, like genetic algorithms and other long-running parameter tweaking.

Disclaimer

Note that a good classification algorithm is not the only important part of a gun. But many top bots have the rest of their gun code polished to a point where classification is the thing we work on most of the time.

Components

Data-gathering bot

A modified TripHammer outputs data for every tick and gun wave it gathers to a gzipped binary file. Each battle gets a new filename and TripHammer writes to it throughout the battle.

You could, of course, modify your own bot to output wave data to a file of the same or similar format. Just take a look at the TickRecord class in WaveSim for guidance on the format, or look at/borrow code from TripHammer's TripWave.writeTickRecord() method.

Battle simulation

WaveSim records the raw attribute data for each tick and fires a wave for that tick. When the wave is collected, it records another type of record with the hitting angle from the wave. By reading these in the same order that they occurred, a targeting algorithm will learn from ticks and/or waves and be asked to choose firing angles in the same fashion it would be in a real Robocode battle.

Classifiers

The Classifier interfaces implement a targeting algorithm. It's pretty simple. The main methods shared by all classifiers are:

• initalize() - Set up data structures, clear any existing data.
• classify(TickRecord) - Attempt to classify (produce a firing angle for) this wave.

There are three Classifier subinterfaces - one of these is what you'll probably be working with:

• PreciseWaveClassifier - Uses precise intersection and processes waves once they have completely passed the enemy bot. Includes the method: feed(TickRecord, PreciseWaveEndRecord)
• TraditionalWaveClassifier - Records only the bearing to the center of the enemy bot once a wave has passed the center of the bot. Includes the method: feed(TickRecord, TraditionalWaveEndRecord)
• TickClassifier - Fed each tick as it happens and ignores waves. This is for non-Wave-based targeting algorithms like Linear Targeting or Pattern Matching, and may also be useful for Play It Forward firing angle projections. Includes the method: feed(TickRecord)

Downloads

WaveSim

• WaveSim 2.1.0 - Source, .class files, Javadoc, and one sample data file.

Basic usage

If you put this on your build path, you should be able to:

• Set WaveRunner.BASEDIR to point to your data files directory.
• Implement the PreciseWaveClassifier, TraditionalWaveClassifier, and/or TickClassifier interface to create your targeting algorithm. Methods for initialization/reset, training a tick or wave, and classifying a wave. (See the Javadoc for reference.)
• Start simulating battles!
• You may need robocode.jar on your build path if any of your gun code needs it.

In the JAR

The doc directory in the JAR describes many of the classes, but the main ones you need to know are WaveRunner, TickRecord, and the Classifier interfaces. Of course, you'll need to gather or download some data first (see below).

There is one sample data file in the JAR at voidious/wavesim/cx.mini.Cigaret 1.31_1_wavedata.bin.gz. Unzip the JAR and try: java -cp ~/robocode/libs/robocode.jar:. voidious.wavesim.WaveRunner -- this will simulate a battle vs Cigaret using Head-On Targeting, Linear Targeting, Circular Targeting, and KNN classifier from Diamond's main gun. (Diamond's gun is what needs the Robocode JAR.)

TripHammer R

These versions of TripHammer will output gzipped binary data files containing all the wave data collected during battles. They are in the format used by WaveRunner. Make sure to up your disk quota in robocode.properties. (robocode.robot.filesystem.quota=2000000000 will give you 2 gigs, enough for a couple thousand battles or so.)

• TripHammer R2.1.0 - Diamond's movement and main gun.
• TripHammer R2.1.0RM - Random movement (ripped from Jen) and Diamond's main gun.
• TripHammer R2.1.0T - Diamond's main gun in TC mode.

The RM version will execute a lot faster while still giving data from real battle scenarios. Your best bet may be to plug in your own movement, since that's really the data "demographic" you'd want to optimize your gun for. You might also consider updating the bullet power selection to match your own.

Data sets (offline)

• dibed25_2.1.0.zip (2.1 GB) - 4 seasons of R2.1.0 vs a field of 493 bots across the whole range of the RoboRumble General 1v1 population.
• dibed41 (25 seasons): 25 seasons vs 400 bots from the top half of RoboRumble General 1v1, split into 5-season chunks (2.3 GB each).

Blog

6/23/2011

Updated to 2.1.0. Now supports non-Wave targeting algorithms!

• Added TickClassifier, which feeds attribute data for each tick as it happens, instead of only after a wave breaks. This allows for better implementation of non-Wave-based targeting algorithms, like Circular Targeting and Pattern Matching.
• Added absolute heading attribute.
• Now includes Linear Targeting and Circular Targeting classifiers for reference. These are good examples of non-Wave classifiers.
• Fixed a bug with gun heat. Gun heat and virtuality are for the tick after the rest of the attributes - ie, they are zero when a real bullet could be fired based on this information.
• Removed "refeed" and "feedOnce" methods from Classifier interface.
• Lots of cleanup of code and Javadoc.

4/13/2011

Updated everything to 2.0.0. Big thanks to Rednaxela for making a huge code contribution in this version! Primary changes:

• Multithreading! Make full use of those fancy multicore processors. Also loads data from disk in a separate thread.
• New file format that outputs binary files and lays the foundation for supporting non-wave gun stuff, like Pattern Matching and Play It Forward. Should be recording all the data needed for those right now, but will need to update the APIs a bit more to make it work nicely.
• Support for precise intersection.
• A fair bit of code restructuring and added/updated documentation.

1/26/2011

Updated everything to 1.2.0. Primary changes:

• Includes traditional MEA and GuessFactors. (Still has precise MEA versions too.)
• Includes orbital Wall Distance, as used by most bots.
• Data files are compressed.
• WaveRunner.BASEDIR is no longer final (doh!), so you can actually use the JAR correctly.
• Option for Random Movement in TripHammer R for faster data collection.

1/25/2011

Been revisiting this, adding stuff that other people would probably need before it's of any use to them, like normal MEA and Wall Distance attributes. Almost ready for a new release. I also plan to add an option (on by default) to use random movement instead of Diamond's wave surfing, for faster data collection, and I'll probably zip up some gigs of sample data for download.

Of course, I ended up getting derailed using it to tune Diamond's gun first. =P

Just cleaned up this page quite a bit too.

3/17/2010

Posted first public version of my WaveSim utility. Have fun. =)

Of course I quickly realized that I overlooked something. The WaveRecord class also needs to be documented. And this uses precise MEA and wall distance based on precise MEA. I should add options for traditional MEA and orbital wall distance.

3/12/2010 pm

Super busy with work, so I can't focus on this much, but I've got the basics working now. For 100 seasons of TCRM:

TripHammer KNN classifier:
Took 1262.0 seconds
Hit percentage: 11.813935415920957
Energy ratio: 0.983354371037666

HOT classifier:
Took 341.0 seconds
Hit percentage: 8.739709205199102
Energy ratio: 0.7279814945288104

HOT score seems pretty high, but it gets 5.9% vs DT, which seems right. At avg distance of 500, you'd get about 5% against a perfectly flat movement.

Best of all is < 1 second per battle for Diamond's main gun. A real TCRM is 15-20 seconds per battle for this gun. Still have some obvious little optimizations to add and there are probably some bigger ones once I focus on it.

3/12/2010 am

I'll post all this soon (except the 4 gigs of data), if anyone's interested in tinkering with it. I may also post info about some of the clustering schemes I've tried. An example of one of the really intensive clustering algorithms that I find interesting is QT clustering. A couple of my previous experiments were inspired by that one.