why virtual waves help

Mind you the firing waves are not totally independent, either, so maybe you should only collect a wave every 50 ticks? =) Just kidding... But I do agree with penalizing non-firing waves, and penalizing them more against surfers. Another good reason to do so against non-surfers is that in a KNN gun, you are only selecting so many data points each time you aim, so favoring firing waves will give you more unique situations. But I've never found improved performance by ignoring virtual waves completely. And hitting top surfers is kind of a black art in any case, and they're the only targets where my Anti-Surfer gun outperforms my Main Gun consistently.

Btw, WaveSim is a good platform for testing this kind of stuff. On my new quad-core box (with 8 threads), I can run my Main Gun through 10k battles of gun data in about 3 minutes. I'd be happy to post some more of my data sets if anyone's interested. =)

As an aside... I almost wonder if there should be a wavesim-based challenge with a huge dataset, to complement the full-robot challenges... I think that could be interesting.

well, the problem with wave sim, at least for me, is that I have some unique attributes (based on PPMEA) so I would need to rewrite a lot of it and collect my own data. I tend to think that once the data is collected, it would be faster and almost as easy to run a classification algorithm in a C++ program.

The point of such a challenge would be to focus on the classification algorithms with a standardized dataset that different people can try to get the best results they can with it. Compared to the conventional challenges that currently exist the results would be much more repeatable and smaller differences could be more clearly discerned.

Also, IIRC WaveSim does store enough information to derive most if not all possible attributes (including PPMEA based ones), due to the way that I added PIF support to WaveSim.

Yeah, you should be able to anything - I've written a Pattern Matcher with PIF, which is about as non-Wave as you can get. But to get the full speed benefits, and for actually improving your own gun, I'd definitely recommend collecting your own data with your own attributes and bullet power scheme. Adding attributes should be pretty easy, really, just adding them to the Wave class then the reads and writes - but then again I wrote the code. =)

I also have an updated TripHammer that I haven't posted yet, after the big Diamond refactor. The code is a lot cleaner and integrates better with Diamond's code, just modifying the gun data manager to write the data files. It also interpolates waves for skipped turns (as Diamond does now), which is kind of cool.

A WaveSim based challenge could be fun, kind of like the Netflix challenge. Not really sure how many would be interested, though, and we'd have to come up with some kind of rule set to prevent over-fitting. Maybe a training data set, then you submit your code to an App Engine instance to run against the real data set, which is how Netflix did it I think.