Talk:Diamond/Version History

For old discussions, see Archived talk:Diamond/Version History 20110905.

(Maybe I should see how this fares before babbling about it, but anyway...)

How this works is: for each attribute, I have an initial weight, a final weight, and a final time (say, 20,000 ticks). The weight shifts linearly from initial to final value until final time, when it stays at final value. Tuning against my 500-bot, 4 season general rumble test bed, I increased my hit percentage from 23.21% to 23.90%. Not sure how much I could've gotten just re-tuning without the shifting, but testing just the initial weights or just the final weights came in around 23.1%. And the previous weights were genetically tuned in a similar fashion, so I think they were close to optimal.

How this came about is I was looking into a custom gun just for the first round of a battle, the way I've been using RetroGirl/Gun for the first few shots. I tried just tuning a KNN gun on 1-round battle data, and the result quickly outperformed my current settings, and the weights looked a lot different. After starting to implement it in Diamond and seeing promising (if incomplete) results, I thought of the more general solution of gradually shifting the weights. This too outperformed my current settings after very few generations. One thing that stands out is that my gun heat dimension starts at a very low weight and ends with a pretty high weight late in the match. This seems to make intuitive sense: as I have more data, it's better to favor firing waves more.

Now to cross my fingers that it produces in the rumble. =)

Neat concept! Now this has got me thinking about trying to associate KNN weightings with all sorts of things besides time... distance maybe?

What do you use for your test bed? RoboResearch?

I've been wanting to systematize my testing more so I can shake things out more thoroughly in an automated way before I throw it up on the Rumble server. I've got RoboResearch ready to go.. I just need to assemble a set of bots to test against.

Since he's using genetic algorithms to optimize it, I assume he's using his WaveSim rather than RoboResearch for speed reasons. Trying to use RoboResearch with genetic algorithms would more or less be insanity I think ;)

Yeah, I use RoboResearch for real testing. For some gun-only testing (like this) I use WaveSim, which is a tool I wrote to test just classification against raw battle data - so it doesn't work for testing bullet power or some other nitty gritty things, but for the things it does work for, it's pretty sweet (and fast).

For test beds, I use User:Voidious/BedMaker, which is a little script I wrote to select random bots from the rumble within certain parameters. But you'd need to get a rumble server API key from Darkcanuck first. I've been thinking maybe I should make a web-based version of that for others to use freely, but I wasn't sure if anyone was interested...

Currently hunting down which of a bunch of changes between 1.7.29 and 1.7.35 caused a decrease in performance. I swear I tested rolling back each one individually yesterday and none of them fixed my score. So now the other way: start with 1.7.29 and add each change individually. Once that's done I can see if this fading KNN stuff actually helps.

I've been pretty fearless with continuous refactoring and bug fixes throughout 1.7.x, and this is the price I pay for it. =) But overall I think it's been worth it, both in terms of code quality and performance.

Orrrrr I've been chasing ghosts... In my 2,000 battle benchmark (250 bots x 8 seasons), 1.7.29 came in 0.18 above 1.7.37 and a dev version of 1.7.35 got the exact same score as 1.7.37. Rolling back individual changes gave me anywhere from 0.14 to 0.22 below 1.7.29's score. Then I recompiled 1.7.29, checked the class files came out the same (ie, I had the right source), and reran 10 seasons... and came in 0.14 below 1.7.29.

Another good reason to focus on big improvements: anything else is too small / painful to reliably benchmark. =)

I'm really curious to see how "cast shadows over firing angles that already would have hit" works out. This is actually a feature I had implemented in RougeDC a long time ago, but I can't remember if I saw much benefit in the rumble or not.

Yeah, it seems like a cool feature, and it doesn't seem possible it would cost you points unless there is some other bug or quirk at play. Looks like I lost 0.1 APS, which could be margin of error, but I didn't really reproduce a score increase in tests either, so I think it's accurate. Another nice thing this would let me do is surf a wave until it completely passes without sacrificing anything. If I'm modeling the danger accurately, I'll pretty much be surfing the second wave as soon as I am now anyway, and even a tick or two sooner in a lot of cases where waves break along my front edge.

The one aspect that probably needs improving is how I apply shadows to the danger calculation. I ignore any firing angles that fall within a shadow, and I multiply the final danger by (1 - the percentage of firing angles that are shadowed). Of course this yielded like 0.8 APS with bullet shadows, so it must be a decent approach, but the right way to do it is obviously with integrals and all that. I think I need to change my kernel density formula to do it right, which is something I still need to figure out.

It's funny, I've thought about this idea before, but it always seemed like a hugely complex thing to try to deal with. This time I already had all the pieces in place to do it without much additional work: precise intersection, bullet shadows, plus a lot of newly refactored wave code that makes my life a lot easier. =)

Ahh, a pity it didn't seem to really help, though maybe it would more with the integral-style danger.

Makes sense. It was pretty natural for me to implement in RougeDC because it had both precise intersection and integral-style danger right from the start.

I'm also surprised this didn't work. With Goto surfing I could understand, as it doesn't make any accel/decel decisions once the wave starts breaking, but with True Surfing the decision to speed up/slow down is made as the wave is breaking over you. Are you creating the wave shadows in your simulations as well, or just using the ones which are actually created by the bot hitting the waves?

And you're making me nervous with your recent gains =)

I'm just creating shadows up to the present point in time, including the firing angles that will hit me next tick (before I can move again). I did consider that as an extension, I could shadow any angles that are totally unavoidable, starting several ticks earlier and simulating until the wave is gone, but I'm taking it one step at a time.

My best guess is that the crudeness of how I apply bullet shadows to my danger calculation is what's holding me back. My kernel density danger calculation just ignores any angles that fall within a shadow. But an angle near the edge of a shadow would otherwise cause me to go the other way, and that's probably worth doing since those angles are just guesses. With these shadows, I'm totally ignoring that firing angle. I'm going to try some integral style dangers and see where that takes me.

It's also worth noting that while it's true those angles really should be viewed as having zero danger, many of them are in common across the movement options, so they may frequently cancel each other out anyway. (Ie, what's the difference if I add zero or some other number to all the movement option dangers?)

I know you can't really say anything from 1 battle, but I don't ever recall seeing a score vs Shadow like this one. =) Yay!

My Anti-Surfer gun tries to predict the enemy's surfing on the nearest wave, then restrict its min/max firing angles to those still reachable assuming the enemy surfed the nearest wave as expected. For figuring out what's reachable, I hacked up my precise MEA calculation with a different starting state. But basically I screwed that all up and it hardly worked at all. (About time I started writing unit tests...)

Despite being broken, I had tested this gun against all my worst matchups and it helped against Shadow. So hopefully it will work better now.

Ok, looks like 1.7.9 finally fixes "the" major bug(s) I introduced with the movement refactoring in 1.7.5. I guess I won't know for sure if it was one of the 1.7.8 changes unless I let it fill out its battles, but it started off tanking pretty hard and the wrong GFs in the flattener (fixed in 1.7.9) is exactly the type of thing I thought would be the problem. (I switched to precise intersection in the flattener, since it was free with the new wave processing, but copied a couple lines from the gun that also used precise MEA GFs.) I'm really happy to have the wave interpolation and bullet shadow fixes too, though, and wonder if they're the reason 1.7.9 is undefeated (that or luck).

Regardless, I'm happy I can move on to the rest of the movement refactor. There's still quite a lot to do. :-) It's amazing I can refactor or rewrite every 1-2 years, and every time, the previous code seems so terrible...

While the RoboRumble results don't bear much evidence, I'm quite happy with the progress I've made against top bots recently (while maintaining or increasing APS!). I mainly focus on Shadow and DrussGT, but also against a test bed that includes 6 other strong bots. I swapped out Scarlet for Tomcat because at least twice I saw a score of 85-90 (adaptive bullet power gone haywire?).

    abc.Shadow 3.83c
    darkcanuck.Pris 0.92
    davidalves.Phoenix 1.02
    jk.mega.DrussGT 2.2.0
    kc.serpent.WaveSerpent 2.11
    lxx.Tomcat 3.17.169
    mue.Ascendant 1.2.27
    voidious.Dookious 1.573cNDS (non-data saving)

Results over 100 seasons (plus an extra 200 of 1.6.12 vs Shadow / DrussGT):

Diamond	Shadow	Pris	Phoenix	DrussGT	WaveSerpent	Tomcat	Ascendant	Dookious	Avg	Seasons
v1.6.4	50.20	51.43	56.41	45.74	52.06	58.94	56.05	51.58	52.80	100
v1.6.12	52.46	60.74	61.13	49.30	57.45	58.66	62.81	58.60	57.64	150

There are very strange results for Tomcat's last version, but his aps against Druss and Diamond is about 50% now

Congrats! Did you test it or just looking at the Rumble? 2-3 battles isn't very accurate... But I should update DrussGT and Tomcat to latest versions in any case.

FYI:

• 100 battles vs Tomcat 3.27: 54.15
• 300 battles vs DrussGT 2.2.2: 49.00

Edit: Oops, Tomcat's at 3.29? Doh!

I lost 7-8 places with the latest version, so congratulations are not appropriate in this situation:) No i did not test, but now there're 9 battles and APS is still about 50%:) And yes, i did my previous post to encourage you to get last version of Tomcat:) Good luck with PM crown hunting:)

FYI, take 2 - 100 battles vs Tomcat 3.29: 52.89. Very nice work!

Thank you, but for me is better -5 APS vs Diamond, but + 1 APS overall:)

So far, this hurts vs Phoenix and Dookious and helps vs Shadow, but that's a trade-off I'm willing to make at the moment. It brings me from ~49 to ~51 vs Shadow. As usual, nothing helps vs DrussGT... There's plenty more I can do to tune this, and I could even end up with multiple Anti-Surfer guns in my VG, but I've made enough changes that I just want a sanity release before going much further.