Talk:Movement Challenge 2K7/Results

From old wiki

Well, it's official, RoboResearch can run this challenge and spit out results to copy-paste to the wiki. I'll replace that bogus score with a real one sometime ... I just wanted to show off. -- Simonton

Spending a week of late nights on RoboResearch: tiredness. Running 50 fast learning seasons in 5 minutes: priceless. -- Simonton

I'm glad to finally have this challenge to test against! I've been wanting to push my new RandomMovement system further along. I think my next step is going to be a good one ... but we'll see. -- Simonton

I just ran 3 500 round battles against Ascendant. The scores: 29.13, 25.87, 21.93. This is unacceptable variance. There is nothing in Ascendant's data directory, so I don't really know how to explain it. -- Simonton

• Hmm - that does seem like a lot. One thing about Ascendant is that it sticks to its VirtualGuns choice permanently once it decides. That could explain some variance, but I don't really know. I can say this much, Ascendant has a KILLER gun - I bet it's only extreme cases where it makes a very bad VG choice. Still, I don't find this variance totally shocking; in 35-round battles, it can take hundreds of battles to stabilize (this is mostly based on testing Dookious vs various bots). It's also entirely possible that Raiko's gun is particularly inconsistent against Ascendant...? Are the other reference bots much more consistent? -- Voidious
• I was testing against CC, but that takes too long. However ... it's probably saving data, isn't it. That's bad. Running against Thorn right now for fast battles, I'll run a few and post scores. -- Simonton
  • Yes, CC saves VirtualGuns data - doh, we should probably disable that for the reference bot version, eh? Should be a super easy fix (I could do it if you want). -- Voidious
  • Please do! Results against Thorn are pretty consistent. 60, 62, 62. I wouldn't expect any different. But I think I'll keep using him as my reference bot for now. My access to the 100 machines has been taken away (I knew it was coming eventually). This 1 machine at a time business is no fun. -- Simonton

Posted one season for Dookious - I'll run more, but it's a pain tallying this stuff manually, so I'll try and update the MCCalc script to handle MC2K7 RoboLeague results correctly. One thing I think should be noted somewhere is that the RaikoGun class that's included has radar handling code in it. I commented that out because I already have radar code in Dookious that will never slip, while that infinity lock would slip sometimes. I don't know how much that will matter for results, but it's something to be aware of. -- Voidious

• Doesn't it say right in the RaikoGun source not to use a different radar? -- AaronR
• Hmm, does it? That seems kind of silly - I don't consider radar part of the gun. Having a buggy radar as mandatory will only make the results more random, won't it? -- Voidious
• It's not an infinity lock, it's a pretty standard radar lock, the same one you have in Komarious if I remember right. If it would ever slip, it would be VERY rare, and I figured it couldn't help to have everyone use their own radar code, in case their code was buggy. I see no problem with it if someone is sure theirs won't slip, though. --Simonton

I used RoboResearch, and it spat out the results in wiki format, results all tallied for me. And I also used my own radar, for what it's worth. -- Skilgannon

Hey Voidious... are your MC2K7 500 results consistent over multiple seasons? It's kinda weird that I'm relatively close to you in the RRAH but way behind in the MC2K7. --David Alves

I'm not sure - I only ran one season because it's such a pain to tally these scores manually. =) I'll run more when I setup RoboResearch, or you're welcome to run them yourself if you want. I can send you the MC2K7 enabled Dookious. -- Voidious

If you send me the MC2K7 enabled Dookious, I'll run some FastLearning seasons. --Krabb

Ouch! I just added a whole bunch of new buffers (and cleaned up my buffer system), but (as you can see) my score against Ascendant took a plunge. I improved quite a bit against the bottom bots, but against the top bots I took a real plunge. What do I need? Some exotic segments, or a flattener, or what?!! -- Skilgannon

First, I'd play with the rolling depth on your surf stats - my advice is to roll them really fast, possibly even with a rolling depth less than one. Second, yes, a flattener is very helpful against some top guns, especially Ascendant. One version of Dookious I remember would go from 54% to 66% if I hardcoded the flattener to stay on all the time (this is averaged over many many battles, not just one). You've gotta be really careful with a flattener, though, as it will crush your scores against weaker bots to turn it on when you shouldn't. -- Voidious

That's the rub - my stats go from a rolling depth of 5 (unsegmented) to 3 (lateral velocity only) to 1 (distance and lateral velocity) right down to 0.2 (fine distance, fine lateral velocity, acceleration, fine time since dir-change). How about a flattener that enables when I have less score than the other bot? That should be fairly easy...just log hits (with a fairly low weight) for every single (real) wave, irrespective of whether it hits. -- Skilgannon

Try lowering the 5 and 3 to one, see what happens. Also, you might want to add a buffer (or two) that's more segmented than your current most segmented one (assuming the one you mentioned is it). Phoenix's most segmented buffer currently has 6 dimensions. --David Alves

I just tried that, but it didn't make any difference. Then I discovered that all buffers were being weighted equally (hiseg to noseg) due to a bug in my compiling a 'best buffer' code. I fixed that, so now I'm weighting buffers according to how many 'nodes' (is that the word used for VCS?) they have. I also added a time-since-vchange segment, and a 'absolute value of the sum of signed lateral velocities last 20 ticks', though they aren't helping so I've disabled them. These scores seem much nicer =). Especially the CC score =) -- Skilgannon

Oh yes, I'm also weighting each buffer by the number of hits it has, a max of 3. That seemed to be worth a percent against Ascendant, though it's tough to say because of Ascendant's sometimes-wrong VG system. -- Skilgannon

After upgrading my debug-painting system to show the danger on a wave, I watched a few games against HawkOnFire. It frequently doesn't shoot completely HOT, and this seems to be where I'm losing points against him. Maybe my surfing is just too pixel-perfect, but has anybody else seen this? The main problem seems to be when I get a bullet-hit-bullet against one of his bad shots, and it corrupts my stats. -- Skilgannon

• This problem definitely exists in the WaveSurfingChallenge bots, too. Considering it's probably a common issue with HOT bots, it's probably something you should deal with in your movement instead of dismissing it as an issue unique to HawkOnFire. Just because your movement is pixel perfect doesn't mean you should stop right next to where an HOT bullet would pass. =) (Well, for example.) -- Voidious
• Wait a second - do you still get 99.87? Man, I wouldn't be worrying about it. :-P -- Voidious
• Does HawkOnFire's radar slip, by chance? That'd be the only way I could explain it. No, you should also check whether it sets the adjust-gun-for-robot-turn thing; that would also explain it. In LifelongObsession I actually make a "head-on-targeting-without-adjusting-for-robot-turn" buffer (note that it did not gain me any points in the rumble ... or maybe like 2 at the most). But I'm very surprised (and a little skeptical) about the WaveSurfingChallenge bots. -- Simonton
• I think Martin discovered it in the WSC bots when he was doing his Grid movement, but I can't find the reference right now. Definitely a gun turn issue, but maybe it wasn't with Bot A. Anyway, Barracuda doesn't shoot true HOT either (its HOT is a bug, it has a broken learning gun) - you should be able to still dodge it really well (see WaveSurfing/BarracudaChallenge). -- Voidious
• Every now and again I see HOF shoot WAY off HOT, it looks like about GF 0.5, or -0.5. It might be when his radar slips, I've seen that happen as well. But yeah, I agree, I should be able to dodge him anyways =). The way I look at it, if I'm getting hit at all by a HOT bot, then either he's not shooting HOT or I've got a bug in my movement. And about Martin's comment, it is on the WSC2K6 page, and it had to do with the gun not completing it's turn before the bullet was fired. -- Skilgannon

Thanks for running Dookious, Krabb! I was going to get around to that sometime ... -- Simonton

Yeah, thanks for running that, Krabb! I was, too... =) -- Voidious

---

So, I accidentally ran an MC2K7 with a non-firing Phoenix... the results were interesting:

davidalves.Phoenix 1.02 (nonfiring)

David Alves

WS

99.84

76.75

79.31

85.30

70.77

64.34

83.43

73.88

50.76

57.58

57.30

55.21

71.29

90.2 seasons

So apparently does way better (wins!) against CassiusClay, Ascendant, and Chalk by not firing than it does by firing with RaikoMicro's gun? Running a real one now... --David Alves

• I don't know if I believe you can beat the rambot without firing ... -- Simonton

That's fun! :) I've always wondered why everyone always fires every chance they get ... seems like ignoring a perfectly legitimate energy management choice. Some day ... some day I'm going to try highly variable, dynamic energy power management based on segmented hit percentages. Perhaps that day will include a "don't fire at all" choice. -- Simonton

Just for reference, Phoenix 1.02 had a final score of 70.23 in the /FastLearning category with 279 seasons run. When it was at 50 seasons, the score was under 70. The highest point was at around 100 or 150 seasons, when it briefly was at 70.52. So it's important to run a lot of seasons if you want accurate results. I suspect this is because the individual score against Waylander contributes so much to the final score. If there were 2 bots in the PM category scores would be a lot more stable. =P --David Alves

Nice scores all around for the new DrussGT! I saw the subject of your post before I saw what page it was on and I got a little nervous for a moment. =) -- Voidious

I suspect you should by nervous anyway, Skilgannon is showing all the signs of a serious throne candidate! ;) -- ABC

I think I'm still a bit behind on the targeting....but be nervous, I'm comin' to get ya! ;-) -- Skilgannon

You could slow down a bit to give me time to catch up, Skilgannon! -- Simonton

There was nothing new there, just an old version with some bugfixes applied. I think I got one bug but missed another. Wavesuffering only gets worse as your bot gets better... it's just that after a while you learn to ignore it. But it's still that little voice in the back of your mind, making you think there's more bugs in your movement. -- Skilgannon

Misc chat

Hmm, I just realized that Dooki's MC2K7 code resets the RaikoGun each round. I wonder if fixing that will affect the score much. =) --Voidious 03:32, 13 July 2009 (UTC)

But... isn't the targeting buffer static? So it shouldn't matter if you create a new instance? --Skilgannon 14:13, 13 July 2009 (UTC)

Ah, cool, I didn't think of that. I was actually worried the reset might have improved my scores against the surfers! --Voidious 14:34, 13 July 2009 (UTC)