Talk:Movement Challenge 2K6/Results

From old wiki

CC 2eta was a test to see if I would gain or lose RR@H rating from concentrating on avoiding simple targeting at the expense of the more advanced targeting avoidance. I lost almost 10 points that way. Now CC 2theta uses two modes. One for simple targeting and one for more advanced targeting. The descision process is simple. If I'm hit more than X times / round after 5 rounds I conclude it's advanced targeting. Seems to work in this challenge. Let's see what happens in the rumble. I have a broken gun at the moment though which I am not sure I want to release... -- PEZ

That's an awesome result ABC. Seems like you should be able to see what made 3.66 so much better against the simple targeters and see if you can make 3.70 movement blend that in when it's advanced movement isn't needed. Then your total score would go through the roof. -- PEZ

Both of your recent scores are majorly impressive, and inspiring. I'm really jealous of CC's score against WaveSurfingChallengeBotC ;) Although a movement change that gained me 25 RR points also cost me 4 points against BotC, so I wonder how important it is compared to the other two. I'd be interested in seeing the rest of the top 5's MC for comparison, and Pear, too. Maybe I'll run some of those this week. -- Voidious

The difference between 3.66 and 3.70 against the simple targeters is all about good distance (and aproach angle) control. 3.70 gets trapped when some very rare starting conditions happen. It should never happen in a normal fight since most enemies move. I'm still trying to fix it, though. Against BotA, f.e., I see it sometimes dodging every single enemy bullet for over 100 rounds and then ramming the opponent or standing still for a whole round while eating all its bullets... -- ABC

Yuk, that sounds like you might take unecessary blows against any targeting. I have had similar problems, though I can't remember what it was caused by. -- PEZ

I had this exact type of thing happen when I took "getting closer" as a negative (like current / new distance ratio) into account in my movement options. I had some relatively complex anti-cornering code in Dookious for a while to deal with the effect of that, but removing all of it actually helped the RR rating at the cost of MC scores, if I remember correctly. -- Voidious

In case anyone's curious, the key to that low APMC score in Dookious 0.851 is this... I've got a hit-% threshold that decides which of two distancing formulas to use, and it's currently set higher than the hit percentage that Che gets in the MC. A quick adjustment to that decision puts it at 38.47, so hopefully the next version's MC scores will be a bit better =) -- Voidious

I don't usually post dev scores, but getting .01 above the 1st place tank, I just couldn't resist... -- Voidious

Excellent! Bloody excellent. -- PEZ

That 40+ APMC score wasn't really a fluke, as further dev versions did reproduce it consistently, but there was a significant bug in my CurveFlattener decision that I just had to fix. I thought I fixed it in a way that would preserve what it was doing against a PM gun (and for the APMC), but I guess it didn't, and it wasn't really top priority. -- Voidious

If anyone's interested, I had modified Pear to be a TC-challenger to run those tests, so I thought I'd share it for anyone that wants to use it (as an MC reference bot) for personal development: Pear 0.62.1MC ... It's a pretty tough gun against surfers. -- Voidious

It surprises me to see that I score better in the WSC2K6 part of this challenge than the #6 WaveSurfer of the RoboRumble. If Axe could pump that up, SilverSurfer would be back in the top-5. -- GrubbmGait

Simply awesome WSC2K6 scores, Corbos, nice work! -- Voidious

I just wanted to share some thoughts I had about the upper limit of MC scores. First, I figure that the maximum distance from the reference bot you could average is like 600 or less - it's 400 to a wall and 500 to the corner if the reference bot's in the center, or 800 to a wall and 1,000 to one corner if he's in the utmost corner. So 600 is just a rough guess, but let's go with it...

Given a flight time of 55 ticks at 600 distance, the full lateral range is about 900, or something like 25 bot widths. Given that, you could say that even random targeting would get 4% hit percentage against a perfectly flat movement; slightly more, given diagonal shots and multiple ticks that a bullet could hit with a speed of 11. Since my FloodHT scores in the upper 60's show a hit percentage between 4% and 5%, I'd say that's getting pretty close to the upper limit versus a learning gun. (PulsarMax and RaikoMX get the highest score at 67%.) I'd guess the upper limit versus any reasonable learning gun is around 70%, at least the way we're doing things now. I think only a paradigm shift like figuring out how they are learning, and updating your stats without getting hit, could yield higher scores than that...

(Yes, I think about Robocode too much... =) )

-- Voidious

Do entrants move to maximize their distance from the challenge robots? Staying farther away from shooters will result in a higher score, but won't represent the robot's true movement. -- Kev

Nope, entrants can do whatever they want, but you're right that that's a potential problem with the MovementChallenge system. I used to use custom distancing for Dookious when in MC mode, but I don't anymore; you're certainly welcome to if you want. PEZ tipped me off to the fact that staying a bit closer against really simple targeters is actually helpful, and I was able to confirm that in my own WSC tests. -- Voidious

I use the challenge as a benchmark for over-all progress so I maintain my Rumble distances. The bit about staying close to simple targeters is worth repeating. Note ABC's comments above, "...good distance (and aproach angle) control...". If you're dodging bullets and you're pretty sure where they're going, it's easier to dodge them one at a time. --Corbos

I've had some initial success with my VirtualMovement-ish idea tonight, and with very little tuning, so I'd say it looks promising. (It's not really VirtualMovement, but that's where I first posted the idea.) I'm already using a bunch of stat buffers in my surfing, so I am basically just keeping ratings on each of them for how "dangerous" they rate areas in which I'm hit, and how "safe" they rate areas where I am when a bullet misses me. Then I just disable the few that are lowest rated at the beginning of each round.

I've cracked 50 against CC once tonight in the CFC2K6, and 49 quite a few times - all personal bests, and possibly records among any of us for that reference bot. But there's still a lot of work to do in tuning and testing before any of this code gets released. It seemed a lot less "weird and stupid" to me once I got it coded and working.

-- Voidious

Just a quick note about the results for Phoenix 0.31: I'm not using a flattener. (yet?) =) --David Alves

I assume those WSC scores are 1,000 rounds, wcsv? Great work in any case. I'd like to hear back from you guys about the 500/1000 round issue - we definitely agreed on 500 rounds when this thing started, so I'd like to get back to that if we can all agree now. Cheers. -- Voidious

500 is ok by me. --David Alves

The results of GrubbmGrb are from 500 rounds (5 seasons), so fine with me. --GrubbmGait

Has anyone thought of having a /ResultsFastLearning for the movement challenge? It seems just as relevant (if not more) as the results for 500 round matches. --David Alves

Yeah, that makes sense. The only problem I have with 35-round batles is that it seems like you need many more total rounds to get an accurate result with shorter battles, but I don't have hard numbers to back me up, just my general experience. -- Voidious

So you're saying you think that it 15 seasons of 35 round battles (~500 rounds) provides less stable results than a single 500 round match? --David Alves

Yeah, I think that's the case, but I could be wrong =) But it is also undoubtedly more representative of the RoboRumble to test over 35-round battles. -- Voidious

Got any tips on how to improve APMC scores? Phoenix 0.752 is very strong in the CurveFlatteningChallenge2K6, but is only mediocre in the APMC (scoring around 35). --David Alves

Ya know, I've had some success with the APMC, but I really have no concrete advice for you. I think any success I've had has come from trial and error with tuning my segmentations. -- Voidious

Given my results, I'm not sure if Perpy is wavesurfing or not. I'll have to look into changing how it works - currently, it's 'true' wavesurfing. I think I'll switch that around to go-to style, with a simplified version of precise prediction. I'm fairly sure I could write the code for it, and it might work better. --Jp

Well, WaveSurfing could mean any type of movement that stays perpendicular to bullet sources... However, you should be able to crack 95% against BotA if you have a bug-free WaveSurfing movement, and probably more like 99% once you get it polished a bit more. Instead of saying the same thing, I'll just paste a quote from PEZ, from PowerHouse/OldChat:

(Pasted here by Voidious) You probably have a bug or two in your surfing. Tweaking buggy surfing can look like it works a while, but then it just drives you crazy instead. I suggest you try this:

1. remove all segmentation and anything that even looks like tweaks and then work with bug hunting your movement until you are closing in on 99% against WaveSurfingChallenge Bot A.
2. then apply segmentation on distance, velocity and acceleration and go for near 98% against WSCB.
3. then release the new beast onto the rumble and start tweaking/adding features to see what works and not. Also in this phase use the MovementChallenge2K6 heavily.

All the while you are struggling with this, remember that it is not called WaveSuffering for nothing. =)

-- PEZ

I'd also recommend checking out the WaveSurfing/Tutorial page (well, I wrote it :P), and the EnemyWaves and GamePhysics pages are good references, too. Good luck.

-- Voidious

PEZ's method is definitely the best way to go at it. If you can't avoid HOT nearly perfectly with no segmentation then you're probably not really WaveSurfing at all <EDIT: actually, you're definitely not>; buggy surfing can make a pretty effective random movement, but it won't get you to 2k. I struggled with bugs in PowerHouse's surfing for a *long* time before I finally got it working, so keep at it. --wcsv

I am testing a new movement and it seems to me that the bots B and C assume that you will move at full speed, rather than taking your present speed. I hadn't anticipated that. I do however have an alternate movement that works just fine against those simple targeters (though it sucks vs. head on). -- Martin

I'm pretty sure that's not true, they use your present speed. The source for C is at WaveSurfingChallenge/BotC, and it clearly uses present velocity. The source for B is in the jar; I'm too lazy to look, but I'm 99% sure it's the same, and C was based on B in the first place. -- Voidious

[Edit conflict]Nope, they calculate things with your present speed (actually they fire with the calculations of the previous tick, like all simple targeters), so if you move at speed 1 in a straight line, they will hit you for sure. But if the calculations would end up outside the battlefield, they fire at the last calculated point inside it with full power. The source of WSCBotA, B and C is inside the jar btw. -- GrubbmGait

Hm.. I thought I was coming to a stop before they fired at me, in which case they should fire directly at me instead of leading me like they are. I've got some more debugging to do it seems. Edit: I am indeed getting some unexpected velocities. I am trying to travel 48.0 units tangental to a firing position (or close to it) and that should take 11 ticks. For some reason I am getting speeds higher than 2.0 rather frequently, sometimes even higher than 7.0, when the next shot is fired. I guess I'll have to do it a little differently. -- Martin

Great scores in the WSC2K6, Martin! I hope it translates into some RoboRumble points for you. -- Voidious

Thanks. Tracking down bugs at this level really required powerful data collection tools. I've had to literally trace the path by movement wants my bot to go (in green) and then trace the path it actually goes right on top (in orange), looking for deviations (any green showing). Then I had to track the same data (and more) in formatted debug lines so I could see exactly where (and why) the deviations were occuring. I've gotten it really close, but there's only so many hours in the day to track this stuff down. More to come .. -- Martin
Actually, Ugluk's initial results are tanking hard. Dunno why. -- Martin

Perpy 16.0 looks to be doing better then 11.0, on the average, in the MC. It incorporates lots of fixes to wavesurfing, and stays further away... but it actually does WORSE against HOT then 11! And that Che score is puzzling - it did about three times better in version 11. Interesting. Nevertheless, it's a better bot overall - I'll put it in the rumble, now, methinks. --Jp

The best way to know is indeed to put it in the rumble. I do not exactly know how you come to 28.xx total score, when I count up your scores (55.29+5.33+9.27) I come to a total score of 23.30, but the MCCalc should do this for you. Being better against HOT, LT and CT often means being more predictable (at least for me), but that can be solved by better/different segmentation (I hope) -- GrubbmGait

Tuning for the APMC is still very mysterious to me, I attribute all my success with it to simple trial and error and just having a bug-free WaveSurfing that is well tuned. You'll find the same type of advice all throughout the wiki, but it's worth repeating here: your best bet is to focus on the simple targeters until you get 99% against HoT and 97-98 against the other two. First, there are a lot of simple targeters in the rumble; second, it will mean you have worked out the basic bugs in your WaveSurfing. Once you have worked out the bugs, tuning it well against stronger guns is a lot easier. -- Voidious

I didn't actually use the MCCalc script - I thought it was average of the scores, not the sum of the individual averages?

But yes - I'm focusing on trying to fix my wavesurfing. I was just puzzled by the serious drop in the PM score. --Jp

Yes, it's an average of the averages, even though that's kinda counter-intuitive. (The idea is to equally weight the scores against simple targeters, PatternMatching, and GuessFactorTargeting.) -- Voidious

Don't worry about your score against pattern matchers. Don't worry about anything except your score against WaveSurfingChallengeBotA. Use a recent version of robocode (for onPaint), and extend DrawingBot. Draw everything onto the screen: where incoming waves are, where your predicted positions are, how dangerous different positions are, etc. Use this to find and fix the bugs in your WaveSurfing. Once you have over 98% against WaveSurfingChallengeBotA, work on WaveSurfingChallengeBotB. When you're above 95% against WaveSurfingChallengeBotB, then and only then should you worry about your scores against any other bot. Trying to dune a buggy wave surfing system won't get you anywhere, you have to get rid of the bugs first. Good luck! --David Alves

Insane WSC scores, Martin, as promised =) Is there still nothing adaptive in that movement? -- Voidious

• It isn't doing anything that you'd call surfing. It switches between two movement styles when it is hit until it has done each 5 times, at which point it only changes if the average time spent using the primary (while bullets are in flight) dips below the next best option. The secondary movement is a rather simple tangental oscillation (with support of anti-wall and anti-robot fields) while the more complicated movement (which does best against dead-on) is similar to a minimum risk movement system, only the evaluation of risk takes the form of checking if the route to the point will cause the robot to be hit by predicted firing angles (in this case only dead-on). It is fun to watch it dance against Bot A. The Grid movement is also fairly random and holds its own against pattern matchers and guess-factor targeting. -- Martin

Release it! =) --David Alves

It's nice to know that it doesn't do too well against CassiusClay or FloodHT. =) (Makes me think Dookious could handle it just fine.) That 54 against Shadow impresses me more than the score against Che, actually. You should get those WSC scores up and release that baby. -- Voidious

OOPS!! Sorry, I had the scores for Flood & Shadow reversed. (*embarased look*). That should make more sense. The only reason this does well is because I reverse pattern match Che, don't expect good rumble results. That's why it's cheating. -- Simonton

Ah, I feel a little better now. =) Yeah, I know it's reversing Che. But if anti-Che gives you a really good Anti-PM in general and you can maintain the general goodness of WaveSurfing against simple targeters, it could give a significant leg up, as a lot of bots (especially micros and smaller) still use some form of PM. -- Voidious

Ok, there. I added in avoidance for bots A, B & C. We'll see just what happens in the rumble! I actually am somewhat optimistic. It's a little buggy I think, and the wave surfing lacks one feature from WeeksOnEnd, but I think the anti-pattern matching is actually going to help. Specializing against head-on-targeting should help, though I haven't yet taken into account HOT bots that don't use setAdjustGunForRobotTurn. Same goes for specializing against linear & circular. There are a couple flavors of linear & circular, too: ones that take into account walls and ones that don't. However, the wave surfing should pick up the linear ones that don't, and I bet most circular targetters do take walls into account (anyone care to correct me?). So let's see what happens! Now HERE is a test of whether coding for the movement challenge will help your rumble scores :). -- Simonton

Wow! WeeklongObsession's new RandomMovement beat WeeksOnEnd's WaveSurfing in /ResultsFastLearning! (Which is the only one that counts in the mini or micro rumbles.) I wish I could put it in the rumble. -- Simonton