Archived talk:Pattern Matcher Challenge 20091015
|
Can a bots gun performance (PMC-wise) somehow be measured in the bullet damage it inflicts on PatternBot in, say, a 100 round match? It seems so to me. If this holds I propose that this be used for a /PMCIndex. Also it would be that this measure discriminates better guns since you by inflicting more damage prolong the matches so that a higher accuracy will have more time to increment your /PMCIndex.
- Yeah, bullet damage sounds like a good measure to me. Maybe we could add an extra column "hit percentage". I'll add a hit perc counter to PatternBot so we all get a fair (well...at least the same) measure. Maybe later we can decide which measure to use for sorting the table. -- Vic
- But the sort will be identical with bullet damage and hit ratio (asuming PatternBot can measure this with enough accuracy). -- PEZ
- It could be, but i'm not sure of it, because in the hit rate I only count from the moment PatternBot goes into its pattern. I do this because it want to measure only the rate on predictable movement. I think in this case a bot that causes maximum initial damage (you can find out to which point it is travelling) and has a hit rate of 98% will do more damage than a bot that does no initial damage and has a rate of 98.1%. I guess this also solves the sorting problem... probably bullet damage is a better measure for performance in the challenge. But still i'm very interested in the hit rates because that's the more fair indication of the true PatternMatching abilities. --Vic
- I realize now that i didn't fully understand what you where saying. You meant to say that better guns have longer matches therefore have more bullet damage. Of course PatternBot would than also have more bullet damage. A hit rate for the whole matches (including the initial part) would simply be: (challenger damage / PatternBot damage) * 100 . This would work because we know PatternBot always has a 100% rate. This hit rate would of course produce the same sort order as bullet damage. -- Vic
- But the sort will be identical with bullet damage and hit ratio (asuming PatternBot can measure this with enough accuracy). -- PEZ
I might get more bullet damage if GlowBlowPMC would stop shooting if he has enough energy left at the end. because it often occurs that it's Energy is above 10 when it had a 100% hitrate during a round. If I'd let PatternBot to come back I could gather more points. other improvements are file saving, to get better results the first rounds, also important might be that I adjust some things when the pattern cuts off (disabling, ramming, changing the round (already done)).
1st: rz.GlowBlowPMC 25459 3400 680 18560 2818 0 0 83 32 0 2nd: challenge.PatternBot 1.0 22558 1600 320 18361 1783 493 0 40 68 0
how about this :), you can see what I mean with the bullet damage. -- rozu
- I see what you mean! Actually it made me realize that when hit rates will increase to nearly 100% (meaning that you inflict a lot of initial damage as well) the PMC Index (bullet damage) will actually decrease because the matches get shorter. Your idea of letting PatternBot come back might be the trick here -- Vic
Tricking the Index like this would go beyond the goal for the challenge which is justing beating PatternBot. So maybe (your_damage / patternbot_damage) * 100 might yet be the better index after all. I just calculated it for Albert's last result, your two results and my two last results:
MicroAspid index 18514 relative damage 99.94 % GlowBlowPMC index 18752 relative damage 100.81 % GlowBlowPMC index 18361 relative damage 101.08 % EnderPMC index 18734 relative damage 100.43 % EnderPMC index 18916 relative damage 100.77 %
This shows that your last GlowBlowPMC has become much better and as mentioned before the index is dropping. It also shows that both your winning bots have a higher relative damage then my two not-winning bots. The current index doesn't reflect that. PEZ, would you agree we change the index to relative damage? --Vic
I agree that the /PMCIndex should be changed to relative damage. I would never had dreamt it would enter pattern matcher so good that they can even consider tricking the simple bullet damage index! -- PEZ
In my last version of EnderPMC there was a bug that resulted in HeadOnTargeting in 1 or 2 % of the matches, meaning EnderPMC lost those rounds with a 30% or so hit rate. This lowered the Index considerably, while EnderPMC did win the match! Also i noticed that there can be big differences in results over 100 round matches. If you're unlucky for example you get a few of those hitwall situations like Dummy describes above. Maybe if we do something like this again we should run 1000 or even 10000 round matches to even out the (bad)luck factor. Any thoughts about that? --Vic
Random chatting
Ender's PatternMatching gun is currently unable to win. I previously boasted a 99.45% accuracy, but i accidently read the percentage for the closest segment only... Actually Ender gets a 93.59% hit rate in its last 100 round match (according to PatternBot dev version) which sounds much more reasonable -- Vic
Cool - I' quite busy now, but I'w try ASAP (may be next week?) -- Albert
- Great! I would be very curious how your NN gun performs -- Vic
Wow, 99.45% is better than WallsKiller gets against Walls I think (closer to 92 to 93%. Of course, that's with power 3 bullets). -- Kawigi
I made a LeachPMC which got humiliated by PatternBot. Maybe a hit rate of 25% or some such... -- PEZ
I'm having lots of trouble hitting PatternBot at the start each round, while PatternBot is getting in position to start its patterns :-(. Perhaps I should tell my gun only to fire when a close-enough match has been found. --Dummy
- If you don't fire until the pattern starts you discard the advantage you are given. And you REALLY need it :-). However, when PatternBot reaches its starting point it will first turn to heading zero. This should be predictable enough to get one or two bullets on the scoreboard early. -- Vic
For symbolic pattern matchers, a special symbol should be used to indicate an empty frame from before the round started. This would allow pattern matching during these frames. -- nano
That's what LeachPMC does. It uses an ArrayList of Frame objects that look like so:
class Frame {
static double heading;
double headingDelta;
double velocity;
boolean isInitialPhase;
Frame() {
}
Frame(double headingDelta, double velocity, boolean isInitialPhase) {
this.headingDelta = headingDelta;
this.velocity = velocity;
this.isInitialPhase = isInitialPhase;
}
static void setHeading(double newHeading) {
heading = newHeading;
}
void advanceHeading() {
heading += headingDelta;
}
char getKey() {
int key = 3;
key = key + 100 * (isInitialPhase ? 1 : 0);
key = key + 11 * (int)((10.0 + Math.toDegrees(headingDelta)) * 3);
key = key + (int)((8.0 + velocity));
return (char)(key);
}
double deltaX() {
return Math.sin(heading) * velocity;
}
double deltaY() {
return Math.cos(heading) * velocity;
}
}
class BreakFrame extends Frame {
char getKey() {
return LeachPMC.BREAK_KEY;
}
double deltaX() {
return 0;
}
double deltaY() {
return 0;
}
}
The BreakFrame gets added at the beginning of each round. I have experimented some with adding that frame also when the radar slips and I get 4+ ticks to interpolate. Never happens against PatternBot when you are standing still though, but Frankie might try that approach again. -- PEZ
You seem to take it for granted that your radar will slip. I don't think this has to be the case. None of my development robots ever misses a scan unless it skips a turn, and the radar still doesn't slip even if that happens. -- nano
Maybe so. I'm probably thinking about the kind of radar I had a fe weeks ago. How if you print out the time since last scan in your scanned event? Print it only if it's > 1. LeachPMC then prints a few 2s an occational 3 and now and then a 4. About 15 times on average per round it seems it breaks the perfection. I then interpolate the missing tick and it works for what I can judge. Please tell how your robots fare here and we might learn something about radar management as well. -- PEZ
I am thinking how a about a RadomMovementChallenge,every competitive bots just can fire 0.5 power bullet to against a PatternMatcherBot (it can be bot with the best ranking in PatternMatcherChallenge). This can see which bot's movement is radomer. -- iiley
That would be way cool. But how about PatternMatcherBot can fire but not the challenger? Then it could fire power 1->3 bullets and it would be a better measure on movement since it's not all that productive to tune movement against 0.5 bullets. -- PEZ
Yeah! Shall we discuss this further on the RandomMovementChallenge page? This page is getting cramped :-) -- Vic
Results :
http://atlas.et.tudelft.nl/klli87/images/pmchallenge.gif ( -Dummy)
Here it go results for MicroAspid:
1st: challenge.PatternBot 1.0 26363 4850 970 16855 3299 388 0 97 3 0 2nd: apv.MicroAspid 15935 150 30 15647 107 0 0 3 97 0
I just removed a low pass filter from MicroAspid (it makes sense only when fighting against random movement bots) and this are the results :-):
1st: challenge.PatternBot 1.0 24183 2650 530 18525 2174 303 0 58 47 0 2nd: apv.MicroAspid 23289 2350 470 18514 1954 0 0 52 53 0
WTF! That's truly amazing Albert! How do you fit that in a micro? What are your max and min match lenghts? --PEZ
Whoa!!! That's really good! You're just a pig's hair away from completing the challenge Albert! I'm in awe!! --Vic
This reminds me of a time when I said a fully-featured pattern-matcher could easily fit into a Minibot. -- Kawigi
MicroAspid doesn't have min or max distances (it just tries to match the longest existing pattern). About fitting it into a Micro, remember NanoLauLectrik also has a variable lenght pattern matcher (it just rebuilds the predicted bearing in a more pedestrian way) :-) -- Albert
Could you post MicroAspidPMC on the repository? I'd love to see it in action ... even if only for the sheer joy of watching near perfection :-) --Vic
here are the results of EnderPMC 1.1:
http:/robocode/uploads/vic/enderpmc_1_1.jpg --Vic
1st: rz.GlowBlowPMC2 24554 2750 550 18752 2502 0 0 68 45 0 2nd: challenge.PatternBot 1.0 23855 2250 450 18601 2152 346 55 56 55 0
doesn't it look nice? -- rozu
O MY GOD!!!! Read 'em and weep.... You are the first to complete the challenge!! Congratulations Rozu! Notice that EnderPMC inflicts just the slightest bit less bullet damage. That shouldn't have made the difference.... but what has? I'll be back later today ;-) -- Vic ps: please crown yourself at the /PMCIndex
1st: rz.GlowBlowPMC 25459 3400 680 18560 2818 0 0 83 32 0 2nd: challenge.PatternBot 1.0 22558 1600 320 18361 1783 493 0 40 68 0
The results from EnderPMC 1.2 alpha:
1st: challenge.PatternBot 25739 3150 630 18770 2904 284 0 83 37 0 2nd: lv.EnderPMC 23285 1850 370 18916 2149 0 0 50 63 0
Look at the bullet damage! I'm clearly losing on survival here... But currently EnderPMC gets a very high hit rate (98.4% over 100 rounds). I found that the period for PatternBot's Heading pattern is 600 ticks, so i just increased my maximum matching size from 200 to 1000. .... ok now let's see why EnderPMC loses on survival.... -- Vic
Results for Nibbler/NibblerPMC
1st: challenge.PatternBot 1.0 25506 3250 650 18666 2686 253 0 70 35 0 2nd: cbot.pmc.NibblerPMC 22299 1750 350 18699 1500 0 0 41 65 0 1st: challenge.PatternBot 1.0 24271 2750 550 18304 2095 513 57 64 45 0 2nd: cbot.pmc.NibblerPMC 23137 2250 450 18432 2005 0 0 47 55 0
Results for MicroAspidPMC:
1st: challenge.PatternBot 1.0 24400 2750 550 18468 2171 460 0 60 45 0 2nd: apv.MicroAspidPMC 23070 2250 450 18496 1873 0 0 48 55 0
LeachPMC: http:/robocode/uploads/pez/LeachPMC_1.1.4.png
Nibbler/NibblerPMC just beat PatternBot! =)
1st: cbot.pmc.NibblerPMC 1.0 24796 2750 550 18770 2725 0 0 64 45 0 2nd: challenge.PatternBot 1.0 23632 2250 450 18608 2036 286 0 62 55 0
- Great! Congratulations Crippa! What did you change? Could you upload your winning bot on the repository? --Vic
LeachPMC again: http:/robocode/uploads/pez/LeachPMC_1.1.7.png It's a bit unfair that PatternBot collects ram bonus. =) -- PEZ
Some tweaks here and there, a bugfix or two, a speed-optimization, and I ditched the data-saving feature:
http://atlas.et.tudelft.nl/klli87/images/pmchallenge2.gif
At least I get to win a few rounds now... :-) --Dummy
- Good work! Ditched data-saving? Why would you do that? --Vic
- Doing the challenge with pre-loaded data kind of made me feel like a cheater. ;-) --Dummy
- Hey ... all is fair in love and PatternMatcherChallenge --Vic
- Nah... other ppl are doing it without saved data from previous battles, so I'm going to do it without. :-) --Dummy
Lol, I just tried FloodHT's gun package on PatternBot for fun (it does choose the pattern-matcher for it), and it was getting about an 80-84% hit-rate without tweaking, and then it fell almost to nothing, and I realized an interesting flaw in the way I implemented the pattern-buffer (using a mod function to put stuff into a finite-length array). I was matching the exact pattern I just recorded and projecting the next part of the pattern that I was going to overwrite. So after round 60-something, it just started to miss a lot, and the hitrate instantly fell from 80 to somewhere under 40, where various stat guns took over. -- Kawigi
Read 'em and weep:
1st: lv.EnderPMC 1.2 26645 3800 760 18772 3312 0 0 87 24 0 2nd: challenge.PatternBot 21626 1200 240 18504 1339 343 0 36 76 0
index 101.45 --Vic
Here it goes the last fight for MicroAspidPMC:
1st: apv.MicroAspidPMC 25630 3150 630 18954 2896 0 0 84 37 0 2nd: challenge.PatternBot 1.0 23526 1850 370 18859 2187 212 46 50 63 0
- Congrats Albert! --Vic
Ok, I'm starting a real go at it with Teancum's gun. It might not be done until tomorrow, I just added a mechanism for preventing any matching or projection across rounds, and I also fixed the mod problem I described before, and I think it's ok. Just fixing the mod problem is enough of an excuse to release a new version of Teancum soon. Now I'm trying to figure out if I should do anything with my projection-into-the-wall mechanism, which fires a faster bullet at the point of impact into the wall in the regular dev version of Teancum. His /PMCIndex right now would fall around 90%-91%, as well as his hit rate, and he wins only occasionally, without really doing anything else that wasn't in the dev version except playing with the match length, or doing anything special to hit him in the beginning. This is probably the best so far for a LateralVelocity-based pattern-matcher. Before I caught that mod problem, I seriously was considering just labeling myself as bad at pattern-matching and sticking with stat guns from here on out. -- Kawigi
- Great! This is turning out to be a real occasion for people to improve their PM guns :-) And a lot of fun to boot. -- Vic
You guys realy slayed PatternBot when i bearly beat him! This realy was a good way to improve our pattern matchers. I'm not sure that i can take advantage this in the real Nibbler but at least it was fun! =) EnderPMC shows that you realy can humilliate poor PatternBot, atleast in survival. =) --Crippa
Just out of curiosity, Vic (since you didn't release source with PatternBot), would you think that pattern-matching PatternBot relative to myself should still be fundamentally correct? Does he move relative to his opponent once he gets into position? TeancumPMC's index is climbing... -- Kawigi
- No, PatternBot's movement is very antisocial. It doesn't care in what position you bot is ;-) I don't know if the other high-index guns use relative data so i can't tell you if it is fundamentally wrong. Obviously EnderPMC has one of the absolute kind. I figured if it can detect absolute patterns then it can also detect relative ones.. --Vic
- Well, I asked just because the only robot on the /PMCIndex page right now that uses it is Mini.FenrirPMC, except for probably an old version of LeachPMC that PEZ ditched because of this challenge. So I suppose that getting Teancum's /PMCIndex over 95% may be that much more of an accomplishment :-) Another question - are people waiting for their guns to turn? Does PatternBot wait for his gun to turn? I'm thinking I'm actually firing fewer shots than him (well, less frequent shots, anyways). -- Kawigi
- Yes, I think 95% with lateralVelocity is awesome. I guess you combine it with retreatingVelocity to calculate reasonable bullet travel times? I'm currently experimenting with how much I should wait for the gun to turn. If I don't wait at all I shoot many stray bullets. If I wait for perfect alignement I shoot too few shots. I have found waiting for "abs(gunHeading - predictedHeading) < atan2(botWidth / 2, distance)" seems to work best. -- PEZ
- I combine it with AdvancingVelocity (-retreatingVelocity) for distance - but the distance in turn gets used (and always has been in Teancum) for bearing offsets, too. One thing that's nice about this approach, though, is that calculating the final bearing is not overly complicated. -- Kawigi
- I wonder if you can get much higher than 95%. Do you think it's possible? On gun turn waiting: EnderPMC always waits for the gun to stop turning completely. Yet it fires faithfully every 12 ticks. Are you aiming only when your gun is cool? -- Vic
- One approach (something I've started and will really try tomorrow) is to aim for a future pattern match - I start out by calculating bulletv (20-power*3) and then I initialize bulletd (the distance the bullet has travelled) to -bulletv*Math.ceil(getGunHeat()/getGunCoolingRate()). If I start doing that a few turns before my gun cools, I may be able to not need to wait for it (by having it already aimed for that specific time). Also, I've seen it do 96% PMCIndex... if I went for more rounds, I may even be able to out-damage him, even without accounting for the starting game in any special way. And I have another possible mathematically picky thing to try, too. But now, I'm going to bed.
YALPMCR (Yet another LeachPMC result):
http:/robocode/uploads/pez/LeachPMC_1.1.10.png
!!! -- PEZ
- -DDDDD (YAEFK)-- Kawigi
Ok, here's my official first-version submission of TeancumPMC:
1st: challenge.PatternBot 1.0 26836 4150 830 18325 3215 315 0 86 17 0 2nd: kawigi.sbf.TeancumPMC 19824 850 170 18059 744 0 0 21 83 0
-- Kawigi
The final(?) version of LeachPMC:
http:/robocode/uploads/pez/LeachPMC_1.2.png
It quite consistently beats PatternBot. It's using a general pattern matcher gun with only a few tweaks to score higher against PatternBot. Mainly a very long "movie", info on pre-pattern-phase in the movie and slightly different matching in the beginning and end of the rounds. -- PEZ
New results of EnderPMC 1.3 after a bugfix:
1st: lv.EnderPMC 1.3 26677 4000 800 18564 3312 0 0 84 20 0 2nd: challenge.PatternBot 20855 1000 200 18245 846 460 102 29 80 0 index 101.75
-- Vic
After fixing BlackPearl's projection error (Thanks Kawigi) Pearl is now getting this result:
1st: challenge.PatternBot 1.0 26935 4800 960 17087 3298 733 57 98 4 0 2nd: jekl.mini.BlackPearl 16606 200 40 16133 233 0 0 4 96 0 Index: 94.42%
- Great Jim! Hopefully you will be able to beat PatternBot soon. -- Vic
Quick update:
1st: challenge.PatternBot 1.0 25566 3500 700 18190 2710 465 0 79 29 0 2nd: jekl.mini.BlackPearl 21240 1450 290 18055 1444 0 0 33 70 0 Index: 99.26
-- jim
I guess we're all just posting our scores like this? No - I'm going to start a results page! -- Simonton
