Shielding Success Rates Mystery

I started doing some research on my shielding success rates. I started with a small set including Virus, Seraphim, Hydra, Crusader, and Engineer. I did a first test run of 20 seasons. Shielding success occurred on 20/20, 19/20, 20/20, 20/20, and 13/20 seasons respectively. I know why it's not perfect for Engineer, but the other 4 are the mystery. In the RoboRumble, my success rates on the first 4 are currently 5/8, 5/8, 5/8, and 4/7 respectively.

None of the Rumble losses were run by my machine. But all of them were run using client 1.7.3.0. Losses only occurred when run by Voidious and DivineOmega. I'm guessing there is something common about the machines used by you two that is key. Are you both perhaps running under Linux? Also, what version of Java? I need to replicate one of your machines as closely as possible to explore the problem further.

Maybe their clients are skipping turns. Missed scans hurt bullet shielding a lot.

If they want to check that, they sort-of can. It only reports on the last battle that it happened, but if my robot has any skipped turns, it will write a file xander.cat.XanderCat_SkippedTurns.txt file into it's data directory that has the number of skipped turns printed in it. It's not as useful as it could be since it just reports on the last battle that it happened on, but could be worth a check.

Are the tests you run also using 1.7.3.0? (BTW, I'm doing some major bullet-shielding-suffering at the moment...)

Yes. I run 1.7.3.0 as well.

My rumble clients are on a Core i7-3770 running Ubuntu 12.04 (32-bit), and I think OpenJDK 6 (will check when I get home). Now that you mention it, I recall a similarly huge discrepancy for the original BulletCatcher on an old AMD Ubuntu machine I was using as a rumble client (couldn't find the discussion though). I chalked it up to the different JVM or even the CPU and ended up just retiring the machine as a rumble client, since it was a fraction of my overall CPU power.

My first guess is that when two bullets are close enough to parallel, one JVM's Line2D says they intersect and another doesn't. I actually have Ubuntu 32-bit, 64-bit, and Windows 8 64-bit all on that same machine, so I should be able to do some decent tests for you with only the JVM or OS as a variable.

Found the discussion: Archived_talk:User:Voidious_20110909#Wierd_scores

When the opponent is moving, my robot stands perfectly still (it rotates but doesn't move position). Thus, I haven't verified it, but the two bullets should be perfectly parallel, and this is likely handled differently on different JVMs as Voidious was talking about. However, when both robots are standing still, my shielding shot will miss, so my robot will move slightly when firing a shielding shot against a stationary opponent. I haven't had enough time to figure out what causes this, but I wonder if the answer will give any further clues. I did it this way because it worked, but more investigation is required for me to figure out why.

I could play around with moving slightly for every shielding shot (and moving back right after the shot). I tried this briefly, but found standing still to be more reliable on my machine. But maybe I can tweak it to where it works as well as standing still. If I can, I bet it would avoid the problem.

I will likely need a few more days to play around with this further. In the meantime, if Voidious has time to try out the other JDK, that would answer another piece of the puzzle.

Standing still works because the enemy gun rotation happens after the bullet is fired, so they shoot from a position of 1 tick ago but use their aim from 2 ticks ago. So if they are moving their angle to you changes and they don't quite shoot at your centre, meaning that you can get your bullet line to intersect with theirs because your bullet comes from your centre (which they aren't aiming for). This is also why it stops working when they stand still, because their last position relative to you lines up with their current position relative to you so you end up shooting parallel.

I'm trying to figure out why my super-advanced precise circle-line intersection methods are failing so horribly at getting shield hits. I get one every now and again, but nothing like what I should be based on the maths and shield size I'm calculating, and nothing like the 3/4 of bullets that a simple linear projection + bullet power adaptation was getting.

I would recommend against using Seraphim in a test bed. It is a very buggy robot. So much so that I think it sometimes acts differently in a rumble environment then in a test environment.

But also some versions of Seraphim if I recall add a minor variance to its gun heading it detects it is against a bullet shielder. Is the version your using the same as the one in the rumble? (I expect so, but it cannot hurt to ask.)

After some testing, maybe we can come up with a more consistent implementation for checking bullet collisions / line intersections in the Robocode engine. Here's the relevant code snippet: [1] (line 76 calling line 113).

Here's one that gets rid of the division which I expect is what blows up on poorly conditioned problems:

	private boolean intersect(Line2D.Double line) {
		double x1 = line.x1, x2 = line.x2, x3 = boundingLine.x1, x4 = boundingLine.x2;
		double y1 = line.y1, y2 = line.y2, y3 = boundingLine.y1, y4 = boundingLine.y2;

		double dx13 = (x1 - x3), dx21 = (x2 - x1), dx43 = (x4 - x3);
		double dy13 = (y1 - y3), dy21 = (y2 - y1), dy43 = (y4 - y3);

		double dn = dy43 * dx21 - dx43 * dy21;
                double dn_sign = Math.signum(dn);
                double dn_abs = dn*dn_sign;

		double ua = (dx43 * dy13 - dy43 * dx13) * dn_sign;
		double ub = (dx21 * dy13 - dy21 * dx13) * dn_sign;

		return (ua >= 0 && ua <= dn_abs) && (ub >= 0 && ub <= dn_abs);
	}

It might even be faster, divisions are about the same speed as sqrt.

Given that it is using it's own code to determine intersection rather than a JVM method, I wonder if the discrepancy between systems is actually in the data stored in the line objects.

I thought maybe I could tweak it using a small amount of movement to make it work on all systems, but so far my attempts have degraded shielding performance unacceptably. One bit of good news -- I tested and found out that Robocode security does not prohibit robots from reading System properties, so if I can figure out how to correct (at least partially) for the problem on other OSs or JVMs, I can test for them and just make those changes on the appropriate systems.

When I experimented with bullet shielding, my bot calculated minimum and maximum angles which would hit an incoming bullet, and only shoot if the difference between angles was above a threshold. It was there to work around floating point calculation errors, which translate into "parallel" bullets.

If the difference is below the threshold, then moving sideways helps increase the difference. Near 100% bullet shield against TrackFire. But against moving opponents and/or weak powered bullets, sometimes my bot moved until it crashed on the wall, never finding a good angle to shoot. And this is where my experiments are stuck right now.

Tonight I ran a test on one of my servers in my basement. The OS on it is Fedora 13. I installed Open JDK 6 (command su -c "yum install java-1.6.0-openjdk-devel"). Once installed, it reports itself as:

OpenJDK Runtime Environment (IcedTea6 1.8.8) (fedora-51.1.8.8.fc13-i386) OpenJDK Client VM (build 14.0-b16, mixed mode)

I ran a number of battles and have yet to encounter any shielding failures using XanderCat 12.2. But I need to do a more thorough test to be sure. Tomorrow I will set up RoboJogger and run a bigger series of tests.

Just checked and I'm also on OpenJDK 6:

java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.5) (6b24-1.11.5-0ubuntu1~12.04.1)
OpenJDK Server VM (build 20.0-b12, mixed mode)

Sorry I didn't get to any real testing today, but I will tomorrow. Now I'm really curious to see if I see the same in another JVM or in Windows...

I see that yours is the Server VM also. I wonder if that makes any difference.

In case it's easier just to copy it from here -- here is a challenge file with 3 opponents that XanderCat 12.2 is nearly perfect against on my machine but only maybe 50 percent successful against on machines that exhibit the performance anomaly.

Shielding Vulnerable
PERCENT_SCORE
35

Shielding Vulnerable {
    apv.test.Virus 0.6.1
    kc.serpent.Hydra 0.21
    trab.Crusader 0.1.7
}

Hi mate.

I was in the mood to check XanderCat 12.2 on my system and maybe it can help you to detect some uncertainties.

Mac OS X 10.6.8

java version "1.6.0_37"
Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-10M3909)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)

Robocode version: 1.7.4.4
CPU constant: 5426594 nanoseconds

Result vs Virus (png) Last Round Log vs Virus (png) Round start log vs Virus (log)

I started a couple of runs and it looks like almost 50% failed because of massive skipped turns if the round starts. This happens if the client runs on low speed and as well on full speed. If I put the client on debug=on, everything works fine and XanderCat win all rounds like expected. Because of the skipped turns XanderCat misses quite a few bullet catches and decides to move with a different movement - from that point he is almost ever doomed to loose some rounds. I hope it helps a little.

BulletShieldTest 1.0 worked so far very good on my system. He is lost against all 'sloppy' guns (HawkOfFire,SpinBot..) but against well coded targeting he has a quite impressive performance. Of course, he has still some minor flaws but i guess that is not unusual for a test version.

All in all i would say there is no difference in how the JVMs handle intersection but there may be a difference how they handle garbage collection and therefor have a different skipped turn behavior.

Man, I wished I had more time these days :( - take care

Thank you, Wompi. That is very helpful. From the round log, looks like there are occasional skipped turns (most likely due to the wave surfing drive and guess factor guns, but these sit idle when bullet shielding is active, and bullet shielding is much less processing intensive). However, the first round where most things are initialized is the worst.

So it's probable the problem isn't with the JVM doing the shielding calculations but rather how much time each system is allowing and how long it takes my initialization code to run (which I can work on, but it may also be true that different JVMs take different amounts of time for different types of initialization).

Looking at the info provided by the run time loggers, the averages between my system and yours aren't all that different, but the peaks shown on yours are an order of 5x what they are on mine. The radar peaks are even worse, which is weird. A typical greatest peak for my radar on my system is 0.25 trialing down to less than 0.1 by the 3rd peak, whereas on your system they start at 9.9 and are still above 0.8 by the 5th peak. (note: peaks are over the entire however-many round battle, as are the averages)

Both your system and Voidious' system are using the Server VM, and that makes me suspicious. I need to try out a Server VM and see what happens.

It should be noted that if this is the problem for Voidious, DivineOmega, and perhaps others, it is affecting overall performance (in the first round at least) and not just on the bullet shielding. If I can find a way to fix it, overall rumble performance could improve some.

I just tried the Server VM on mine but didn't see anything change.

Running some tests now. Starting with 20 seasons, single-threaded on the Ubuntu/OpenDJK setup. 10 seasons in and see it failing a lot vs Virus (~76%), rarely vs Crusader (~90%), and working well vs Hydra (96%). I'll post more details and the full RoboRunner logs when I've got some more data, and try it through the UI to see what I can tell about skipped turns.

If it is a skipped turns issue, I know there is one trick to put heavy initialization stuff in a static block and it won't count against your CPU time. I really hate Robocode's skipped turn setup, though I think limiting the CPU time in general is great. I'm definitely taking a different approach in BerryBots. It's probably too late to completely change it in Robocode, but we could at least look at an average over the last 5 or 100 ticks and penalize based on that instead (which is part of what I'm going to do).

Wow, ok.. So 4 threads was slightly worse overall, but 20 seasons is hardly enough to cover the margin of error. Now testing on Windows, and at 98 overall score after 8+ seasons. This is Oracle Java 7. Also interesting to note that the CPU constant was set to about 5 ms in Windows, vs 7 in Linux. Also this is all on Robocode 1.7.4.0, sorry about that, but I don't think it makes a difference.

Ok, still want to test on Ubuntu 64-bit and with the Oracle JVM on Linux, but here's a bunch of data for now.

config	Virus	Hydra	Crusader	Total
Linux/OJDK/1 thread	76.14 +- 8.29	95.93 +- 0.37	89.26 +- 5.17	87.11 +- 2.66
Linux/OJDK/4 threads	81.45 +- 7.6	82.77 +- 6.51	90.91 +- 5.14	85.04 +- 2.92
Windows/1 thread	98.89 +- 0.25	96.09 +- 0.33	98.67 +- 0.34	97.88 +- 0.17
Windows/2 threads	97.18 +- 1.91	96.31 +- 0.35	98.92 +- 0.3	97.47 +- 0.48
Windows/3 threads	95.4 +- 3.19	96.39 +- 0.23	88.88 +- 5.49	93.56 +- 1.55
Windows/4 threads	98.66 +- 0.31	92.2 +- 3.5	92.42 +- 4.31	94.43 +- 1.41

It's worth noting that I feel comfortable running my own benchmarks 6-threaded on this machine and I usually run 4 RoboRumble clients at a time.

Thank you, Voidious, for all the data. Don't forget to take a peak at skipped turns under Linux/OJDK. It gets printed to the game log, and you can also check what is printed in the SkippedTurns text file it writes to the data directory.

For now, I'm going with the assumption that I need to target reducing initialization overhead due to skipped turns. Once I have that done, I can prepare another version of XanderCat, which if I'm lucky you will be willing to run through your Linux/OJDK setup again to see how it changes things.

So, upon not getting to the bottom of this, I stopped running my rumble clients. Where are we with the discrepancy in shielding success on different systems? I don't particularly mind keeping my clients off for now, since I'm not submitting bots, but I do have quite a bit of CPU power I'd be happy to contribute.

I also still want to test with the newer XanderCat, Ubuntu 64-bit, and a different JVM on Ubuntu, but that requires Voidious-time and not just CPU-time. :-)

Not much new at this point. I greatly reduced my processing overhead in XanderCat 12.3+ in an attempt to improve on the problem (nearly cut average and peak processing times for targeting and surfing in half; see Version CPU Usage section of my XanderCat page; noting that my CPU constant is around 10ms), but since the problem seems linked to garbage collection, I don't think it will result that much of an improvement (I'm guessing far fewer skipped turns in later rounds but still a bunch in the first round, which will still break shielding). I may be able to reduce the amount of garbage collection by trying to eliminate as many local method-scope variables as I can, but that would be a lot of work for something that will likely make the code less readable and I don't even know for certain if it will help. Thus I haven't tried it yet.

Wompi indicated that using -XX:+UseConcMarkSweepGC to use the concurrent mark and sweep garbage collector eliminated the problem, but I don't think anyone was sure whether it was a good idea or not to use this as a solution. Not sure if it could cause other problems.

For now, I don't mind if you restart your clients. XanderCat 12.5 already has over 10K battles and I don't intend to replace it with a newer version for awhile.