CPU benchmark advice
Just in the last week I got a AMD Phenom II X6 1090 at 3.2GHz here. Sure, it's slower per core than a high end i7 like the 2600K, but on the other hand 1) The CPU is practically half the price of a 2600K, and 2) six cores rather than four is nothing to sneeze at for robocode purposes.
Running Diamond 1.6.7 versus itself, 35 rounds:
- 50.265s average (Trials: 49.735s, 43.292s, 51.542s, 53.741s, 50.764s, 52.277s, 52.839s, 47.930s)
- This is without GUI, and including robocode startup time (about 1-2 sec)
Running Diamond 1.6.7 versus itself, 35 rounds in 2 separate robocode instances:
- 29.305s per battle
- 58.611s per instance (Trials: 58.016s, 61.019s, 56.540s, 59.753s, 55.980s, 62.418s, 57.420s, 57.741s)
- Robocode startup time increased to 4 seconds. This would not be a factor in a battle runner which runs multiple battles in the same JVM!
Running Diamond 1.6.7 versus itself, 35 rounds in 4 separate robocode instances:
- 16.294s per battle
- 65.174s per instance (Trials: 65.207s, 66.350s, 67.326s, 66.458s, 62.473s, 62.683s, 65.189s, 65.709s)
- Note, robocode startup was already seemed highly parallel, because robocode startup now took up to 8 seconds for one instance! As such, about 6 seconds of the increased time can be attributed to robocode startup.
Running Diamond 1.6.7 versus itself, 35 rounds in 6 separate robocode instances:
- 15.736s per battle
- 94.417s per instance (Trials: 91.325s, 92.572s, 93.809s, 94.710s, 95.344s, 95.738s, 97.421s)
- The gains seem to flatten out about here. One note is, because one instance of robocode on it's own uses something like 115% of a core, I should reach a CPU limit at 5-ish instances, not 4-ish, so I suspect I'm hitting a memory bandwidth bottleneck
One instance of Robocode can use more than 115% of a core. It oscilates between 100% and 200%. It is expected to see a performance decrease in benchmarks when you run more instances than half of your cores (all instances using 200% at the same time).
Why would it use 200%? According to Pavel, different robots can run on different cores, but they are synchronized so only one is running at once, basically capping your actual performance at the speed of one core. So it should be 100% + some JVM / Robocode engine overhead, I'd think.
That overhead happens about 30% of the time, so an instance uses about 130% cores average. But there are peaks of 200%. When I run 3 instances on 4 cores, they use all cores most of the time, but you see one idle core sometimes (and it´s not uploading).
When running test beds, I run one instance per core (and disable turn skipping), so all cores are used all of the time.
Running a benchmark restricting each instance to a single core would remove that parallel overhead.
