BeepBoop seems to be losing ~1.5 APS
Comparing to the nearly identical 1.20, you can clearly see some random drops, which I suspect to be skipped turns on some old hardware.
The previous results of APS ~94.8 can be reproduced on my computer, so I think the previous results can be trusted.
@Beaming seems to be contributing to RoboRumble the most recently, we could work together to see if something could be done to ensure reproducibility of RoboRumble.
I think I found a potential problem spot. One of my computers was 4 times slower and was using cpu constant from a 4 times faster computer. I recalculated cpu constant (by deleting the config file) and hope that the APS drop would resolve. It might explain why a better (subjectively) version of my bot in development performs worse than the old one.
It would be nice if rumble client recalculated CPU constant at the start. It take very little time and provides more stability. But I also recall discussion that active throttling in modern hardware makes this number just an estimate.
By the way in 2014 we had interesting discussion about Thread:User talk:Beaming/Smart bots competition allowing long calculation time for bots. Maybe it time to revive it since ML approaches developed quite a bit and they are CPU intensive.
From other hand making a top ranking fast bot is a challenge in itself.
I agree. Enforce recomputing of CPU constant at the start and per e.g. 100 battles is necessary, as it highly affects results and is easy to get wrong. By recomputing periodically, it can also solve the problem that other heavy tasks are affecting RoboRumble, without adding too much overhead.
I'm also thinking about adding some test set before allowing score submission, but that would be a long term plan.
I'll submit a PR for recomputing CPU constant, any suggestions are welcomed.
