APS 95.43 is crazy

That is very solid. Btw how can you sure that each changes grant you positive APS since it’s very small, being hard to evaluate.

I do test changes against a big subset of the roborumble bots (with many seasons to make sure results are significant)

Thanks, that sounds practical. I’m always doing a few seasons on the full population to test out small changes, which takes a few hours on newest hardware. However, it is still hard to make 0.1 APS changes significant.

Yeah, you need to run a lot of seasons, but of course that's slow. I think the right way to develop a strong bot would be to operate like the stockfish chess engine where every change is evaluated for improving performance, but I mostly just develop on my laptop so I don't have enough compute to do that.