Density Calculations

My Anti-Surfer gun uses 4 data sets. They are k=3 of last {125 | 400 | 1500 | 4000} data points. So the least-decaying one has ~5 rounds of data, and many of the selected data points will overlap.

8% vs 12% of raw hit rate over a few battles doesn't could still be just fluctuation. If it's a normalized hit rate that might be significant... I generally run a few hundred to a few thousand battles to test a change, for what it's worth.

Its normalized calculated by DrussGT. I just did 15 battles, it seems unlikely due to random fluctuation. Is your anti-surfer gun a composite of multiple other guns? How does it work?

Well, I consider it one gun. :-) But instead of a single KNN search, it does 4 of them and combines the results.

You should be able to hazard a guess at the margin of error by looking at the distribution of those 15 values. Isn't it 1.96 * standard deviation = 95% confidence interval? Edit: Er, then divided by sqrt(num battles)? I don't know, but it's not hard to find. :-)

Yes, different trees. It does assume a normal distribution, but I think that's at least in the ballpark.

I'm not sure, but if going from k=1 to k=2+ produces a significantly different result, that seems strange. I'd probably spend some time trying to suss out bugs. Like what if you do k=2 and only consider the 2nd point, so there's still only 1 point in your kernel density? Or what if you do k=1 but then feed two of that point into your kernel density?