calculating confidence of an APS score

Fragment of a discussion from Talk:RoboRunner
Jump to navigation Jump to search

I'm using the same Monte-Carlo method for confidence either way. I hadn't run too many side by sides yet, but I'll do some more soon. Over night, I ran a test of 25 seasons of TCRM in regular vs smart battles mode on my laptop. They took about the same amount of time, and both ended up showing +- 0.363. But the smart battles came out to 89.32, very close to the 89.31 I got when I ran 100 (non-smart) seasons before, while the normal battles ended at 88.76.

So I'm a little disappointed it wasn't faster nor showed a better confidence, but it was a lot closer to the true average. And I guess my confidence calculation sucks or something weird happened, since 88.76 is much farther than .363 from the true average. (And yes, my TCRM score has tanked that much since its glory days!)

Voidious‎13:47, 14 August 2012
Retrieved from "http://robowiki.net/wiki/Thread:Talk:RoboRunner/calculating_confidence_of_an_APS_score/reply_(5)#calculating_confidence_of_an_APS_score_1446"