calculating confidence of an APS score
Fragment of a discussion from Talk:RoboRunner
Jump to navigation
Jump to search
The per bot +- is the 95% (or 97.5%?) confidence = 1.96 * standard error = 1.96 * standard deviation / sqrt(num battles).
It probably is something silly like the one battle per bot you mentioned, but at a glance it seems like the overall confidence calculation isn't doing anything stupid. I'll have a longer look this evening. I do think the smart battles are working well, though, I'd just like to have some numbers to back me up. =)
The spread is a bit more interesting in real battles. HOT bots with 99.9% scores will get 2-3 battles in 12 seasons. RamBots get lots of battles because they have fairly high variance and run super fast.
