calculating confidence of an APS score
Fragment of a discussion from Talk:RoboRunner
Jump to navigation
Jump to search
Yeah, that's a good point, especially with the TC bots that are just simple random movements and no gun. If the variation in confidence is higher than the variation in speed, it could take longer for same number of battles. I guess the puzzling thing is the overall confidence calculation showing the same both ways. With a limited amount of sample data, I guess it can only be so accurate, but I'm thinking I may have a bug there. The spread was:
apv.AspidMovement 1.0: 95.6 +- 0.83 (16 battles) dummy.micro.Sparrow 2.5TC: 98.43 +- 0.64 (13 battles) kawigi.mini.Fhqwhgads 1.1TC: 96.95 +- 1.11 (21 battles) emp.Yngwie 1.0: 98.15 +- 0.77 (14 battles) kawigi.sbf.FloodMini 1.4TC: 94.91 +- 1.25 (24 battles) abc.Tron 2.01: 88.15 +- 1.42 (26 battles) wiki.etc.HTTC 1.0: 88.83 +- 1.45 (28 battles) wiki.etc.RandomMovementBot 1.0: 92.23 +- 1.04 (22 battles) davidalves.micro.DuelistMicro 2.0TC: 86.22 +- 1.61 (31 battles) gh.GrubbmGrb 1.2.4TC: 81.29 +- 1.87 (33 battles) pe.SandboxDT 1.91: 85.48 +- 1.8 (31 battles) cx.mini.Cigaret 1.31TC: 86.82 +- 1.62 (31 battles) kc.Fortune 1.0: 80.6 +- 1.77 (29 battles) simonton.micro.WeeklongObsession 1.5TC: 87.02 +- 1.48 (26 battles) jam.micro.RaikoMicro 1.44TC: 79.16 +- 1.8 (30 battles)
Going to leave some tests with Diamond 1.8.16 in real battles running today and see how that compares.