Inconsistent APS with LiteRumble

Quoted from APS:

The server calculates APS for each bot by:

1. taking the average percentage score of all battles against each opponent separately to get an APS for each pairing,
2. then averaging all pairing scores to obtain the final average.

Formally, it's

mean(challenger_score / total_score)

for each pairing.

LiteRumble seems to follow this algorithm.

However, RoboRunner is using:

sum(challenger_score) / sum(total_score)

instead for each pairing.

This causes scores calculated by RoboRunner to be different from LiteRumble.

Here is a Python script that calculates APS correctly. Use this script when you want to align with LiteRumble.

import os
import gzip
import xml.etree.ElementTree as ET

from statistics import mean
from collections import defaultdict


roborunner_dir = os.path.expanduser('~/roborunner')  # replace with your roborunner directory


def get_aps_dict(bot):
  scores = ET.parse(gzip.open(f'{roborunner_dir}/data/{bot}.xml.gz', 'r')).getroot()

  aps_raw = defaultdict(list)

  for bot_list in scores:
    for battle in bot_list:
      battle_scores = {}
      total_score = 0

      for robot_score in battle.findall('robot_score'):
        name = robot_score.find('name').text
        score = int(robot_score.find('score').text)
        battle_scores[name] = score
        total_score += score

      aps = 100 * battle_scores[bot] / total_score

      for name, _ in battle_scores.items():
        if name != bot:
          aps_raw[name].append(aps)

  return dict((name, mean(aps)) for name, aps in aps_raw.items())

Update: The APS fix is now included in the newest release of my fork. A PR is also made to Voidious’s version.

Did this fix also apply to melee score calculation? I've been having difficulty getting my offline results with roborunner to align with literumble for the nano melee rumble.

Yes, the definition of APS applies to all of the rumbles.

I think I've solved this now, it looks like the problem was with the way I was generating battles.