Version 0.9.2 Bugs
The highlighted comment was created in this revision.
I try and do a tcas/tcrm challenge, but I get a Challenge not supported. Unsupported scoring type: AVERAGE_BULLET_DAMAGE.
Doesn't robojogger support this scoring method? Or does it have a different challenge file syntax? If so, what is it?
Unfortunately, I need to go back to roboresearch.
EDIT: After some research, I found it was "BULLET_DAMAGE", please make it support "AVERAGE_BULLET_DAMAGE" as well, if only as an alias. So that we can just copy and paste roboresearch challenge files. :)
It seems to fill in the results with 100, despite the actual scoring in the roborunner output being something else. The results in the results window do not update during the running (not even the erroneous scores). The confidence scores were always 0.0, just as the scoring was always 100.0.
Even after a full three seasons, the scores did not correct themselves. The correct results are in RoboRunner output of course.
Hehe, while I wish I could believe I made a gun which produced such scores, I didn't.
The scores not updating during running is normal. I'm waiting for the next release of RoboRunner before I implement that. As for the messed up scores, I would guess this has something to do with an error in what I'm doing with the scoring type. All of my testing so far has been with PERCENT_SCORE. I'll start testing with other scoring types and fix whatever problems I find.
Okay, so I looked into what is happening with the scores under the BULLET_DAMAGE scoring type. What RoboJogger is doing is taking the score for the scoring type for the challenger and dividing it by the sum of the score for the scoring type for both the challenger and opponent. I think this seems right, but I don't use scoring modes other than PERCENT_SCORE very often, so I'm not entirely sure.
Why this comes up with a bad number is because when the scores are loaded from the ScoreLog (ScoreLog is part of RoboRunner), the bullet damage score (which the BULLET_DAMAGE score type relies on) is always 0 for the opponent. This might be a bug in RoboRunner -- either the scores not getting saved correctly or loaded correctly by the ScoreLog. I also noticed that the energy conserved values were also 0 for both the challenger and opponent, so may be a related bug there too.
TC scoring is bullet_damage / number_of_rounds. Which produces an output between 0 and 100 (not 0.0 and 1.0).
The your_score / (your_score + enemy_score) is a percent index. It can't be used with bullet damage where one of the robots does not fire (which is what happens in a TC). The reference robot (the one moving) will never have any bullet damage.
I guess I assumed all scores were just percent scores based on different scoring metrics. I need to figure out where I can find information on exactly how each challenge type is scored. I'm not following your explanation on TC scoring, as I don't see how a good robot that doesn't run itself out of energy with misses would ever score anything other than 100. If the opponent doesn't fire (or hit walls), the challenger would score 100 on every round.
Well, that could happen, except you are only allowed to pass 3.0 as the fire power. Which means that no current robot is able to get 100 bullet damage all of the time.
