Thread:User talk:Xor/General Ablation Study Framework

To accurately measure the performance of different components of a bot, we do ablation study by disabling them. However, due to the popularity of hand-made bot frameworks, it is generally hard to compare your bot to other bots component-wise, even when source code is given.

Since we cannot have everyone rebase onto a common framework, another approach would be replacing part of the behavior of a robot dynamically. This should also work with bots without source code.

A robot can be divided into 3 basic components: Movement, Gun and Radar. Replacing behavior from one component is as simple as hooking calls like setBodyXXX, setGunXXX, setRadarXXX to IAdvancedRobotPeer. This way we can e.g. easily compare different guns from different bots while having movements fixed, even without doing anything with the code.

There are certainly drawbacks:

1. Replaced components are still consuming CPU while contributing no behavior.
2. No fine-grained way of comparing sub-components, e.g. Bullet Shadow
3. Certain features may not work properly without manually changing code, e.g. bullet shadow may have wrong gun firing angles

Also, frameworks like this should only be used for experimental purposes, and forbidden in competitions unless given permissions from original authors.