Please not another SittingDuck...

With the scoring system as it is in melee, the action of adding and removing sitting ducks is a bigger problem than their presence or absence. Due to how melee scores are based on battles with a random sampling of robots in the rumble at the time, a change in the probability of a sitting duck being in the battle affects newer bots more than bots who already have most of their battle count. Adding the ducks gives a bias in favor of newer robots, and removing them gives a bias in the favor of older robots.

Of course... this problem happens in the melee scoring system for addition/removal of any bot. It's just a more blatent with sitting ducks than more "typical" bots...

ELO Rating system suffers less from this addition/removal problem in melee.

Not really. ELO doesn't take into account which other bots were also in the battle for each pairing, which is the problem with our melee ranking system.

ELO has a behaviour similar to Rolling Averages. It gradually forgets the past, so, past battles with retired bots stops affecting the ranking after some more uploads are made.

ELO was designed taking in account a changing environment. i.e. chess players improving their skills over time, new players, retired players...

The changing environment in this case is a bit different from chess though due to how there are many participants in each match, and it's known which participants that may have changed, versus not changed. This extra information could be leveraged.

I haven't looked deeply into it yet, but I imagine that it would be possible to construct a melee scoring system with greater immunity to this type of distortion, by considering the full list of partipants in each battle instead of tossing out information by splitting it into pairings. (Of course, the rumble client doesn't even give the necessary information to explore improved melee scoring systems, because the splitting into pairings is done at the client rather than the rumble server)

I'm unsure if it was done that way with ELO specifically in mind... or if it was more "1v1 oriented" with melee as an afterthought.