Is there a mistake in Rumble battle staging

Jump to navigation Jump to search
Edited by author.
Last edit: 04:57, 22 February 2023

What do you mean by avoiding particular bots. From the code, LiteRumble selects missing pairings first, then pairings with less battles, etc.

Given enough time, each bot should have similar amount of battles.

Anyway, it's not a good idea to rely on LiteRumble to accurately measure how good each small change of your bot is. LiteRumble is designed to give everyone a ranking, well suited when you have a big change (proven to be effective) and want to publish the result. Small changes are prone to noises from the rumble environment (e.g. running on different computers for different versions, having updated opponents), thus may report unreliable result.

Rumble clients are also much slower than running battles locally (with RoboRunner), due to added network communication (which isn't parallelized), as well as running battles from other updated bots. One can also notice that once each bot reaches ~5000 battles, most battles are then nano battles instead of randomly distributed across the entire rumble, due to some "propagation" effect, further slowing down the stabilization of the general rumble.

Xor (talk)04:46, 21 February 2023

Ups, I posted wrong link to Jen rating, see http://literumble.appspot.com/BotDetails?game=roborumble&name=voidious.micro.Jen%201.11&order=-Battles

I do use LiteRumble to check for my bot improvement (if I run clients anyway, why not update "real" rating). But the issue I report is unrelated to my bot.

Beaming (talk)03:59, 22 February 2023

OK, a thought the missing pairings is related to your bot (which just get updated), but it turns out to be an ancient one.

Xor (talk)06:06, 22 February 2023
 

If you look at Jen pairing, you will see almost 200 bots, with which it has only one pairing. This is not normal for a bot which is many years in the rumble. http://literumble.appspot.com/BotDetails?game=roborumble&name=voidious.micro.Jen%201.11&order=-Battles

Beaming (talk)05:54, 22 February 2023
  1. We need to find out whether they have only 1 pairings 10 years ago. If so, we could add back the missing pairings from backup.
  2. From the code, "Battles" is merely a counter. If it shows 1 pairing, then there must be data loss unless it's 1 initially.
Xor (talk)06:10, 22 February 2023