Talk:RumbleStats

Glicko-2 scaling

Oops, I forgot that the RoboRumble Query API is not yet scaling the Glicko-2 rating as it does for the web interface, so right now this isn't going to post the right Glicko-2. You'll either have to go back and edit that in manually, or wait until Darkcanuck gets that fixed. --Voidious 02:43, 19 August 2009 (UTC)

(edit conflict) Cool! I'll send you another API key for this app -- that way I can track it's usage separately from your twitter app. I'm assuming it uses the same query strategy?

I thought I fixed the glicko-2 scaling when you brought it up? --Darkcanuck 02:46, 19 August 2009 (UTC)

Hmmm, fixed the 'rankings' query but not the 'participants' -- should be fixed now! --Darkcanuck 02:54, 19 August 2009 (UTC)

Awesome, thanks! I'm not sure what you mean by "query strategy"? Some of it's very similar and I was able to adapt some of the twitter app code, but this is PHP while that is Perl, so there are some differences. Oh, if you mean about watching until it hits 2000 battles, no ... this is just taking a snapshot from the server right now. So you'd have to use this after it's stable. --Voidious 02:57, 19 August 2009 (UTC)

I meant how you're querying the data. Assuming that you're doing a rankings query, then maybe a participant query if the bot isn't active? --Darkcanuck 03:02, 19 August 2009 (UTC)

Ah... Right now, I'm actually doing both each time, but I can tighten that up a bit. =) --Voidious 03:05, 19 August 2009 (UTC)

No need, I'm sure anyone using this to update their version history has probably refreshed the rankings at least 100 times already. =) --Darkcanuck 03:08, 19 August 2009 (UTC)

Format

If anyone would like some other formatting options, let me know. I don't want to get too crazy with lots of configurable options, but it should be very easy to add new formats. For instance, a third argument of "zyx" could do it like YersiniaPestis/VersionHistory. --Voidious 14:56, 19 August 2009 (UTC)

If you don't mind, I'd like an argument so that it only displays APS, Survival and PL, with PL just as number of losses (ie. no total score). Basically, just like it is on DrussGT/Version History =) Thanks. --Skilgannon 15:37, 19 August 2009 (UTC)

Personally, I like DrussGT style, except I am also interested in Glicko2 rating, as I think that would be less affected by some other bot in the rumble not being finished it's pairings. --Rednaxela 15:45, 19 August 2009 (UTC)

Ok, check it out:

• {{subst:rumblestats:roborumble|jk.mega.DrussGT 1.3.10|gt}}
• APS: 86.92 Survival: 94.1 PL: 2 losses
• {{subst:rumblestats:roborumble|ags.rougedc.RougeDC balance|gt2}}
• APS: 82.79 Survival: 91.33 PL: 10 losses Glicko-2: 2086.9

Rednaxela, is that OK for Glicko-2 placement? And let me know if you greatly oppose "gt" and "gt2" as the style names =), but it seemed descriptive and short. --Voidious 22:31, 19 August 2009 (UTC)

Nice. I wonder though.... why don't you do the simpler thing for styles: Have the extension output wikitext like {{RumbleStatsDefault|name=voidious.Diamond 1.272b|rumble=RoboRumble|link=http://darkcanuck.net/rumble/RatingsDetails?game=roborumble&name=voidious.Diamond+1.272b|aps=84.58|apsrank=4th|plwin=720|plloss=17|plrank=14th|glicko=2124.1|survival=92.51}}, then we fill Template:RumbleStatsDefault with something like [{{{link}}} {{{rumble}}}] - APS: {{{aps}}}% ({{{apsrank}}}), PL: {{{plwin}}}-{{{plloss}}} ({{{plrank}}}), Glicko-2: {{{glicko}}}, Survival: {{{survival}}}%. That way, the styles would be part of the rumble and the extension never needs to care about styles except for accepting an optional parameter of what template to use :D --Rednaxela 00:07, 20 August 2009 (UTC)

Honestly, I was a little bit against doing some kind of "infinitely configurable" robust solution like this, because I don't think we will need that many formats and they're easy to add by hand. But what you propose sounds pretty solid. I'm not sure what you mean by "styles would be part of the rumble". But yeah, you could pass the template you want to use as the third argument (defaults to "RumbleStatsDefault"), the extension returns the wikitext template code like you say, and I think you'd need an additional "|subst=subst:" in one of these places to allow the nesting of the templates. Maybe I'll tinker a bit later... --Voidious 00:24, 20 August 2009 (UTC)

"styles would be part of the rumble" was a typo for "styles would be part of the wiki" :) --Rednaxela 00:28, 20 August 2009 (UTC)

Ah, gotcha. =) I should add that when I was semi-against an infinitely configurable solution, I was thinking more of a printf style thing where you pass the format string, and it just seemed like overkill. This seems pretty elegant. --Voidious 00:31, 20 August 2009 (UTC)

All set, and very cool I must say. No additional "subst" tags needed, after all. Take a look at the source of Template:RumbleStatsDefault for all the tags to use (slightly changed from your post above). We should update the RumbleStats page to include all this template usage info sometime, and maybe list some of the alternate styles available (once we get a few). But for the moment, I'm tinkered out. =) Enjoy. (PS - this means "gt" and "gt2" don't work any more, but I did make GTStats.) --Voidious 23:47, 21 August 2009 (UTC)

Error in calculation

I think there is something wrong with the way positions are calculated, when the stats for YersiniaPestis 3.1 were calculated, version 3.0 and Shadow were tied at first, and then DrussGT and version 3.1 were tied. The RumbleStats said my PL rank was 2nd, but I believe it should take into account the fact that the first place is a tie. Currently there are 742 robots, I will add stats for ry.Worst 1.0 which has a PL score of 2 tied to no one, if the PL rank it shows is 742 then I guess it was bad luck of when I added the stats, maybe it was at second place and I saw it wrong in the rumble for some reason, but I think it will show a much lower value (RoboRumble ‒ APS: 0.42% (742nd), PL: 1-740 (742nd), Survival: 0.85%). --zyx 22:55, 30 August 2009 (UTC)

It showed 742, I withdraw all, I probably made something wrong before. --zyx 22:56, 30 August 2009 (UTC)

Actually, the reason it showed 2nd is: when calculating the rank of a bot, it ignores any other versions of the same bot. I think that's generally the desired result, since your YP 3.1 would have been tied for 2nd (not 3rd) if 3.0 were removed. If a different bot had been tied for 1st, YP 3.1 would have shown 3rd. (Of course, to truly ignore the other version of same bot, it should adjust everyone's PL score to ignore that bot, but doing that would get pretty messy.) Hope that makes sense. --Voidious 23:05, 30 August 2009 (UTC)

Ah ok, that not only makes sense but is also good to hear I'm not crazy =), thanks for the reply. --zyx 23:13, 30 August 2009 (UTC)

Link this from somewhere

I'm not sure if this article is linked from somewhere, but I couldn't find that link (I used twitter to get here). I think the RumbleStats are usefull enough that they should be mentioned somewhere, although I'm not sure from where that should be. --Positive 16:40, 21 September 2009 (UTC)

I believe this page is linked from Category:RoboRumble Stats and all the stats template, but I agree with our, actually. » Nat Pavasant » 17:01, 21 September 2009 (UTC)

Hi, Voidious.

Please update RumbleStats to use the LiteRumble instead of Darkcanuck's server.

Also, since the LiteRumble is a lot faster than the old rumble, is it still strongly requested that we use the subst: tag?

I will, but it's not trivial since I need to write parsing code for the LiteRumble pages to get the data. I also need to update VoidBot so we can have the RoboRumble archives and Twitter account. But I did forget about RumbleStats so thanks for mentioning it.

And yes we should still use the subst: tag - why not? Otherwise it needs to hit the rumble server for every page view.

Thanks.

I see why it would be a bad idea to use RumbleStats without the subst: tag on long version histories and the like, but it would be nice on bot pages so the RumbleStats would always be accurate and up-to-date.

The current RumbleStats requires a version, so it wouldn't quite be a drop-in that would keep the rating for the current version on a bot page without some changes to RumbleStats first. But I do love the idea of having such a thing. It could be a good task for a wiki bot instead of pinging the server on every page view. I've already got something monitoring changes in the rumble, so maybe at the same time it tweets, it could check for a wiki page with the bot name, and if it has a special tag, or any RumbleStats tag, replace it with the RumbleStats for the new version.

We'd have to make an initial pass first for existing bots. That could possibly also be aided by a wiki bot. Like for every bot in the rumble, check if it has a wiki page by the same name, with an info box and no RumbleStats tag. If all those things are true, put the RumbleStats somewhere in the info box. (Just trying to be conservative about screwing things up.)

I like this idea. How about parsing the participants list to see what the latest version is?

Oh yeah, you don't need to hit the rumble server directly for that initial pass, just let RumbleStats do it. But for keeping it up to date thereafter, I think we might as well let the @roborumble monitoring code also handle updates instead of doing a full rerun every time. I guess what that would miss is any existing bots that don't get new versions but later get bot pages.

I'll see if I can add a nice API for data retrieval later this evening, now that I've got the priority battles sorted out and the rumble stabilisation coming along nicely.

API is available!

Ok, this is all good to go. Didn't include ANPP in the default template but it's there if you want it. (Skilgannon, np to remove it from here and rankings archiver if you remove it.)

I just re-implemented the BatchRankings using numpy (which uses C bindings for acceleration), and it also reduced my memory usage from a peak of ~900MB to ~270MB, so no worries any more about ANPP being removed =)

I also added a transform on my APS matrix so that (A)NPP and KNNPBI is perfectly consistent, even if the pairings are not synchronized.

I'm also considering doing a linear regression for the KNNPBI so that we get better interpolation of the expected PBI, particularly at the end points. I'm not sure if this defeats the whole KNNPBI concept though, by introducing 'expectations' based on the APS.