View source for Talk:LiteRumble

I see quite a few robots that haven't changed are re-running pairings today as if they had new versions. Any idea why that is?

Looking into it myself. From what I can tell a bunch of battles didn't load into the Batch Rankings, so it assumed that they didn't exist and pulled them from the participants scores list. They've been slowly added back by clients over the last few hours.

I've removed the section of code that removed the battles from the Batch Rankings, but that is just putting a bandaid over the problem. I'll have to look deeper to see what caused them not to load in the first place.

Looks like half of General 1v1 has incomplete pairings now. Should I put my clients into overdrive to fix it, or am I making it worse by running clients because of some bug?

Running a bunch of clients isn't going to make it any worse, from what I can tell it was a once-off problem to do with the backend instance being unable to load data. I've removed the mechanism it used to remove the bots, but I'm still not sure (and may never know) why it happened.

I've also just identified a bottleneck/threadlock which will severely limit the ability to upload from multiple clients at once without increasing upload latency to where it will spawn new server instances and cause my quota to be hit again, but I have a fix for that which I'll implement and test tomorrow. The load right now seems pretty healthy though, I see in the logs uploads from you, MN and Wompi, thanks guys. I'll let you know when you can unleash the full power of your machine(s) =).

When a competitor is removed from the rumble, is it´s data also removed?

It's data isn't removed, so you can still see it in the BotDetails, but the pairings info in the other competitors which points to it is removed. Otherwise over many versions the access to other bots will get slower and slower due to increased serialising costs.

Keeping pairing data for a while can help protect the database against faults in clients removing competitors from the rumble, only to be re-added again some time later.

That sounds reasonable, yes. Perhaps adding a 30 day error window, so only if the last battle was more than 30 days ago the pairing data in the 'alive' bot gets purged. Until then it is just marked as 'removed'. I think this purging and checking will have to happen in the backend, because the frontends are fully loaded right now with your and Voidious's uploads.

That is exactly what I had in mind.

The number of bots with not full pairings has gone up - we were under 400 yesterday and back up to 471 now. I noticed an over quota message from last night, was there another loss of data?

Let me know if I should dial back my clients or if there's anything else I can do.

The source of the problem has to be tracked down or the rumble will never stabilize.

I guessed it was the excludes feature from the clients erasing pairing data in the server. But looks like it is something else.

I suggest tons of logging, tracing all requests from clients.

Sorry guys, I was trying to see if I could use the marshal module to do my serialisation instead of cPickle because my local testing showed it is about 50% faster, but it corrupted a few bots from each pairings dict so I quickly changed it back. I'm not sure why it had these issues since I tested locally on the dev server and it worked fine, but anyway it is fixed now, and was a completely different issue to what happened before.

It did hit the quota last night, so perhaps tone down the clients a little. There's a threshold below which it is cheap to run, but as the load increases I start leaving the free quota for the instances as well (not just database writes), which gets expensive much more quickly.

Took my clients from 4 down to 2.

Can you protect against us overloading your server? Both to avoid hitting quota, and to avoid someone DDoSing your bank account :-), it seems like it would be good to have some throttling or something in place.

Refilling the pairings is going to take a while. Is it possible to tune things (for now) to support a higher client load, to prioritize overall throughput over losing pairings here and there, while consuming less quota?

I've been trying to think of a good way to do that, but the 'recommended way' using Task Queues (which I can then limit to 3-4 queries a second instead of the 5-6 I was getting yesterday) will break any reasonable way of having priority battles.

Also, there is no way to programatically retrieve the current quota usage stats, which means I can't do any auto-throttling.

I can tune it not to do database writes unless a bot has x or more pending battles, which is how I did it previously when on free tier, but the majority of the time is actually taken up with (de)serialising the pairings data, which is why I was trying to shoehorn in marshal. I'll add a min pending battles limit, and you can turn those clients back on, we'll see what happens. Of course, it will probably only hit quota tomorrow night if it's an issue, since today has been pretty slow.

I´m thinking in building a custom client which groups results from all local clients and uploads them in a single thread, so the server needs only a single instance per user to receive data.

Combined with multi-threading, clients can keep running battles in parallel while a single thread uploads everything, making it faster than the current client, while at the same time consuming less server resources.

That would be great. I'm not sure how you'd do priority battles though, would you have a local queue which would be filled, and you just take from there? I guess I could sort-of do this with task queues, but it wouldn't be very pretty.

Priority battles are downloaded by the uploader thread after each pairing is uploaded. They would be sent to a queue, which would be consumed by the clients.

Battle results would be sent to another queue, which would be consumed by the uploader thread.

That is the basic idea. You can add some logic inside the queues to make them smarter, like dealing with duplicated battles, excessive amount of data, or lack of data and fallback to random battles.

I've essentially implemented what you've said here but on the server side using a Task Queue, the only thing we lost was on-the-fly updated battle numbers, but those aren't really being used now that we have priority battles. Also, priority battles are delayed by up to 100 pairings per rumble, but this new design should mean that stuff sticks around in local memory longer than before.

Once I add contributor stats I'll also add information about the current amount of queue backlog, so people can decide whether or not to run a client.

If you check your clients you can see that the uploads are going much quicker, and it tells you it is adding it to a queue instead =)

Yay, back to full pairings in General 1v1!

Same for PanzerGeneral in minimelee: [1]

OK, I modified the code that checks that pairings are still in the rumble to also check against the bot name. They should be fixed next time they get a result upload.

Sweet, already got those fixed with my clients. LiteRumble's looking pretty awesome!

Dunno if this is the same bug or a different one, but wompi.Kowari 1.4 and ag.Gir 0.99 show 989 pairings in the main rankings, but 988 in the bot details. (Should have 988, there's 989 bots.)

This was something different, but thanks for catching that! Fixed.

If you refresh your LiteRumble home page, you'll see a link at the bottom which takes you to the all-new LiteRumble Statistics page. This page is updated once an hour, on the hour, and contains the latest contributor information as well as how big the queue is and the expected processing backlog based on how many were processed in the last minute.

I also improved the OverQuotaError handling so that from now on your rumble client will see a regular server response and won't go crazy with the uploading errors. Also, from now on, when the queue is full it doesn't give an OverQuotaError, but instead waits half a second and sends a nice message back to the client saying that the upload was discarded.

Enjoy!

Is this upload queue a task queue from app engine?

Yes. Because of this it doesn't have pure FIFO behaviour, sometimes new items get run before old ones, but for the most part it is FIFO.

Is the upload queue size the amount of battles which still needs to be processed?

Which means while the queue size is greater than zero, we can stop uploading battles and the server still has work to do?

Exactly. The amount of time it is estimated the work will take is the Queue Delay, but this is only based on the average speed in the last minute so it fluctuates a lot.

Did the queue replace the older cache? No more evicted battles?

Also, I know there is a quota for task queues, of about 100,000 messages/day.

No, the older cache is still there. It serves two purposes, 1) I don't have to write to the datastore every single pairing, which is slow, and 2) often a single bot gets lots of battles all in a row (eg just released) and in this case my database writes will be cut almost in half if I am caching them. There are virtually 0 evicted battles now that I keep just two queues for 1v1/melee, instead of 1 queue per rumble with a minimum of 5 unsaved battles before a bot was written and a minimum of 10 bots needing to be written.

I think that was the old quota for task queues and it has now been increased. In the control panel I see a quota of 1,000,000,000 and I'm less than 1% of that.

Hi mate. I just wanted to thank you for all the effort you put in keeping LiteRumble up and running - it is very much appreciated by me. Awesome job. Of course thanks too Voidious, Sheldor and all the others who spend her time to keep RoboCode alive. Right now I have not much time to help out - I barely could write a couple of bot code even on easter weekend :( . But let me know if I could drop some money or something else to show my respect for your work.

take care

Thanks. At some point I might ask for some paypal donations to cover costs, but not yet =)

Yeah man, LiteRumble is great! It's pretty remarkable Darkcanuck's server could just disappear one day and we have a viable alternative, with rankings already up to date, to switch to on the spot. And you've been lightning quick in adding polish and fixing stuff since it became the canonical rumble server. Great work, and I too am ready to pitch in for costs whenever you say.

Wompi,
Thanks for the mention. Though I doubt that my contributions to the RoboWiki have done much for Robocode itself.

I look forward to seeing how Kowari turns out.

---

Skilgannon,
Thanks for saving the RoboRumble. :)
Great job!

---

Voidious,
I agree. The LiteRumble's awesome!

I noticed pairings between 2 competitors are being scored differently. A vs B score is different than B vs A score.

LiteRumble could stabilize twice as fast if a given battle upload is updated for both competitors involved.

Certainly looks that way. At this moment, XanderCat 12.6 has run against all other opponents, but if I go look at some of the opponents who do not have full pairings, ones that XanderCat 12.6 has run against, XanderCat 12.6 doesn't show up.

Also, a number of values are coming up 0. Will those eventually update when pairings complete, or are not all scores currently tallied (either intentionally or due to a bug)? I know there was talk of dropping ANPP, so maybe that is 0 on purpose, but Vote for XanderCat 12.6 also comes up 0, even though XanderCat 12.6 shows up against PolishedRuby, who XanderCat gets top score against.

Those complex rankings are coming up 0 because they didn't have some battles run against them after the batch rankings was run, and got evicted from cache before their new data was written to disk. I think I'll do a disk dump at the end of each batch processing job, see if that sorts this out.

A lot of these problems are showing up because I put a bit too much 'eventually' into the 'eventually consistent' model.

Some of the complex rankings are updated in batch at intervals.

They are both updated at the same time, but for a long time I had very aggressive caching on to keep within the free tier. Because App Engine Memcache can be cleared without warning, this meant that often one bot would lose a battle before it could be written to disk, but not the other. Ironically, this is one of those situations where a higher client load on the server means that less data gets lost, since the bot was more likely to hit the 'write threshold' before being evicted from the cache. Now that I have more writes to play with, I've lowered the write caching thresholds a lot, and we should eventually see everything settling down.

Is the cache independent for each rumble? Or one big cache for everything?

One big cache for everything. I used to have a queue for each rumble that would fill up, and each time a bot had over X battles it would be written to the database. Of course, when the nanorumble only gets 1/10th or less the battles of the main rumble, the bots in it are more likely to be evicted, resulting in incomplete and asymmetric pairings. So I've just rewritten that section to keep just two queues (only split between 1v1 and melee), and bots are written as a batch as soon as the queue is a certain size. That way, even bots getting less battles will still be written because they are all pushing each other through, so it should stabilize better. Unfortunately I couldn't do this before because I was too limited with writes.

What is the queue size?

30. Bots with more than 1 unsaved pairing only count once, though.

Is the queue size per competitor instead of per pairing then?

I keep a dictionary of {bot.Name:unsavedBattles} and once there are more than 30 entries it gets written.

Let me explain that better: Each time a pairing is uploaded, the two bots are added to the dictionary with unsavedBattles of 1. If they already existed in the dictionary, their unsavedBattles count is incremented. Once the dictionary size is bigger than 30 (size = number of entries, not sum of unsavedBattles), all bots listed in the dictionary are written to database.

Dictionary size reaches 30 when at least 30 different bots had pairings uploaded.

I configured my clients to run/upload 30 battles per iteration. With meleerumble still having 3 battles per iteration. If all battles have at least one different participant, the server flushes the queue at least once per iteration, with higher chance of it happening exactly after the last upload leaving an empty queue.

It is an attempt to minimize the amount of evicted battles.

I miss the ELO rating column in the pairings page. When you viewed all pairings a bot has, ELO rating column allowed you to sort all pairings by opponent's strength.

There is no column which allows you to do that anymore. A sortable column with APS rank or league APS would be nice.

Interesting, that was never something I used. I'll look into it.

ERROR WRITING DATA!! QUOTA REACHED?

Oh no! I guess I'll stop my GigaRumble client. =) I've only been one running one slow client on my laptop, and with 30 slow bots it's pretty low volume - I think ~20 minutes per 25-battle iteration, at least.

Don't worry, I don't think it's you. I have 8 clients running at the moment in my lab, and it seems that they suddenly started producing a whole lot more battles over the last few hours (I'm not sure why). It's quite interesting, on the requests-per-second graph I can see when the latest Diamond was released quite clearly =) Check it out:

Again, I have no idea why that sudden burst of data came in. Considering it's about triple what I normally get, it would correspond with ~24 clients running regular rumble battles. Very strange. And no wonder I'm out of quota...

Hmm. If you have battles per bot 2000, and then they stopped running DrussGT / Diamond battles, I could easily see the battle rate tripling when switching to minibots or random battles.

It shouldn't switch to random battles though, because I've got priority battles going the whole time and the 1.7.3.2+ clients ignore the BATTLESPERBOT if there are priority battles waiting. I think I've identified a bug in the priority battle selection algorithm though, it is giving higher priority to pairings with more battles, rather than less (although that only starts happening once all the pairings are complete, at least).

Also, it hasn't been waiting on Diamond and DrussGT to fill out their battles like Darkcanuck's rumble, because until just now *everybody* was below 2k battles.

Ruh roh, my client's hitting this again.

OK. mrm.MightyMoose .2 vs logiblocs.Fire 1.0 received in 365ms
ERROR WRITING DATA!! QUOTA REACHED?

That makes no sense, it is well below hitting anything. There was also an issue with priority battles not having any options because everybody had full pairings, which was some errors, as well as malformed requests which were coming from my university connection, I think they had something to do with the SEACOM cable currently being down.

I've added some logging, so next time it happens I can see exactly what the problem is.

OK, fixed. It was having trouble with a mixture between numpy floats and native Python floats when writing them to the database.

Ah, so this was just some other error with the same message? That's a relief. :-) Turned my clients back on.

Try adding flags to the tables, it makes a tremendous difference.

I don't want to deal with the admin of uploading flags for new entrants. If I do add flags I want it to be able to be changed without any admin access.

A long time ago someone proposed some kind of automatic flag maintenance, much like how the participants list is maintained today. People add their flags in the Country Flags page and the server adapts.

On the note of possibly making this the default server, here is my unfinished flags pack. Now I say unfinished because I am using ISO 3166-1 alpha-3 for the country codes, but I don't have a flag for every country code. I was working on it. I have about 1/5th of them, but I lost interest when people really stopped mentioning the light rumble.

Thanks! I've been thinking about this for a long time. I've got a code-freeze for my MSc coming up at the end of the month, so once that hits I'll see if I can add some flags to the LiteRumble.

Flags are up!

Looks great!

I did notice one thing, though... Maybe make the blank flag transparent instead of white? It looks odd on the grey rows.

Done. You'll have to do a CTRL-SHIFT-R reload though, to clear the cache.

A feature: you can sort by flag type, which I think is something no rumble has had before!

It would be interesting to have a country leaderboard using aggregated score of bots from each country :)

Cool!

It was a good idea to change BLANK to NONE. Good job. :)

Currently the rumble uses ISO 3166-1 alpha-3 for the country codes. If anyone wants a flag that doesn't exist, i'll make up the flag. Until then, I suspect the current subset to be sufficient.

Currently supported are ABW, AFG, AGO, AIA, ALA, ALB, AND, ARE, ARG, ARM, ASM, ATA, ATF, ATG, AUS, AUT, AZE, BDI, BES, BRA, CAN, CHE, CZE, DEU, ESP, FIN, FRA, GBR, HUN, IRL, ITA, JPN, KOR, LTU, NLD, NOR, POL, PRT, RUS, SGP, SRB, SWE, THA, USA, VEN, and ZAF. Unless Skilgannon added more.

I actually didn't notice the BLANK until I had a good portion of the code up, then I just decided to stick with what I had :-p I added CHN, NZL and SVK.

I've also set out the RoboRumble/Country_Flags so that it makes it all pretty clear. I have a cronjob which parses the page once every 2 hours.

And thanks for all those flags BTW, saved me a ton of work!

You're welcome.

Oh on that note. The css style:

img {
    width: 30px;
    height: 18px;
}

Should help prevent any strange image sizing bugs.

I'm pretty hopeless with css, if you could provide that as an entire code snippet with example <img> tag and everything I'd be happy to integrate. Is there any chance of it going wrong if I keep all the images 30x18 though?

Gah, liquid threads ate my response again.

Anyway that is a complete CSS code snippet. It is just saying all images (rather all <img> tags) should be displayed at 30x18. If you just leave this bit of code out, the <img> may display at any browser specific size while loading (before the browser can load the image and see how big it is). I know some of them default to things like 100x100 while loading. Which may make the page look... weird while loading.

This was more of a problem when each package had a separate flag (and thus we had to load more images, which took longer), but it is still a valid problem.

So I just stick this in my styles.css file and everything sorts itself out?

Pretty much.

Unless you add more images which are not flags. That css snippet affects all images on the page.

Yeah, I said "all images" and "all <img> tags".

Ok, I have a simple API up. Just add &api=True to any Rankings or BotDetails page and it will return in nice easy-to-parse JSON-ish format. I say JSON-ish because real JSON puts double-quotes around everything, whereas I'm lazy and don't feel like doing a ton of double-quote escaping, and besides, we don't have commas or colons in our data so there isn't any risk.

If you don't want the entire pairings detail from the BotDetails, add &limit=0 to the page and it will leave them out.

Some usage examples:

http://literumble.appspot.com/Rankings?game=nanorumble&api=True

http://literumble.appspot.com/BotDetails?game=nanorumble&name=sheldor.jk.Yatagan%201.0.5&api=True

http://literumble.appspot.com/BotDetails?game=nanorumble&name=sheldor.jk.Yatagan%201.0.5&api=True&limit=0

Of course, I'd rather you call Rankings once than call BotDetails with &limit=0 985 times, because although it doesn't generate the JSON, it still has to pull all the data in, which adds up for lots of requests. I'd ask that if you are getting more than 3, rather use the Rankings.

If there's anything else you'd like me to add to the API just ask, if it's already saved in my data it should be easy to whip something up to return it.

Sweet! I'll take a look at updating RR API clients in the next couple days. (That's @roborumble on Twitter, Category:Archived RoboRumble Rankings, RumbleStats, and BedMaker.)

I'll let you know how it goes or if I hit any snags. Hopefully the JSON libs I use don't mind JSON-ish. ;)

Well, the Perl JSON library is indeed unhappy about the lack of quoting, but I think I've got it covered:

$content =~ s/([\{,]\s*)([^:\{,]*):/$1\"$2\":/g;
$content =~ s/:([^\[][^,\}]*)([,\}]*)/:\"$1\"$2/g;

=)

Cool, got it working with archived rankings, eg: RumbleArchives:RoboRumble 20130324. This checks back hourly when rumbles don't have complete pairings, but that may be forever without priority battles so I'll kill it in a couple days. =)

Only useful thing that I found missing so far is that Darkcanuck's results came back sorted and had a "rank" field. Seems like you have logic for sorting in there so it'd be nice if you could honor the "order" param like you do for the normal pages, but not a big deal (/shamefully hides his bubble sort). The quote thing wasn't a huge deal for me but that might also screw some people up since I imagine most clients use a JSON parser instead of doing it manually.

And @roborumble, yay: [1]

I'll look at RumbleStats and BedMaker tomorrow. Will be nice that other people can actually use BedMaker now. =) If anyone's interested, I had to make one more tweak to my regex to properly quote the &limit=0 format:

$content =~ s/([\{,]\s*)([^:\{,]*):/$1\"$2\":/g;
$content =~ s/:([^\[][^,\}\n]*)([,\}]*)/:\"$1\"$2/g;

OK, you've convinced me, I've added the quotes :-p I've also added "rank" to both Rankings and BotDetails, and the sort works too.

Nice, thanks! I turned off my quoting regex this morning and it parsed fine.

I didn't think adding qoutes would be that hard if you don't have to escape anything.

I would like a query which returns the full pairing matrix, with scores from all pairings. Number of battles from all pairings would also de nice.

Probably the costliest query anyone could ask.

Unfortunately, that would take more memory than I have available in a frontend, and frontends only give me 10 seconds to respond or the process gets killed. Besides, all I'd be doing in the background is doing a Rankings query to get all N of the bot names, then N x BotDetails queries to get all of the pairwise data, so it's nothing that you wouldn't be able to do just as effectively with the tools you already have.

If you do something like this, I'd ask that you put a half-second delay between all of the BotDetails queries just to leave spare capacity for client uploading.

But I agree, it would be quite interesting to see what trends etc could be found in the data.

It would be used mostly to try different ranking systems over the scores, and to build custom priority battles systems.

For building batch rankings systems I can understand, but it would take too long to do priority battles, which need to be run much more regularly.

Hmm, getting an error from API calls for Rankings, but not from the normal pages: [1]

Ah, thanks. Flags are up now, and the API returns the flag code as well, which is how that error crept in (index array out of bounds, now that I have another element).

Cool, I'll get the flags into the archived rankings next time. Thx for the quick fix.

I know this is about as low priority as it gets, but... The always opening a new tab when I select a rumble annoys me. =) I'm pretty conscious of ctrl or shift clicks if I want a new window or tab, so I'd rather just keep control of that myself.

I'm also starting to get annoyed by it, I thought it might improve useability not having to click 'back' the whole time. It is now gone.

Please switch clients over to 1.8.1.0 to take advantage of priority battles for mini, micro and nano as well as fix the skipped turns issues.

1.7.4.2 will be disabled in the next few days.

I was playing around with the backend and restarted it a few times, which chowed my new (increased) daily quota pretty much instantly. So just hang in there for another hour and a half and it will be back to normal, I promise.

Aaaand it's back.

Guys, if anybody has some spare CPU-power lying around, could you point it at the 1v1 and melee rumbles? It seems that I was getting bad stability in the mini/micro/nano rumbles, and I've just fixed that, but we have a bunch of pairings which need to be filled in. I also want to see how it handles a higher load, so I know whether it is worth upgrading to paid. Thanks!

Sure, I'll go from 1 to 4 in a couple hours. ;)

Seems to handle my four clients ok so far, running mostly/all nano battles too.

Edit: Oops, not all nano battles. But they're running pretty fast. :-)

Ok, looks like we managed to overload it. ;)

That's from hitting the Read quota. That hasn't happened before, it might have something to do with me clearing the memcache so that I could enable priority battles for mini/micro/nano. The reset is in an hour, let's see what happens ;-)

Are there priority battles happening for the lower weight classes? Even Mini 1v1 mostly doesn't have full pairings. Was it just way behind or is it not working correctly?

The client doesn't accept priority battles from the lower weight classes. I've filed a bug and Fnl has fixed it, so as soon as 1.8.1 is released we'll be switching to that.

https://sourceforge.net/p/robocode/bugs/355/

I've made the plunge, LiteRumble is now on the paid tier. So, open the taps and let's see what this puppy can do!

Sweet! Let me know if there's somewhere (eg PayPal) I can drop a donation. :-)

Also it's probably time we update a bunch of wiki pages to note the LiteRumble as the main rumble server.

I created a fresh "superpack" pointing at Skilgannon's rumble server: LiteRumble 1.7.4.2 superpack (31.5 MB) ... We can put this somewhere more prominent if we want to make the transition more official.

I fixed the participant list links to darkcanuck.net to point to Rednaxela's archive instead and downloaded most bots for all the rumbles in the above zip. There are still some stragglers with broken links that we should fix on the participants pages. I also set all the configs to ITERATE=NOT and updated the shell scripts to loop, which is my preferred setup. Just change your name at the top of roborumble/*.txt before running.

I don't know how many clients Skilgannon's server can handle - last I heard it was just a few. For now I'm just running 1 client for General 1v1.

Thanks. My client is in my university lab and I need the processing power during the day for the next few months at least, so feel free to run as many as you like. It should also have a more graceful quota-exceeded behaviour than it used to, but we'll actually have to hit that before I can be sure.

If you remove the CPU constant from the robocode.properties file, won't it automatically recalculate it? If so, might be a good idea to remove the CPU contant from the robocode.properties file in the superpack. Otherwise, anyone who runs the superpack without thinking about updating the CPU constant will be running with the CPU constant packaged in the superpack.

Ah, good call. I'm not sure it works that way but I'll test it. I'll also shore up the missing bots.

Alternatively what about just outright removing robocode.properties? That's what I always did when I made superpacks.

I gotta say, I love the Vote ranking. Gives you a completely different perspective on the rumble.

Yeah, I really like it as well. I'm thinking of doing away with (A)NPP though, it seems a bit redundant, and it uses tons of memory to calculate.

I have no problem with that. But I think we should get some other opinions on it.

ANPP is actually the one I love =), but I'm fine with dropping it if you want. I find it pretty meaningful in the GigaRumble.

Hmm. My main concern with it is that it requires me to do a full nxn grid of scores, which is taking up a full GB of RAM on the backend, and causing soft-kills. If I could somehow implement it as an incremental score, which is updated along with APS, I wouldn't mind it so much.

At least with the KNN-PBI I only need sqrt(rumblesize) in memory at once.

Pretty sure you could do it without nxn grid. You could do min/max for each bot just loading one row (n) at a time. Then ANPP vs each bot the same way. Then each bot's avg with another pass one row at a time. Unless even that process requires nxn with the code/data model. I certainly don't see anyway to do it drastically quicker, like on par with APS.

It's still probably the most CPU intensive and least useful of the rankings, so I totally agree it's a good candidate for the axe.

I think incrementally updated ANPP could actually be very fast most of the time if you had a table to cache the highest and lowest scores against each bot. Most of the time, the bot holding the highest/lowest score against another won't change, so those times only the bot with the newly submitted score would be affected. It could also be made less resource intensive by not including a bot in ANPP until it's pairings are complete, further reducing how often the min/max score against it changes. If the highest/lowest score changes, it affects the resulting ANPP score of all other robots, but that update could be done with a low memory footprint I'd think.

Yeah, I was trying to think that through... Agree that obviously min/max change is fairly rare and that's a fast case if it doesn't change. If min/max does change, you need to recalculate everyone's ANPP vs that bot. Then you need to update everyone's overall ANPP, but that doesn't need to be a complete recalculation, just (((overall * numBots) - oldScore + newScore) / numBots).

Is Vote determined by how many opponents a robot gets top score against? I vaguely recall something like that. Kind of interesting to see the shielders near the top when sorted that way.

Yeah, my next comment is that LiteRumble could use some info on what all these crazy rankings are. :-)

Vote is what % of bots you are the best against. (Each opponent "votes" for its worst matchup.)

I like the Vote ranking. It finally gives exploitative bots (like Epeeist) the recognition that they deserve.

I like it too - Diamond is #1. ;) Of course I presume EnergyDome steals quite a bit of score from DrussGT.

Been thinking I'll probably try a LiteRumble instance when I get to creating a BerryBots Rumble. I was wondering if you could say, at a high level, if there's much Robocode-specific about it? Just looking for some general info, I'm happy to tackle the details myself when I get into setting it up.

I know obviously the protocol is whatever the RR client uses, and the scoring is sorta Robocode-centric. But that shouldn't be too hard to adapt if I need to. I guess the next thing that comes to mind is any CodeSize related stuff? I'm starting on the control API stuff soon, which would enable writing a rumble client, and someone posted the first public user-created bot on the forums today (woohoo!), so I might experiment with it soonish.

Thanks!

It doesn't have anything Robocode dependant, and doesn't even know that the roborumble is more related to the mini/micro/nano than, say, the meleerumble. The homepage does a manual categorisation based on whether they are 1v1, teams, melee etc., but that's as far as it goes. You could even create, say, a 'berryrumble' on the same server if you wanted.

Awesome, thanks!

Aw, I saw in the diff you removed the "in case I ever get" line and thought maybe that meant you'd added it. ;) Though I'd probably still be a ways behind you for a while...

I wasn't keeping track of who anything was from, so it would start from scratch =)

I'm thinking of something like "who contributed in the last hour", or something along those lines, just so that it's easy to see if anything needs running.

Is there any way to see individual battle scores in the LiteRumble? I didn't see any way to do that. It is the one thing I would really miss about Darkcanuck's server if we only had LiteRumble in the future. I find individual battle scores to be of great use from time to time (e.g. if you have a score of 62 , it makes a big difference if the individual scores are 61, 63, 62 or 86, 88, 12). And what client gave that score is also of use. It can help identify Robocode version issues. For example, just go look at the history of battles for Krabby2, and it becomes obvious something broke in Krabby2 after Robocode 1.7.3.0+ came into use.

All that aside, if Darkcanuck's server ever goes away, LiteRumble as it is would be far preferable to no rumble at all.

Unfortunately it only keeps the averaged result and the number of battles. If I changed it to keep pairings as well I suspect that each bot entry might exceed 1MB, which would break Google App Engine rules. I've considered the possibility of keeping 'min' and 'max' though, would that satisfy you?

That would be nice. Or maybe some kind of confidence or standard deviation or something. Not a big deal, but I would use it if it were there.

If you persist each battle independently, without associating it with other records (no foreign key constraints), then the only limit is total database size.

It would also increase total database writes, which is my current bottleneck.

Any idea how much load your server can handle now? Just curious... Still thinking about the feasibility of switching to a new rumble server.

If I upgrade to paid it would probably handle as much as Darkcanuck's server. As it is it is mostly limited by number of database writes, probably 4 1v1 clients and 2 melee clients at i5/i7 execution speeds would be the limit. If I/we upgrade I could also tune the caching so that it loses less data.

I'm actually looking at doing a Java implementation just because the number crunching is so much faster. Also now that I've done the Python one it would just be a translation and not an entire rewrite.

Does it mean you have a bug if I have NPP over 100 against some bots? [1] Or are the ranges only recalculated at some interval, so maybe I scored higher than the previous max score against that bot?

Because the different halves of each pairing are stored separately, each might have a slightly different idea of what its score is against each other because one of the scores being dropped from cache before it is written to disk. Now that I'm not trying to complete pairings for Melee and Roborumble I might as well reduce the caching to less dangerous levels. I've also considered fixing this in my batch processing by taking averages, but never got around to it.