Talk:Darkcanuck/RRServer/Roadmap

Why Re-Create Battles?

I think "delete all my battles & re-load all battles from ABC's server" should definitely be on the list of next steps. It might even encourage my contribution to world peace. --Simonton 16:46, 26 September 2008 (UTC)

I didn't realize that all battles were available on the old server architecture. For some reason, I thought it only kept summaries, but if I can import all that data, that would be great! I won't be deleting any battles from the new server though -- almost all of the data submitted has also been relayed to ABC's server, so his server should be a superset of the two. For an import I would just pull in all of ABC's data (plus the Elo rankings, it would be nice not to have to recalculate those) and there's a duplicate checker which would throw away duplicated results (bot, challenger, timestamp + user IP determine uniqueness). But if you want to contribute to both, just upload your results to mine and they'll also be relayed to ABC's. And keep your "remove old" and "ratings" URLs pointed at ABC's server if you want to flesh out the pairings over there, my clients will keep the pairings here in line. -- Darkcanuck 17:01, 26 September 2008 (UTC)

The old server never deletes battles, it keeps a (huge) log of all of them. That's actually what led to the first file corruption problems, disk space usage can grow a lot when you leave it running for years. Keep that in mind when you design your battles table, it can easily grow into the millions of rows. BTW, in my server I did a backup/reset of that log somewhere around 100 battles per bot, you should download the battles_* files form the backup dir and append the ones in the main dir. --ABC 17:22, 26 September 2008 (UTC)

Cool. I had a look at those files, it should be straightforward to create an import script, the format is simple. But weren't there some issues with bad results early on? Did you remove all of them by hand, or do I need to filter them out? Also need to figure out how to keep the two systems relatively in sync. Duplicate checker or not, if all that data gets imported and we want to import some updates later on, it might crush the server to keep having to re-import a 40MB battle log.

Storage space is something I've kept in mind. I've tried to keep the battle results limited to fixed-width binary fields and have some more space-saving plans in the works. Should be much more compact in a database than in plain text files. But I'll probably periodically archive retired bots' results to keep the table lean & mean. -- Darkcanuck 18:24, 27 September 2008 (UTC)