Talk:RoboResearch

SVN Update Notes

If you update from SVN, be ready to change your .cfg files. The example "run.cfg" is updated - it's still easy. Perhaps easier than they were! --Simonton 04:25, 16 September 2008 (UTC)

HSQLDB No More

Beware of abrupt shutdowns of the database server. I just lost all my scores between Simonton/PFResearch 0073 and 0083, apparently because I closed Eclipse which simply killed the database process, instead of shutting it down nicely. I certainly thought it would handle such things better than that, and I have done it many times before with no consequence, but apparently my timing was bad this time. Does anyone have a good recommendation for a more reliable database that can be run without firing off a separate process, if desired? --Simonton 07:51, 18 September 2008 (UTC)

It looks like the Apache Derby implementation distributed with Java 6 would be the natural choice! Expect that change to come in the near future, for fear of losing more of my data otherwise! --Simonton 15:04, 18 September 2008 (UTC)

It happened again. This time I only lost some data for 0088. I wasn't going to be upset if I lost alot more, but it just confirms this is a switch that has to be made. For now I'm making a copy of the database directory before I kill the database server. Sending the sever the shutdown command via SQL would work, but that's too much effort. They should make it accept a clean shutdown command from the terminal in which you launch the server, that would be handy. --Simonton 02:21, 23 September 2008 (UTC)

Buggy when running from GUI

Hey Simonton if you're around. I found a bit of a odd bug when trying to run the TCRM here. Under the GUI it always fails with:

Thread 1: Unrecognized output from robocode, "Aborting battle, could not find robot: apv.AspidMovement 1.0".  Killing battle.

whereas in the CLI it works just fine. Not sure why all this is but if I find a fix I'll tell let you know. --Rednaxela 07:10, 21 December 2008 (UTC)

Hmm, okay, I discovered that the problem disappaered if I cleared the working_dirs directory, and renamed all the bot jars to match the version number they have in the properties file. It seems that RoboResearch doesn't like jars that are differently named than the default, and some such non-default names are included in the zip files for challenge bots. Anyways I have it working now, and maybe this info posted here might help someone else. --Rednaxela 19:06, 21 December 2008 (UTC)

Ran into this - thanks for the info --Miked0801 02:13, 22 May 2009 (UTC)

Melee

Has anyone used RoboResearch to run melee battles? Looking through the code, it seemed like the part that runs battles would support running a melee battle, but I didn't see any way to configure this (from a .rrc file or otherwise). My melee Test Bed process is currently pathetic and verging on useless... I'd love some RoboResearch support to help me out. --Voidious 15:06, 18 May 2009 (UTC)

I'll take that as a no on anyone using it for Melee? I'm going to try and get it going, then. I'm already like a fish out of water in the Melee arena, a powerful testing tool would help me big time. --Voidious 15:41, 19 May 2009 (UTC)

Sweet, got Melee working (thanks to a lot of the guts already supporting it), though it's not incredibly elegant and the output isn't quite right yet. Good enough to be my new Melee testing utility, though. =) I'll post it or contribute it back to Simonton if/when I clean it up a bit. --Voidious 02:04, 20 May 2009 (UTC)

Got a much more polished version of the Melee support working, doing it in a way that seems mostly elegant in the RoboResearch code. My previous hack was ugly and just didn't always work right. Most of the internals actually already supported Melee, so it wasn't that hard once I figured out the overall structure of the code.

• If you have "Melee" in your challenge name, it uses Melee mode. This probably should be done another way, but this makes sense, works fine, and it's easy to make it something else if we want.
• If it's Melee, all the bots in a category are put in the battle at once. Multiple categories gives you multiple battles per season.
• 10 bots per battle max, a limitation in the existing code / database structure. Also, it uses 1000x1000 for battles with > 2 bots, that was already in there too.
• Scoring is displayed just like it would be in the MeleeRumble: one battle yields 9 separate scores, with the category score being the average of those, and overall score being average of categories.

What I don't have done:

• When the CLI is showing what battle it's running, it just says "<challenger> vs <first reference bot>".
• I haven't worked with the GUI at all because I am using the CLI. The only change I needed to make to the CLI (besides the previous cosmetic change) was for the JAR copying: before, it was copying the first bot from each battle, while obviously I need to copy all bots from each battle.

I'll share this and/or share it back to the SourceForge project shortly. I feel so much less lost in the Melee world being able to run some serious benchmarks. =)

Please do - Infinity and DustBunny would love some number tweaking help to unseat Kawigi's monster Lib. The GUI stuff is my current preference, but beggar's can't be choosers :) --Miked0801 16:26, 28 May 2009 (UTC)

--Voidious 03:12, 28 May 2009 (UTC)

Alright, just made a few tiny cosmetic changes to the CLI and the GUI (the "<your bot> vs <reference bot>" stuff). Viewing results for 45 bots (my current melee test bed) in the GUI isn't pretty, but you can do the "Copy as wiki" into a text editor if you need it more compact. Not that it's that pretty as pure text, either, but at least it's not a 2500 pixel wide window.

Anyway, here it is: void_roboresearch_melee_01.zip. That's the same base stuff you'd get from the SVN checkout plus the binaries compiled for 1.5. You can just overwrite your 'bin' dir with the one from this .zip if you already have RoboResearch setup. (Hopefully Simonton doesn't mind me distributing it like this, but I saw no such mentions in the source and I'm going to get this stuff back to him sometime...) Let me know if you hit any problems - enjoy. =)

--Voidious 01:08, 29 May 2009 (UTC)

Grabbed and in use - thanks! --Miked0801 10:23, 29 May 2009 (UTC)

Getting started instructions

Has anyone here got this to work at all?!? It'd sure help with testing... --Miked0801 23:36, 18 May 2009 (UTC)

You mean like to work for 1v1 / not Melee? Yeah, I've been using it. What kind of problems are you having? I tried long ago and hit issues which I attributed to the Mac JVM. Since I've been Robocoding again and using SoyLatte Java, it's worked fine, but it's also a different version of Robocode, so it could be that too. --Voidious 02:35, 19 May 2009 (UTC)

Mainly, I'm not a java hack. There are no step by step instructions nor a downloadable distribution so I didn't even know where to start. Being able to run batch battles locally would be awesome. --Miked0801 05:04, 19 May 2009 (UTC)

I bumbled through a lot of stuff, myself, but I did get it working. And it's great once it's working. =) I've posted a quick and dirty write-up of the process below. It's for Mac / Unix, while for Windows you'd use ; instead of : for Java classpath stuff. But I also zipped and uploaded my roboresearch dir (minus a few dozen of my dev versions) here, if that helps. Indeed we should update the page with some polished instructions, but I just wanted to get something up before bed here...

Thanks for the files, I bumbled around and actually installed the sun java compiler, then decided to just use your files. Everything seems to run just fine, though the CLI output is ugly. I'll dork around with the GUI next. --Miked0801 01:05, 22 May 2009 (UTC)

And I have fallen in love with the GUI version of this. This will allow me to test locally instead of testing distributed over the internet. Wee! --Miked0801 02:13, 22 May 2009 (UTC)

to install from scratch:
svn checkout https://roboresearch.svn.sourceforge.net/svnroot/roboresearch
cd ./roboresearch/trunk
mkdir bin
javac -cp src:hsqldb.jar -d bin src/roboResearch/CLI.java
install a copy of Robocode to roboresearch/trunk/robocode_install

put your bots in robocode_bots

run battles with something like:
java -cp bin:hsqldb.jar roboResearch.CLI myBot.cfg

for multi-threaded, setup server with database_server.sh, then 
do something like:
java -cp bin:hsqldb.jar roboResearch.CLI -S -t 2 myBot.cfg

docs/getting_started_running.txt has some info, 
but some is outdated (boo)

--Voidious 05:28, 19 May 2009 (UTC)

Oops, I nearly forgot, one more important detail: I did have to actually edit the source to fix one thing. Some Robocode message had its wording changed, and RoboResearch was aborting if it didn't recognize the message. (I should find that and fire it back to Simonton.) So anyway, you should definitely grab my .zip and use that or compile from that source. --Voidious 05:33, 19 May 2009 (UTC)

• Well, or instead of the CLI thing, you can use the newish GUI version. I find it's rather nice, and if you want to do multi-threaded it can manage all the threads and has no need to seperately start the database server etc. I find it rather nice to be able to compare versions directly in the GUI version as the battles run... ;) --Rednaxela 07:21, 19 May 2009 (UTC)

Again, put a distribution or something together with instructions. I'm a C/C++/Asm programmer, not a Java dude - as you'd know if you look at my very C like code :) --Miked0801 16:10, 19 May 2009 (UTC)

Hi! I'm also interested in RoboResearch but I don't get on well with it so far. :(

1. I downloaded voidious's zip and tried to run the run.bat. : [1]
2. Then I modified TUI to CLI in the batch file. : [2]

--HUNRobar 17:13, 13 June 2009 (UTC)

run.bat is out of date and only runs some of Simonton's tests anyway. Best way to get started is to use the GUI: "java -cp bin:hsqldb.jar roboResearch.GUI" (mac/linux) or "java -cp bin;hsqldb.jar roboResearch.GUI" (windows). Remember to start this from inside the roboresearch dir. --Darkcanuck 17:27, 13 June 2009 (UTC)

Feature Request

One thing that is sorely missing is a standard deviation calcuation per battle (and/or overall) so that you can get a feel for just how accurate your results are. It would go a long ways towards telling someone just how many seasons they should run to get stable results. --Miked0801 20:56, 2 June 2009 (UTC)

I believe it shows something like that if you hover the mouse over a score for a moment in the GUI. ;) --Rednaxela 23:05, 2 June 2009 (UTC)