Talk:RoboJogger

I have been using RoboJogger for movement for about one week and it was working perfectly. However, when I tried testing my robot on TCRM and MC2K7 it didn't work. My robojogger's robocode is 1.7.3.0 but I don't think that's the case since it produced similar results to what I had on my robocode 1.9.3.2. It seems like RoboJogger only accepts PERCENTAGE_SCORE. Below there is my TCRM file. Thank you in advance.

Targeting Challenge RM
AVERAGE_BULLET_DAMAGE
35 rounds

Easy {
   apv.AspidMovement 1.0
   dummy.micro.Sparrow 2.5TC
   kawigi.mini.Fhqwhgads 1.1TC
   emp.Yngwie 1.0
   kawigi.sbf.FloodMini 1.4TC
}

Medium {
   abc.Tron 2.01
   wiki.etc.HTTC 1.0
   wiki.etc.RandomMovementBot 1.0
   davidalves.micro.DuelistMicro 2.0TC
   gh.GrubbmGrb 1.2.4TC
}

Hard {
   pe.SandboxDT 1.91
   cx.mini.Cigaret 1.31TC
   kc.Fortune 1.0
   simonton.micro.WeeklongObsession 1.5TC
   jam.micro.RaikoMicro 1.44TC
}

An addition: Same things happen when I use robocode 1.9.2.5

This question probably has a really simple answer but I have been trying for over a month and still, I couldn't test one robot on it.

Oh, I am so dumb. I gave the new Robocode version; now it works perfectly.

2 bugs so far that I have found:

1) A left over java process seems to hang around for each full execution of RoboRunner. This needs to be tracked down and eliminated (could it be the callback queue in RoboRunner?)

2) When RoboRunner throws an error (for example, due to a missing robot), RoboJogger does not realize that RoboRunner has died and the controls to "stop" RoboRunner do not function. This results in RoboJogger being stuck and requiring a force quit. This should be fixed so that either RoboJogger correctly detects the failed start of RoboRunner or at least is able to reset everything if the command is given to stop RoboRunner.

Are these bugs present in 0.9.6? I'm finding it quite annoying to have to keep restarting robojogger =)

I'll be looking at fixing these bugs soon. I kind of forgot they existed for awhile.

I'm currently on hold because my basement server died awhile ago, and it had my source repository on it (in addition to my data backups). I have mostly new hardware now, but I'm still waiting on a new hard drive for the OS, as I at first bought a refurbished one from Newegg and it was dead (last time I ever try to buy a refurbished part). Once the final replacement hard drive arrives, I will have a mostly updated system with an OS drive and 2 1TB data drives. The old system was a Pentium 4 with 1GB DDR1 RAM, so it was definitely due for an update. I will also be installing the latest version of Fedora, which also means I can easily use Java 7 on it and possibly finally set up a distributed Robocode node on it (if that project is still alive).

Back on topic, I'm glad to hear that someone is still using RoboJogger. Knowing that gives me encouragement to get back on it and make it better. I will also explore the possibility of moving the source to a public location, should anyone want to tinker with it.

Hi, since i am only user of drc, the project is more dead, than alive:) But if you install it and experience problems, then you can email me (see Contacts) or patch it yourself: github repo :)

Not sure if its a bug with RoboJogger, RoboRunner, or my robot. However I'm getting spurious results vs some bots.

For example vs Tron 2.01 in the TCRM challenge I'm getting an average of 87% percent when running 10 seasons manually, but in RoboJogger I'm getting an average of 61%.

I'm running Robojogger on the Mac, latest version of Robocode. I'm aware that Tron seems to be quite a slow bot, I was wondering if my bot was generating skipped turns for some reason as my bot is not too fast either. Notably in Robocode I am not generating any skipped turn events from what I can see. How do I access the Roborunner bot output? Is there a way or not?

Manually means running by hand in Robocode directly FYI incase it was unclear. :)

Oooh just saw there was a Robocode update that fixes some skipped turns, it literally came out today so I'm going to re-test with that and see what happens!

Tried the new Robocode version and I'm still getting the same result - running manually in Robocode gives a higher score by around 25% for my bot vs Tron compared to running in RoboJogger. :(

Are you sure you're using the same scoring mechanism? I know the TCs define score as TOTAL_BULLET_DAMAGE/ROUNDS.

Yes, the setup for the RoboRunner config file is using AVERAGE_BULLET_DAMAGE. Note that the results for all the other bots in the challenge look correct. It only appears to be vs Tron for some reason, which is why I postulated it was because Tron appears to be a slow bot which may cause skipped turns.

It might be because Tron starts firing - if it doesn't get its config file which puts it in TC mode copied correctly, for instance.

Yeah, I bet Tron is firing. Pretty dumb we don't just have a non-firing version in the TCRM downloads? Or do we? I remember doing that manually for a long time, anyway. Same with DT.

Bot output doesn't go anywhere in RoboRunner. Actually not exactly sure how to catch it but it would be a nice feature. You could log stuff to files though.

Try this: IBattleListener.onBattleMessage(BattleMessageEvent event)

Makes sense, but I checked the .data directory for tron for the 4 instances of RoboRunner that RoboJogger generates and all of them have a properties file with "challenger" set. :(

Oh, hmm. That sucks. 61 is a pretty unthinkably low score vs Tron, looking at the TCRM results. I guess it would be useful if RoboRunner/RoboJogger had a switch for displaying battles to help debug this kind of thing.

Can you try running battles manually from one of those Robocode instances?

Found the cause of the problem but im not sure why. Running Robocode from inside Eclipse means my robot doesn't get any Skipped Turn events, but running robocode from the robocode.sh file means my robot does get skipped turn events. That means two things:

1) There is a difference between running robocode from inside and outside eclipse 2) My robot is running dead slow (but mainly versus tron?!)

The second problem is something that I need to deal with, but the first is interesting. My command line arguments for running Robocode in Eclipse are "-Xmx512M -Dsun.io.useCanonCaches=false -Ddebug=true". Would this cause Robocode to ignore skipped turn events?

Yes, I believe setting debug=true disables skip turns. (Also turning on debugging graphics.)

-Ddebug=true does disable skipped turns, I checked the engine source. It is made exactly for that, so you can pause and trace execution step-by-step.

Ahhhh cheers problem found. I guess I need to look at optimising my bot. Unfortunately I've found that I can greatly increase its score versus a lot of bots by doing a lot more work. Sigh :(

There needs to be a way for a third party (e.g. RoboJogger) to access battle results (on the fly results highly preferable). It looks like I could use ScoreLog to read results from the XML result file after it is created, but it would be preferable to have a way to add a listener that is notified each time a new battle result is available. For example, RoboRunner might allow third parties to add their own BattleResultHandler in addition to or in replacement of the one RoboRunner uses internally. Thoughts?

Yep, that makes sense. I think when you're listening to RoboRunner, you might want some higher level data too, like avg score and number of battles. So I think adding a new / similar listener to the RoboRunner class makes more sense than just letting you add more custom instances of the existing BattleResultHandler. The new interface method could take the raw scores and elapsed time, as now, plus whatever other summary data you want. At the end of RoboRunner's BattleResultHandler.processResults(), we call the higher level listener, if it's been set.

Does that sound about right?

Sounds good. Is this a change you would like to make or would you prefer I make the change and send it back to you for review?

I say just make the changes you need and go with it, and I can merge them into the main branch whenever. But I'm happy to look over stuff or even write up the interface if you need me to.

I still haven't gotten around to working on this. I suppose I would be happier if you added it such that it can be implemented in the manner you feel is best, but I will work on it once I run out of other tasks if you don't have time for it. I've actually gotten quite far into development relying only on what I can get from ScoreLog. Other than not having a nice way to track battles on the fly, the only thing ScoreLog doesn't provide that I wish it did is a way to retrieve or calculate the confidence values. I love the idea of computing confidence values, and am determined to include it somehow.

Sure, no problem, I'll take a look sometime today or tomorrow. It will be trivial to pass along the raw scores, some summary data, and the basic per-bot confidence interval stuff. Passing the groups and overall scores and confidence intervals might prompt some refactoring to do it cleanly, since right now they're kind of just calculated in-line before printing them, but it shouldn't be too tough.

I see in 0.9.5 notes: "Added a basic output parser for RoboRunner. Challenge run progress information in the main window is now updated on the fly..."

Does this mean you took care of this listener thing yourself since I never actually did it? Or are you still lacking some way to get updated with RoboRunner's results (and/or confidence estimates) on the fly?

Also, I'd like to merge back whatever changes you made to RoboRunner, at least to the GitHub repo. I guess I'll just wait and take a look at the source after 0.9.6.

The only information the output parser extracts is a count of battles completed. It is not able to determine the scores, nor does it parse out the confidence. The only thing it provides is on-the-fly information on how many battles have been run, in order to update the progress in running a challenge. It was a way to do a little more, but does not take the place of having a listener that actually provides score information.

The changes to RoboRunner so far have been very minimal. My focus has been almost exclusively on RoboJogger. The changes I have made to RoboRunner include only the following:

1. In BattleRunner, I made it so that getAllFutures(List<Future<String>>) could be interrupted in order to provide a way to stop RoboRunner. However, my particular implementation needs to be done in a better way than I did it. It sets a flag on InterruptedException that causes all remaining futures to be cancelled (it calls future.cancel(false) on all remaining futures). It needs to be improved such that it can safely call future.cancel(true) without potentially corrupting a score log, or have the futures be able to detect the stop request to shut down cleanly if running.
2. In ScoreLog, I added an ReentrantLock that is locked whenever loadScoreLog(...) or saveScoreLog(...) is called. Thus, only a single score log can be loaded or saved at a time. This was necessary as both RoboRunner and RoboJogger call the loadScoreLog(...) method and thus it needed some kind of synchronization. I know it would be safe to load and save different score logs at the same time, but I decided that there would not be enough concurrent loading and saving for it to be worth the added complexity.
3. In ScoreLog, I close the XMLEventReader and input stream after a ScoreLog is loaded. Not sure if this really matters or not in practice.

And that's it. Even though you may not have specifically designed it for being used by another program, RoboRunner is quite easy to interact with and it required very little modification to put my UI on top of it. Very nice.

I'm still losing chunks of score logs on rare occasions, and I have no idea why. I still need to add checks to ensure RoboRunner is cleanly shut down before exiting RoboJogger, but I've had data loss at a time when I was absolutely sure RoboRunner was finished running, so there is still a problem somewhere that needs to be solved. If I can track down that problem and take care of the other things noted above, I will be very close to having a non-beta-ish release.

I just made another change to RoboRunner that will be part of 0.9.7. Again, I kept it pretty minor, but that makes 4 minor things now, so I should probably put the source out there where you can get to it. The new change I made was to add isTerminated() methods to RoboRunner and BattleRunner. These new methods return whether or not the main thread pool and the callback pool are terminated (by checking isTerminated() on both thread pools and then feeding that info back up the hierarchy).

isTerminated() returns true on a thread pool when shutdown() has been called and all threads in the pool are done. This is important to RoboJogger in determining whether or not RoboRunner has finished shutting down before starting a new runner or allowing a user to exit the program.

Another alternative would be to add a shutdown method to RoboRunner that is blocking, but I didn't think that would work as well, so I didn't take that approach.

Updating the result windows on the fly doesn't seem like it would be particularly difficult. But I don't know the structure of your code at the moment, so I could be wrong.

Assuming a singular method that displays the windows given a result set.

a la "void showResultsWindow(resultSet)", could make it "ResultWindow showResultsWindow(resultSet)", keep it in a map "HashMap<ResultSet,ResultWindow>", and update the window based on updates in the set "ResultWindow window = map.get(resultSet); window.updateData(resultSet); window.markTableDirty();"

Mark table dirty would be this piece of thread safe code "RepaintManager.currentManager(table).markCompletelyDirty(table);"

Keeping the window in a map also allows you to just move that window to the front if someone tries to open the same result window twice. Though you make require a listener to determine if the window was closed, so you can remove it from the hashmap. Just hiding it could be problematic if someone keeps the application open for a long time and runs many different challenges.

It would be nice to have a season counter as well. That you can see without getting the wiki output.

Also I saw the logo in about, and my eyes bled. So here is a doodle.

http://i2.minus.com/iCUtdsJ56Qgpr.png

Also before I forget. You should have it do at least 3-4 battles per bot before 'smart' battles kick in. This is just in case you got two flukes in a row. Where as 3-4 in a row become considerably less likely. I only mention this because I just got two flukes in a row (a really high score at that).

It does sprinkle in random battles among the bots with lowest battles. I didn't want to do 3-4 for everyone up front because that largely nullifies the usefulness of smart battles if you're doing 500 bot test beds. But as it is, if you're running several thousand, everyone will get to 3 or 4 at least, I'm pretty sure.

Ah good to know.