RoboJogger

RoboJogger is a user interface for RoboRunner. RoboJogger together with RoboRunner provide functionality similar to that of RoboResearch -- the ability to run robots in challenges for the purpose of testing, diagnostics, and comparison -- but with a few distinct differences. The primary advantage of Robojogger/RoboRunner is that RoboRunner uses the Robocode API to run battles rather than running Robocode through a runtime execution for each battle, reducing the amount of time it takes to run a challenge.

RoboJogger is currently being developed by Skotty.

Downloads

Platform	Version	Download
non-Mac	0.9.5	robojogger.zip
Mac	0.9.5	robojogger-mac.zip
non-Mac	0.9.4	robojogger_0.9.4.zip
Mac	0.9.4	robojogger-mac_0.9.4.zip

Versions prior to 1.0 may still have issues but are available for review to anyone who is interested. If you do try it, feedback is welcomed. Should interest in this exist and grow, I'll look into hosting it from someplace more official; for now, it's just a ZIP of jars and files hosted off my personal website.

Version Compatibility

RoboJogger 0.9.x

• RoboRunner -- Packaged with a custom version of RoboRunner 1.2.3 that is currently dubbed as RoboRunner 1.2.3.1.
• Robocode -- Known to work with Robocode 1.7.3.x. Not yet tested on other versions, but likely works with many versions up to current.
• Java -- Requires version 1.6+ for most platforms. On Mac, with the current build, RoboJogger may not work with any Java version other than a recent copy of 1.6. With 1.7, my understanding is that Apple turned Java over to Oracle, and I am unsure whether or not the Apple Java Extensions used in RoboJogger will function properly, or if they will need to be removed for 1.7+.

Development Progress

The RoboJogger UI is being built using Skotty's existing Swing codebase as a starting point. This means some of the background code may not be entirely clean or relevant, but it will speed development considerably.

At present, there is no licensing restrictions on any of the code, as it is all written entirely by Skotty and Skotty generally doesn't bother with licenses for his own code. However, some kind of open source license may be applied to RoboJogger before it is released, provided there is any desire for such.

An initial trial version is complete -- version 0.9.x -- but it's features are reduced until an updated RoboRunner is released that provides a way to get on-the-fly battle results. Until then, version 0.9.x uses a very slightly modified version of RoboRunner 1.2.3.

The app is initially set up for using Robocode 1.7.3.0. If you switch Robocode versions when running Setup, you will have to manually copy the correct robocode.jar file into the robocode_jars folder that RoboJogger uses (you will be prompted about this in the program when you run Setup). Part of the issue is that the setup function is part of the application rather than a separate script as in RoboRunner, and I therefore cannot specify a classpath via the command line like RoboRunner itself does. Instead, RoboJogger uses robocode jars from a folder named robocode_jars. When setting up with a new version, most of the jars are automatically copied into this folder, but the robocode.jar file cannot be. When RoboJogger is running, robocode.jar is in use and cannot be automatically replaced. Thus the required manual copy.

One potential major problem right now has to do with reading and writing the score log files. Although some score log synchronization was added in RoboJogger version 0.9.4, there is still a chance of score log corruption under certain circumstances (like on exit).

As of 0.9.4, there are still no on-the-fly results. To see the latest results, you have to close any open results summary windows and reopen them, such that they pull the latest data from the RoboRunner score log files. Likewise, the status information for RoboRunner at the bottom of the main RoboJogger window currently does nothing. Also, RoboJogger calculates confidence values itself. Confidence values for individual robots will match what RoboRunner calculates, but confidence values for challenge groups and overall total will not be the same. The method RoboJogger uses is not the same as what RoboRunner uses. This will likely change in a later version. A more detailed look at computing confidence is shown in a later section.

A quick explanation of what some of the jars in RoboJogger are:

• robojogger.jar -- the main RoboJogger classes
• core.jar -- generic set of Swing classes RoboJogger relies on; this could be trimmed down to only what is needed when 1.0 comes out
• zenput.jar -- input validation classes that RoboJogger uses
• applestub.jar -- stub classes that match the signature of the Apple Java Extensions classes used on Macs; this jar is included on the non-Mac version to prevent class not found exceptions.
• roborunner-1.2.3.1.jar -- slightly modified version of RoboRunner 1.2.3 used by RoboJogger
• swing-worker-1.2.jar -- a version of the SwingWorker class that is a backport of the Java 6 SwingWorker that works with Java 5. I believe this version is also not susceptible to certain SwingWorker locking bugs in Java 6 and Java 7 [1], which is why I used it, but I need to verify this. There is also a workaround for the locking problem which I might could use instead, and I may switch to one of these workarounds.
• guava-12.0.1.jar -- used by RoboRunner 1.2.3
• log4j-1.2.11.jar -- logging engine used by RoboJogger

Calculating Confidence

This section needs to be reviewed for correctness

Confidence interval calculations are based on the assumption that data points will match a certain type of distribution. Whether it is the best match or not, both RoboRunner and RoboJogger currently assume scores against a robot will follow a normal distribution. Furthermore, both RoboRunner and RoboJogger use a 95% confidence interval, meaning with a large number of repeated samples, 95% of the calculated confidence intervals would include the true value of the parameter.

For battles against a single robot, the confidence interval calculation is reasonably simple, and both RoboRunner and RoboJogger calculate this value in the same way: X +/- 1.96 * d / sqrt(n), where X is the average of all scores, d is the standard deviation, and n is the number of battles.

However, how to calculate the confidence for a group or for the total is a more complicated question. RoboJogger and RoboRunner take differing approaches for this.

For RoboJogger, group and total confidence intervals are determined by using a single data point for each season. For example, if you had a completed challenge with 10 seasons, an average for each season would be computed, giving 10 averages that then serve as data points for the computed confidence interval.

This becomes further complicated when using smart battles, as then there will be a differing number of battles for each robot in the challenge. In this case, RoboJogger pads the number of battles for robots with fewer battles by creating fake random scores for those robots where the random scores are obtained by randomly choosing a score that falls within the confidence interval for that specific robot.

Next progress update will be on or around 1/31/2012.

RoboJogger Version Notes

Version 0.9.6 (In Development)

Definite changes:

• Any bugs shown on the 0.9.5 Bugs thread will be fixed.
• Subtotals and overall total on a season by season basis is now shown on the details window that displays scores for all seasons. I'm not sure if anyone else will find this useful, but I do. (I have a couple of spreadsheets that provide certain statistics that rely on multiple data points per robot; in that way it is similar to how RoboJogger calculates significance, using a data point for each season.)

Planned changes, in priority order:

1. Allowing multi-select on the challenge runs table to make moving and deleting groups of challenge runs less tedious.
2. Adding a more complex renderer on the result tables to more clearly distinguish subtotal and total columns.
3. Adding two new columns to the main challenge runs table -- battles run and total battles -- and possibly keeping but hiding the percent complete column (note that you can turn columns on and off already by launching a pop-up menu while on top of a column header).
4. Storing battle completion information in the RoboJogger settings file along with the challenge runs. This will avoid the overhead of having to read through score logs when starting to calculate completeness on the challenge runs.
5. Adding some sort of setting to control whether the column heading for groups is shown in the traditional and bland way (sub1, sub2, etc) or by using the actual group titles.

Things I am not currently planning on addressing in this release:

1. Improving melee challenge support. I haven't even tested this. And even if it works fine, I know I need to do some work to show more intelligent column header names.
2. I am not planning on tracking down and fixing the somewhat rare occasionally data loss in the score logs unless I just happen across it. I will leave that for 0.9.7 or later.

These plans are based on where I see the most benefit coming from in changes for this version. Please provide feedback if you would like to request a higher or lower priority on anything, or some other potential feature not currently listed.

Version 0.9.5

• Added a basic output parser for RoboRunner. Challenge run progress information in the main window is now updated on the fly; however, result windows must still be closed and reopened to pick up updated data.
• When challenge run completion information is updated for a challenger, challenge completion information is now updated for all challenge runs with the same challenger.
• Fixed nasty bug where clicking the close button instead of either the OK or Cancel button on the New Challenge window could cause an exception that would corrupt the list of Challenge Runs.
• Added ability to run with the "-clean" parameter that will attempt to remove any invalid challenge runs from the challenge runs list. Use this if a prior version corrupted your data and RoboJogger won't start properly.

Version 0.9.4

• Challenge table challenge completion information is now updated when RoboRunner is manually stopped.
• Challenge table challenge completion information is now updated when the number of seasons for a challenge run is changed.
• Added reentrant lock synchronization to the RoboRunner ScoreLog load and save methods to help prevent score log corruption.
• Added some additional exception handling so that RoboJogger can still run when there are corrupt score logs.

Version 0.9.3.2

• Fixed bug where having score results for challenges with different numbers of rounds would result in unmodifiable exception.

Version 0.9.3.1

• Fixed typo on enum where "PERCENT_SCORE" was accidentally typed as "PRECENT_SCORE". This resulted in being unable to create new PERCENT_SCORE challenges unless the challenge name was typed to match the misspelling.
• Delete challenge confirmation prompt now shows the standard challenge run title.

Version 0.9.3

• Fixed bug where battle scores were not filtered to ensure only battle scores with the same number of rounds as the challenge were used.
• Fixed bug where challenge run completion percentage did not change to 100% after challenge run was completed.
• Score type values AVERAGE_BULLET_DAMAGE and AVERAGE_ENERGY_CONSERVED are now recognized.
• Scores for all scoring types should now be correct (hopefully).
• Added option to create Wiki formatted text to the result summary window popup menu.
• Added preference for Author name used with Wiki formatted text.
• Reduced distribution size further by eliminating duplicate jars.

Version 0.9.2

• Added ability to show custom Descriptions for Challenge Runs.
• Added ability to edit Challenge Runs (though Robot and Challenge cannot be changed)
• Columns in the Challenge Run table on the main window can now be toggled on and off using a popup menu on the table header.
• Column sizes, order, and visibility in the Challenge Run table is now saved on exit and restored on startup.
• Fixed bug where Robot would fail to load if the Robot jar file contained multiple .properties files.

Known/Possible Issues

• There is still the possibility of one or more threads misbehaving. I will be looking into this closely in the next couple of weeks.

Version 0.9.1

• Added new window for setting preferred Look and Feel. If no preference is set, the system Look and Feel is used.
• Fixed bug where some functions were not properly re-enabled after RoboRunner finished processing all challenges.
• Fixed bug where RoboRunner threads would remain active and continue to consume CPU after stopping RoboRunner.
• Only prompt user about replacing robocode.jar on setup if robocode.jar changes (rather than prompting for every version change).
• Reduced the packaged copy of Robocode to the bare minimum to reduce distribution size.

Version 0.9

• Some features are not fully functional until a later update to RoboRunner is available. This includes:
  • On the fly battle results not yet available. Also, the RoboRunner progress and status area on the main window are not functional.
  • Completion percentages for challenge runs currently only update on program launch or when RoboRunner fully completes a challenge run.
  • Confidence values for groups and overall total will not match between RoboJogger and RoboRunner.
  • It may be possible for a score log to become corrupted under certain scenarios. While unlikley, if you wish to avoid this possibility, do not open challenge results windows while RoboRunner is running, and do not stop RoboRunner in the middle of a challenge.
  • Yes, I know the About menu icon is ridiculous. Sometimes I like ridiculous things. But I will replace it with something nicer for the finished 1.0 release.
• All errors and a few information messages are logged to the file robojogger.log.
• On first run (or any time the robocodes folder is empty), the RoboRunner setup window will appear automatically on launch. However, you can run setup anytime by lauching it through the Tools menu.
• Robocode version 1.7.3.0 is currently packaged with RoboJogger, and that version serves as the default when you run RoboRunner setup.
• If you change the version of Robocode when running RoboRunner Setup, you will have to manually copy the robocode.jar file from the new version of Robocode into the robocode_jars folder. You will be prompted about this by the program.
• All robots must be in the bots directory. When adding a new challenge, robots listed as part of the challenge must be in the bots directory, but the robot selected to participate in the challenge will be automatically copied into the bots directory if it is not already there.
• Development robots are not currently supported.
• Popup menus exist for the challenge run table on the main window and for the challenge result windows that show the score summary. This is particularly important to know for the challenge result windows, as at the moment the popup menu is the only way to access the challenge result detail window that shows results for every season.
• When RoboRunner is running, incomplete challenges will be processed from the top of the challenge run table to the bottom, until RoboRunner is stopped or no incomplete challenges remain.
• Currently distributed as a simple ZIP archive, but later can provide installer for Windows and .app for Mac if desired.

Screenshots

Main RoboJogger window (version 0.9.5). When RoboRunner is started, it will begin running any incomplete challenges in the table of challenges one at a time, starting from the top of the table, until all challenges have been completed.

(side note: the total seasons in this screenshot for each challenge is quite low because at the time I was running challenges that had over 250 robots each, and my older PC can only run about 7500 battles per day; the challenges in this screenshot represent about 2 solid days of execution on my PC; there is an advantage to this approach in that you can see the results of a lot of different challenges more quickly and you can always go back and edit the challenge runs to add more seasons)

---

Dialog for running RoboRunner setup. Currently accessed via menu: Tools -> Run Setup.

---

Dialog for adding a new challenge (version 0.9.1). Currently accessed via button on main window, menu item in popup menu over table, or in the main menu under: File -> New Challenge.

---

Window showing summary of battle results. At present, it uses the top row to show scores, the second row to show confidence in the scores, and I was also thinking of adding a third row showing battles run per opponent, which would be of interest on an opponent by opponent basis due to how RoboRunner smart battles works. Center data area is separate and scrollable to better handle huge challenges (suggested by Voidious).