Difference between revisions of "Talk:Compressed Serialization"
(moving discussion from Compressed Serialization page) |
RednaxelaBot (talk | contribs) m (Using <syntaxhighlight>.) |
||
| (One intermediate revision by one other user not shown) | |||
| Line 1: | Line 1: | ||
== From old wiki's Compressed Serialization page == | == From old wiki's Compressed Serialization page == | ||
| − | [[David Alves]] mentioned that [[Paul Evans]]' SandboxMini has two easy-to-use methods for this as well, <nowiki>PEWriteObject and PEReadObject</nowiki>. | + | [[User:David Alves|David Alves]] mentioned that [[User:Paul Evans|Paul Evans]]' SandboxMini has two easy-to-use methods for this as well, <nowiki>PEWriteObject and PEReadObject</nowiki>. |
---- | ---- | ||
| − | Is anybody else but me using serialization to save data on enemy bots? I have a big problem. Since [[Marshmallow]] takes quite a while to learn what works best against a given enemy I want to train the robot before I package it and upload it. The serialized files get's packagedas they should but something goes wrong when they are to be deserialized so it always starts afresh when the packaged robot meets an enemy (for the first time after being packaged that is). I'm suspecting this could have something to with what version of some Java API objects that get serialized is used on the machine packaging and the machine running the packaged bot. Someone has an idea how to solve this? Sorry if this is confusing... The caveat is that it seems that my robot can't take advantage of saved data once packaged and running on another computer. --[[PEZ]] | + | Is anybody else but me using serialization to save data on enemy bots? I have a big problem. Since [[Marshmallow]] takes quite a while to learn what works best against a given enemy I want to train the robot before I package it and upload it. The serialized files get's packagedas they should but something goes wrong when they are to be deserialized so it always starts afresh when the packaged robot meets an enemy (for the first time after being packaged that is). I'm suspecting this could have something to with what version of some Java API objects that get serialized is used on the machine packaging and the machine running the packaged bot. Someone has an idea how to solve this? Sorry if this is confusing... The caveat is that it seems that my robot can't take advantage of saved data once packaged and running on another computer. --[[User:PEZ|PEZ]] |
| − | What standard-java-type objects are you trying to serialize within your objects? Usually the Java documentation has a note somewhere if serialized objects under one implementation of a class will not be compatible with future releases. -- [[Kawigi]] | + | What standard-java-type objects are you trying to serialize within your objects? Usually the Java documentation has a note somewhere if serialized objects under one implementation of a class will not be compatible with future releases. -- [[User:Kawigi|Kawigi]] |
| − | The objects I'm serializing are simple. Containing two '''double'''s, two '''int'''s and one ArrayList containing even simpler objects consisting of only '''double'''s and '''int'''s. Could it be some versioning problem with my own objects? I don't think so since I would then expect to see the failure without the packaging/unpackaging. -- [[PEZ]] | + | The objects I'm serializing are simple. Containing two '''double'''s, two '''int'''s and one ArrayList containing even simpler objects consisting of only '''double'''s and '''int'''s. Could it be some versioning problem with my own objects? I don't think so since I would then expect to see the failure without the packaging/unpackaging. -- [[User:PEZ|PEZ]] |
| − | Hmmm... not sure what to say. The javadocs don't say anything about problems with serializing ArrayList objects... -- [[Kawigi]] | + | Hmmm... not sure what to say. The javadocs don't say anything about problems with serializing ArrayList objects... -- [[User:Kawigi|Kawigi]] |
| − | Well, it might be some aspect of Robocode that I have misunderstood. If "train" a development version robot and then package it and then run the packaged robot it seems unable to read the serialized objects. It doesn't matter that it is the same computer and JVM running. While using the development version there are no problems at all reading those objects. And the packaged robot has no problems reading its object files once it has reacreated them... -- [[PEZ]] | + | Well, it might be some aspect of Robocode that I have misunderstood. If "train" a development version robot and then package it and then run the packaged robot it seems unable to read the serialized objects. It doesn't matter that it is the same computer and JVM running. While using the development version there are no problems at all reading those objects. And the packaged robot has no problems reading its object files once it has reacreated them... -- [[User:PEZ|PEZ]] |
----- | ----- | ||
| − | Are there examples of bots storing whole movement array's? -- [[Loki]] | + | Are there examples of bots storing whole movement array's? -- [[User:Loki|Loki]] |
| − | What do you mean with movement arrays? Like pattern matcher data?. Between rounds all pattern matchers do this I would guess. Between matches some do it. [[Nibbler]] is one. -- [[PEZ]] | + | What do you mean with movement arrays? Like pattern matcher data?. Between rounds all pattern matchers do this I would guess. Between matches some do it. [[Nibbler]] is one. -- [[User:PEZ|PEZ]] |
yes, i mean pattern matcher data. Between rounds i 'store' the data by using static's. Now i am looking for a way to store the data at the end of a match and retreive it at the start of a new match. | yes, i mean pattern matcher data. Between rounds i 'store' the data by using static's. Now i am looking for a way to store the data at the end of a match and retreive it at the start of a new match. | ||
| − | I have been looking into the JavaDoc, but find Java's IO terribly complex with al those nested streams. The code on this page shows an example and works for a simple case, but after i started saving data from my targets i now get all kinds of "inconvertible types" errors. Isn't it possible to save int's and double's? -- [[Loki]] | + | I have been looking into the JavaDoc, but find Java's IO terribly complex with al those nested streams. The code on this page shows an example and works for a simple case, but after i started saving data from my targets i now get all kinds of "inconvertible types" errors. Isn't it possible to save int's and double's? -- [[User:Loki|Loki]] |
| − | [[PEZ]], is your problem with the deserialisation of data (mentioned at [[CompressedSerialization]]) solved? | + | [[User:PEZ|PEZ]], is your problem with the deserialisation of data (mentioned at [[CompressedSerialization]]) solved? |
| − | Sure you can just save primitive data types. Look at SittingDuck for one way of doing this. [[Nibbler]] uses SymbolicPatternMatching and saves the string. That's one more advantage of symbolic pattern matching. Yes, I still have problems with deserialization. I haven't looked in to it, but now it is starting to become a problem again. I haven't tried to use the serial version id thingy yet. If that doesn't work I'll go for externalization instead. Should be able to save some space that way also. -- [[PEZ]] | + | Sure you can just save primitive data types. Look at SittingDuck for one way of doing this. [[Nibbler]] uses SymbolicPatternMatching and saves the string. That's one more advantage of symbolic pattern matching. Yes, I still have problems with deserialization. I haven't looked in to it, but now it is starting to become a problem again. I haven't tried to use the serial version id thingy yet. If that doesn't work I'll go for externalization instead. Should be able to save some space that way also. -- [[User:PEZ|PEZ]] |
Found the solution for my "inconvertible types" errors: | Found the solution for my "inconvertible types" errors: | ||
I am using <nowiki>ObjectInput/OutputStream</nowiki> as in the example. When reading data from file you have to cast the object into the desired object. Example: | I am using <nowiki>ObjectInput/OutputStream</nowiki> as in the example. When reading data from file you have to cast the object into the desired object. Example: | ||
| − | < | + | <syntaxhighlight> |
StringBuffer pattern = (StringBuffer)in.readObject(); | StringBuffer pattern = (StringBuffer)in.readObject(); | ||
| − | </ | + | </syntaxhighlight> |
When reading primitive types you have to use the appropriate methods from the ObjectInputStream class. Example: | When reading primitive types you have to use the appropriate methods from the ObjectInputStream class. Example: | ||
| − | < | + | <syntaxhighlight> |
int historyCounter = in.readInt(); | int historyCounter = in.readInt(); | ||
| − | </ | + | </syntaxhighlight> |
| − | -- [[Loki]] | + | -- [[User:Loki|Loki]] |
| − | New question: I found in the Robocode version info that the quota for files is 200k (as for robocode version 1.06). How can you prevent running out of 'disk space' -- [[Loki]] | + | New question: I found in the Robocode version info that the quota for files is 200k (as for robocode version 1.06). How can you prevent running out of 'disk space' -- [[User:Loki|Loki]] |
| − | The only way must be to check if what you want to write fits in the space left and then you have to either save less or delete something from your directory. Check TheArtOfWar code, it handles this situation. -- [[PEZ]] | + | The only way must be to check if what you want to write fits in the space left and then you have to either save less or delete something from your directory. Check TheArtOfWar code, it handles this situation. -- [[User:PEZ|PEZ]] |
---- | ---- | ||
Did any one try this with jdk1.4.2 beta. I have some problems to save or load data. I tried Marshmallow as well it also fails. Any ideas? --[[SSO]] | Did any one try this with jdk1.4.2 beta. I have some problems to save or load data. I tried Marshmallow as well it also fails. Any ideas? --[[SSO]] | ||
| − | What problems do you encounter? I use jdk1.4.1 b.t.w. --[[Loki]] | + | What problems do you encounter? I use jdk1.4.1 b.t.w. --[[User:Loki|Loki]] |
| − | In what way does Marshmallow fail? -- [[PEZ]] | + | In what way does Marshmallow fail? -- [[User:PEZ|PEZ]] |
Mainly it's security problem. My bot has the same problem. Here is the exception | Mainly it's security problem. My bot has the same problem. Here is the exception | ||
| Line 81: | Line 81: | ||
--[[SSO]] | --[[SSO]] | ||
| − | And does SittingDuck have the same problem? -- [[PEZ]] | + | And does SittingDuck have the same problem? -- [[User:PEZ|PEZ]] |
NO! --[[SSO]] | NO! --[[SSO]] | ||
| − | Strange, I was almost sure it should... -- [[PEZ]] | + | Strange, I was almost sure it should... -- [[User:PEZ|PEZ]] |
| − | Do you use getDataFile() or whatever? -- [[Kawigi]] | + | Do you use getDataFile() or whatever? -- [[User:Kawigi|Kawigi]] |
I had the same problem. Solved it by | I had the same problem. Solved it by | ||
| Line 103: | Line 103: | ||
Wait, I'm confused; how come it lets you catch an AccessControlException and allows the bot to keep going? When I was testing out the effects of referencing GL classes on regular Robocode, even if I caught the AccessControlException, Robocode would still disable my robot and kill its thread. | Wait, I'm confused; how come it lets you catch an AccessControlException and allows the bot to keep going? When I was testing out the effects of referencing GL classes on regular Robocode, even if I caught the AccessControlException, Robocode would still disable my robot and kill its thread. | ||
| − | Also, does that zipping code write the .zip header with the file, or does it just write the zip data bare? Because if it is, you're saving a zip header for nothing. I mean it could only be a few bytes per file, but still... -- [[Vuen]] | + | Also, does that zipping code write the .zip header with the file, or does it just write the zip data bare? Because if it is, you're saving a zip header for nothing. I mean it could only be a few bytes per file, but still... -- [[User:Vuen|Vuen]] |
| − | Such is true. You can also be classy and make a zip file with a file in it for each enemy if you wanted. Or you can go SandboxMini's route (and presumably SandboxDT's) and use a GZip compression thingy. (I don't know if that changes the header stuff). Or you could be REALLY like [[Paul Evans]] and make your own complete compression scheme. -- [[Kawigi]] | + | Such is true. You can also be classy and make a zip file with a file in it for each enemy if you wanted. Or you can go SandboxMini's route (and presumably SandboxDT's) and use a GZip compression thingy. (I don't know if that changes the header stuff). Or you could be REALLY like [[User:Paul Evans|Paul Evans]] and make your own complete compression scheme. -- [[User:Kawigi|Kawigi]] |
I am suffering with the java.security.AccessControlException problem since upgrading to Java version 1.4.2. I have observed the following... | I am suffering with the java.security.AccessControlException problem since upgrading to Java version 1.4.2. I have observed the following... | ||
| Line 115: | Line 115: | ||
My present solution to work around the problem is to do all file reads in round 1, storing the data in statics - catching the exception and re-reading if necessary. At worst you will loose round 1 because of being disabled. For any other file I/O perform at then end of the round - retyring if the output is important. | My present solution to work around the problem is to do all file reads in round 1, storing the data in statics - catching the exception and re-reading if necessary. At worst you will loose round 1 because of being disabled. For any other file I/O perform at then end of the round - retyring if the output is important. | ||
| − | -- [[Paul Evans]] | + | -- [[User:Paul Evans|Paul Evans]] |
Latest revision as of 09:33, 1 July 2010
From old wiki's Compressed Serialization page
David Alves mentioned that Paul Evans' SandboxMini has two easy-to-use methods for this as well, PEWriteObject and PEReadObject.
Is anybody else but me using serialization to save data on enemy bots? I have a big problem. Since Marshmallow takes quite a while to learn what works best against a given enemy I want to train the robot before I package it and upload it. The serialized files get's packagedas they should but something goes wrong when they are to be deserialized so it always starts afresh when the packaged robot meets an enemy (for the first time after being packaged that is). I'm suspecting this could have something to with what version of some Java API objects that get serialized is used on the machine packaging and the machine running the packaged bot. Someone has an idea how to solve this? Sorry if this is confusing... The caveat is that it seems that my robot can't take advantage of saved data once packaged and running on another computer. --PEZ
What standard-java-type objects are you trying to serialize within your objects? Usually the Java documentation has a note somewhere if serialized objects under one implementation of a class will not be compatible with future releases. -- Kawigi
The objects I'm serializing are simple. Containing two doubles, two ints and one ArrayList containing even simpler objects consisting of only doubles and ints. Could it be some versioning problem with my own objects? I don't think so since I would then expect to see the failure without the packaging/unpackaging. -- PEZ
Hmmm... not sure what to say. The javadocs don't say anything about problems with serializing ArrayList objects... -- Kawigi
Well, it might be some aspect of Robocode that I have misunderstood. If "train" a development version robot and then package it and then run the packaged robot it seems unable to read the serialized objects. It doesn't matter that it is the same computer and JVM running. While using the development version there are no problems at all reading those objects. And the packaged robot has no problems reading its object files once it has reacreated them... -- PEZ
Are there examples of bots storing whole movement array's? -- Loki
What do you mean with movement arrays? Like pattern matcher data?. Between rounds all pattern matchers do this I would guess. Between matches some do it. Nibbler is one. -- PEZ
yes, i mean pattern matcher data. Between rounds i 'store' the data by using static's. Now i am looking for a way to store the data at the end of a match and retreive it at the start of a new match. I have been looking into the JavaDoc, but find Java's IO terribly complex with al those nested streams. The code on this page shows an example and works for a simple case, but after i started saving data from my targets i now get all kinds of "inconvertible types" errors. Isn't it possible to save int's and double's? -- Loki
PEZ, is your problem with the deserialisation of data (mentioned at CompressedSerialization) solved?
Sure you can just save primitive data types. Look at SittingDuck for one way of doing this. Nibbler uses SymbolicPatternMatching and saves the string. That's one more advantage of symbolic pattern matching. Yes, I still have problems with deserialization. I haven't looked in to it, but now it is starting to become a problem again. I haven't tried to use the serial version id thingy yet. If that doesn't work I'll go for externalization instead. Should be able to save some space that way also. -- PEZ
Found the solution for my "inconvertible types" errors: I am using ObjectInput/OutputStream as in the example. When reading data from file you have to cast the object into the desired object. Example:
StringBuffer pattern = (StringBuffer)in.readObject();When reading primitive types you have to use the appropriate methods from the ObjectInputStream class. Example:
int historyCounter = in.readInt();-- Loki
New question: I found in the Robocode version info that the quota for files is 200k (as for robocode version 1.06). How can you prevent running out of 'disk space' -- Loki
The only way must be to check if what you want to write fits in the space left and then you have to either save less or delete something from your directory. Check TheArtOfWar code, it handles this situation. -- PEZ
Did any one try this with jdk1.4.2 beta. I have some problems to save or load data. I tried Marshmallow as well it also fails. Any ideas? --SSO
What problems do you encounter? I use jdk1.4.1 b.t.w. --Loki
In what way does Marshmallow fail? -- PEZ
Mainly it's security problem. My bot has the same problem. Here is the exception
pez.Marshmallow 1.5.3: Exception: java.security.AccessControlException: Preventing pez.Marshmallow 1.5.3 from access: (java.io.FilePermission E:\java\robocode\robots\pez read): You may only read files in your own root package directory. java.security.AccessControlException: Preventing pez.Marshmallow 1.5.3 from access: (java.io.FilePermission E:\java\robocode\robots\pez read): You may only read files in your own root package directory.
at robocode.security.RobocodeSecurityManager.checkPermission(RobocodeSecurityManager.java:314) at java.lang.SecurityManager.checkRead(SecurityManager.java:863) at java.io.File.exists(File.java:678) at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:358) at java.io.File.getCanonicalPath(File.java:513) at java.io.File.getCanonicalFile(File.java:534) at robocode.peer.robot.RobotFileSystemManager.getReadableDirectory(RobotFileSystemManager.java:110) at robocode.security.RobocodeSecurityManager.checkPermission(RobocodeSecurityManager.java:299) at java.lang.SecurityManager.checkRead(SecurityManager.java:863) at java.io.File.exists(File.java:678) at java.io.Win32FileSystem.canonicalize(Win32FileSystem.java:358) at java.io.File.getCanonicalPath(File.java:513) at java.io.File.getCanonicalFile(File.java:534) at robocode.peer.robot.RobotFileSystemManager.getWritableDirectory(RobotFileSystemManager.java:125) at robocode.AdvancedRobot.getDataFile(AdvancedRobot.java:363) at pez.Marshmallow.writeObject(Unknown Source) at pez.Marshmallow.saveStatistics(Unknown Source) at pez.Marshmallow.onRobotDeath(Unknown Source) at robocode.peer.robot.EventManager.onRobotDeath(EventManager.java:598) at robocode.peer.robot.EventManager.processEvents(EventManager.java:742) at robocode.peer.RobotPeer.tick(RobotPeer.java:1024) at robocode.AdvancedRobot.execute(AdvancedRobot.java:186) at pez.Marshmallow.run(Unknown Source) at robocode.peer.RobotPeer.run(RobotPeer.java:616) at java.lang.Thread.run(Thread.java:534)
--SSO
And does SittingDuck have the same problem? -- PEZ
NO! --SSO
Strange, I was almost sure it should... -- PEZ
Do you use getDataFile() or whatever? -- Kawigi
I had the same problem. Solved it by
while (!dataIsLoaded) {
try {
tryToReadData;
dataIsLoaded = true;
} catch (AnyException e) {}
}
this looks like something that should never work, but it does. about every 10th try to read anthing, the exception occurs. randomly. so i just try to read the file ovr and over again until robocode gives up, and i get the file. same for saving. -HoD
Wait, I'm confused; how come it lets you catch an AccessControlException and allows the bot to keep going? When I was testing out the effects of referencing GL classes on regular Robocode, even if I caught the AccessControlException, Robocode would still disable my robot and kill its thread.
Also, does that zipping code write the .zip header with the file, or does it just write the zip data bare? Because if it is, you're saving a zip header for nothing. I mean it could only be a few bytes per file, but still... -- Vuen
Such is true. You can also be classy and make a zip file with a file in it for each enemy if you wanted. Or you can go SandboxMini's route (and presumably SandboxDT's) and use a GZip compression thingy. (I don't know if that changes the header stuff). Or you could be REALLY like Paul Evans and make your own complete compression scheme. -- Kawigi
I am suffering with the java.security.AccessControlException problem since upgrading to Java version 1.4.2. I have observed the following...
- It is possible to trap this exception, however the only advantage in trapping is that you can try to read/write again instead of crashing.
- If the exception occurs your energy is dropped to zero and you are disabled.
- The exception only occurs once per round (probably).
- It appears to occur more often when watching a battle.
My present solution to work around the problem is to do all file reads in round 1, storing the data in statics - catching the exception and re-reading if necessary. At worst you will loose round 1 because of being disabled. For any other file I/O perform at then end of the round - retyring if the output is important.
-- Paul Evans
