Serialization

[Anvilfolk]

In an attempt to start some development talks here so that more people move to the forums, I have a few questions about serialization.

I don't have a terrible lot of experience in programming (only a 2nd year university CS student), and everyone on usenet has been talking about how important it is to set up a serialization system up front so that you are able to save and don't have to change huge amounts of code later on.

The main reason for this seems to be related to pointers. I've heard someone say that he assigned unique integers to every instanciated object (I believe), and instead of pointers, he kept those integers, which were looked up in (possibly) an array. This would eliminate problems with referencing. I'm guessing he's doing this so that there's a single way to refer to an object throughout the project.

But doesn't that actually already happen with memory pointers? I mean, an object can only be in one memory place at a time AFAIK. Same object pointers will have the same value throughout the project, right? In this case, wouldn't it be possible to code a "Serialization Engine", with serialize functions whose arguments are each possible class/structure ?

While it was serializing, it could keep an up-to-date table with pointers so it knows what has already been serialized and what hasn't. When it de-serializes, what was a pointer before becomes a unique ID anyway.

How wrong am I?

Also, if anyone has any links on serialization, could you please post them? I don't want to know about Java serialization though. That's all I can seem to find right now (not much time anyway). I think I'll cover this stuff in the next semester... but I need to know the basics so I don't make bad decisions right at the start.

Thanks!

[copx]

The problem with (C style) pointers starts when you have to write your pointer based structure to disk. You cannot save/restore pointer values directly, because memory addresses are not static i.e. just because your object was at address XYZ at the moment when you decided to "save and quit", does not mean that it will be at the same address when you start the game again and attempt to load the save file. I had this problem with Warp Rogue. The solution I used was to translate pointer values to list index numbers. This translation happens during saving/loading. The approach you read about makes this step unnecessary, because you already have  (static) index numbers. So you can write those to disk directly.

[Anvilfolk]

Hi! Thank you for replying! I've given WarpRogue a few tries this past week or so, and I find it has a huge amount of potential, so I hope you keep going, and good luck!

About serialization:

I understand that the same objects won't have the same memory address. However, it should still be unique throughout a single execution. Also, there won't be two pointers to diferent objects having the same value.

I'm guessing that what you do when you serialize is just write the list/array down, one by one, in the savefile, is that correct? I see how that works now...

However, theoretically... instead of doing it one-by-one, list/array style, couldn't you have a serialization engine, as I said, with functions for each class/struct type and call SerializationEngine::Serialize(MainGameClass/struct) on your main instance? What I'm thinking it could do is embed/serialize the object based on the first memory pointer occurence. It's late, so I'm not sure how clear I'm being - sorry about that.

Basically, I'm trying to say that when a pointer which the (de)serialization engine does not recognize is first found, you serialize or deserialize it immediately, embedding it within the holding class's serialization output. In deserialization, you'd probably create a new instance and "fill it up". You'd also end up knowing what the new pointer value is, because you'd just allocated it.

This way, the previous execution's pointers become a unique ID with which to identify which object is where in the new execution. You can easilly set that up in a table somewhere - much like you're doing in-game, except there's less overhead from having that general resource list/index.



Oh, poo. I think this is exactly what you just said. I'd appreciate it if you let me know if I got it half-straight though. Also, you could probably get away with no embedding by "recursively" serializing the "containees" and serializing the containers last - as long as you don't have double-references (this points to that, but that points to this). Might be a little tricker then.

Thanks in advance! And again, good luck with WarpRogue, I look forward to playing the next version!

[stu]

Ive got an interger memeory handle system in my app that your talking about. serialisation + deserialisation is a piece of cake with it. (to serialise the entire game to disk is 21 lines of C code)

pointers make things easy but serialisation really hard. handles give things an extra step but make serialisation _really_ easy.

if you dont start serialising your code right from the start and doing loading/saving at the get go, it becomes VERY hard to add it later on.

[Slash]

For Java, I have stick with its default serialization engine, which makes things almost ridiculously easy, to the cost of not being too efficient.

Serializing a game to disk is like 3 or 4 lines of code here.

[Anvilfolk]

Hmm, I'm currently using the STL. That makes things slightly harder. I guess if you've got "simple" structs it might be easier. But I won't forgo using classes, or the STL (for now).

I'm still amazed how 21 lines of code do all the work you need. I guess you just iterate through the main array.

[copx]

As pointed out by Slash's comment: serialization is highly language specific. Java and Python have built-in serialization frameworks for example, C does not. I barely know anything about C++, but I guess serializing objects is not the same as serializing pointer based structs. IIRC there are serialization frameworks for C++, but I am not sure.

[Krice]

if you dont start serialising your code right from the start and doing loading/saving at the get go, it becomes VERY hard to add it later on.

"They" always say that. I haven't done any kind of load/save in Kaduria since I changed to classes. I was thinking to do it last, when all object stuctures are ready.

[Anvilfolk]

Yeah, I've thought about it as well. Why would it get so very hard? I'm guessing it would be very time-consuming, yes, but not hard. It might be more confusing to have to update your serialization thingie every time you change something along the course of development.

Java's system is quite nice - every class is responsible for knowing how to serialize itself. Couldn't get much simpler than that. All you have to do is define some "protocol" to write everything down to the savefile, maybe one of those graph two-pass algorithms, or something like that.