Best way to store the RL data (during runtime)?

[RogueMaster]

Hi @'s

I'm developing a Roguelike, nothing really special, and I came to a point where I don't know what method should I use to save the RL Data during runtime.

I explain: right now, i'm using various UDTs (User Defined Types) to save the player, map, monsters, items...

The problem comes with the "savegames". A really headache, you know.
I read the article in roguebasin about savegames, but didn't clear much.
The first problem is that i cannot save the data as plain text since it's too much data, and also UDT's into UDT's, and Arrays of UDTs. It can be a hell to save all as plain text.

But if I save the data as binary data, the  problem is that the UDT's into another UDTs are not saved, since they are not really part of the second UDT, but pointers to data.
I can save the inner UDTs first, then go saving outer UDTs, but that can bring another problem: knowing where stop and where start reading data for arrays of UDTs!!!.

Some simple example to clear this out:
__________________________________________________________
Type ThingType
....thingName as String
....thingValue as UInteger
End Type

Type PlayerType
....health as UInteger
....mana as UInteger
....inventory(10) as ThingType
End Type

Dim as PlayerType player
__________________________________________________________________

Then, what method should I use to save the "player" variable, in this example?
If I simply make a "Put #1, ,player", only the UIntegers data is stored correctly, but the data refered by inventory(10) is not the real data, but pointer values and whatever...

So, any tip about savegames?

I'm using FreeBasic version 0.20BETA (i know there is now 0.21, i must retrieve it).

Thanks in advance.

[corremn]

Hmm, Messy. Going to need a lot more implementation detail. I will assume stuff anyway. I think UDTs are called structures in most languages.

I assume that ThingTypes are stored in a list somewhere. This list can be saved to file in order. For PlayerType you will have to save each element separately. I.e save health, then mana and then you will have to save a reference for each inventory item. The reference is a number that represents the order that the ThingType is in the original list. ( or you could also add a reference number to ThingType structure for convenience)

When loading save files you read the ThingTypes first, then the player health and mana. Then for each inventory item, read the reference number, find the corresponding THingType from the previously loaded list and assign a pointer to the inventory item in the player structure.

Anyway for each variable list that you have to read, you have to tell you program the number it will read. Therefore before you save a variable list you have to write the number of variables that you will read.

OR change language that has better serialization

P.s You can save in plain text, who cares how big your file is!  Reading large files should be quick for a modern computer, assuming you have an optimized release build.



Good luck.

[linux_junkie]

I'd strongly recommend using Lua for savefiles.  You can store the data in script format, and to reload the data, simply run the script file.  I did that in my project, and I've yet to find an easier, or better system.  And if you compile the Lua file, people will have a harder time editing the savefiles.

[Krice]

The first problem is that i cannot save the data as plain text since it's too much data

The first thing you need to figure out is what to save. In theory only dynamic (changing) values need to be saved and that has a lot to do with how you organize the data in your objects.

I think it really helps when you simplify the thinking of save game as collection of data belonging to objects. You don't save objects, you save the raw data. Then in load game objects are re-created and filled with saved data.

Plain text can sometimes reduce the size of save game, since small values can take two or three bytes instead of usual four (size of int). Also, thinking binary vs. text is outdated. I made the save game file of Teemu a hybrid one, saving number values as text and arrays as (packed) char arrays, including a size tag for each section. You can look at the source code of Teemu, especially tarball.cpp, to see the incredible cleverness of it. It's C++, but the idea can be used in most languages I think.

http://koti.mbnet.fi/paulkp/teemu/teemu.htm

[RogueMaster]

I'd strongly recommend using Lua for savefiles.  You can store the data in script format, and to reload the data, simply run the script file.  I did that in my project, and I've yet to find an easier, or better system.  And if you compile the Lua file, people will have a harder time editing the savefiles.

But I don't want (don't have the time) to learn a new language syntax. I heard LUA is pretty usefull, but for now, i will leave it. Maybe in future I will mess with it.

The first problem is that i cannot save the data as plain text since it's too much data

Ok, here's a explanation problem.

I know that even write 50MBs of data of a savefile could be only a bit of time.
But I dont want to mean that. I want to say that the player and map UDTs/Structures contains way lot of dirfferent variables, including other UDTs/Structs, so making a procedure to "run" through all structures, and through the the structures into structures can be excesively long and tedious.

But well... if there is no other solution... 

@ Corremn: also I would prefer to save as binary, mostly to avoid a lot of code lines, I will try your "save as text" method. Since the inventory into the player UDT/Struct is always the save maximun size, this could not be so annoyng as i though ;-)


@ Krice: Yes, that's the idea, of course: save only the data that could change during game. I never said to save all monsters data, map symbols or whatever. Anyway, saving the UDTs/Structs as they are (this is, "Put #1, ,Player) is faster and nicer that use nested for...next, do...loop, etc
Also, I will check your piece of code out. Although C++, I think i can understand the idea.


Thanks guys for your tips.

[Bear]

I have some pretty strict rules about runtime data, which are there to facilitate save & restore.
The most important one is that there is never more than one live pointer to any heap-allocated thing. If you seem to need more, then you need to make a collection of objects of that type, give each one a unique ID, and then implement all operations on that type using the collection and the unique ID.  This assures that they never get "leaked" when we drop too many pointers, and that no dangling pointers to them can ever be hanging around. 

Unique ID's can be dropped resulting in stuff that stays in the collection, but we can always get to it by iterating over the collection (which means we can detect and drop it) and Unique ID's can be hanging around after an item is deleted, but then the collection just returns an error code when you try to do something to it. So you can detect and handle the error locally instead of crashing. At first blush, you'd think that garbage collection should handle this kind of problem for you, but if you're doing save/restore, it most emphatically does not.

Saving the game then becomes mainly a matter of saving all of the managed collections.  A big list of actors (items and monsters and the player), a big list of mapdeltas or "digs", a big list of message-history prototypes, a big list of input-history, a big list of pending actions ... plus a few odds & ends like the interface settings, and save is done. In each case, just emit an XML tag to start the list, emit the list in XML format, and emit an XML tag to end the list.

In the middle of saving & restoring the elements in the managed collections, there is one other kind of dynamic data that I save.  It's a little hash table, used to store attributes.  It's implemented as type "deck", where each entry or "card" has a "suit" or attribute identifier and then a few machine words of information whose meaning depends on the suit.  But decks of attributes appear in every type that needs dynamic attributes. Actors, events, in maplevels, etc.  I didn't need to write separate save/restore code for all these things' dynamic attributes, I just wrote save/restore code for decks, once, and call it when saving or restoring each kind of thing.

So, anyway, save/restore is one of the main things that drove my runtime representation choices. 

Hope that helps,

Bear

[RogueMaster]

I have some pretty strict rules about runtime data, which are there to facilitate save & restore.
The most important one is that there is never more than one live pointer to any heap-allocated thing. If you seem to need more, then you need to make a collection of objects of that type, give each one a unique ID, and then implement all operations on that type using the collection and the unique ID.  This assures that they never get "leaked" when we drop too many pointers, and that no dangling pointers to them can ever be hanging around. 

[...]

Bear

Good tips. Will have into account.

But finally I managed to save everything as plain text, after around 350 - 400 lines of code, mostly of them just For...Next