[Python] Splitting up source code

[Shaggy]

So up until this point, I have been working all from one main.py source file. Now that my code is starting to grow, I've begun trying to split it into seperate code files. However, I've come to realize that this can be a bit of a hastle depending on how things are set up. I followed the code for the python + libtcod tutorial to start off, so there are still some references and such that I'm not 100% aware of.

That being said, how do you suggest splitting up code in a larger project? Currently, my source files are as follows; src_AI, src_Classes, src_DGen, src_Globals, src_Main, and src_Objects. I think that keeps everything separate without having too many source files.

(Also, if anyone is interested in helping, I could explain further the issues that I'm having right now with getting everything working.)

[george]

How many lines of code are we talking about?

I think what is important to understand is that while you can look at source organization as a code organization issue, in many ways it is just as much a tooling issue. What you may want to consider is changing (I hesitate to say 'improving' as that is quite subjective ) your tooling to reflect your growing codebase.

For example, some editors let you have multiple views of a file and some do not. Some editors allow for an outline mode that lets you organize the source in a way completely different than the way the source is actually saved in files.

So before you start splitting up source files and bogging yourself down with imports and Object.object.object references up the wazoo, consider if changing your tooling a bit can improve your workflow to the point where you don't need to worry about it so much, for in the end this is a relatively trivial matter that is not so important to the production of your game.

[tuturto]

It's a good idea to split code into smaller pieces. They are more manageable, easier to understand and faster to navigate.

How to split depends somewhat on the project and people working on the project. I like very small files, so I end up with lots of them. You should group modules (files) into packages (directories) to keep related things together.

Your idea of splitting the code looks sound. src_Classes sounds like a module that might end up being quite big though and quite generic. What would it contain?

Another idea would be:

main.py
items.py
monsters.py
levels.py
generator.py
user_interface.py

For a more complex example, check: https://github.com/tuturto/pyherc/tree/master/src , but that might be an overkill. It all started from https://github.com/tuturto/pyherc/tree/b96339fbd36a26ff04dfc5c6457a5367b58235a5/src/pyHerc though.

I would drop src_ from namge of modules. So instead of "src_Classes", there would be just "classes".

You don't have to split them in a perfect way on a first try. Just break them into something that makes sense now and change that later if you feel that there is something that needs changing.

I know some Python and would gladly help. However, I have never used libtcod, so problems specific to that library are something I probably can't help with.

[TheCreator]

Doesn't matter how you would call your files, as long as they are small. Yes, there are tools that do outlining and other stuff, but nothing helps you better than keeping source files as small as possible. It is arguable whether you should, for instance, split your magic system so that each spell goes into a separate file, or rather keep them all together in one large file, but certainly having spells mixed up with keyboard handlers and file loaders and script interpreters and AI routines and everything else is not a good idea. Also keep in mind that you will not find a tool that would save you from the mess resulting from having too large files because tools, unlike humans, can't distinguish between AI routines and spells.

[Shaggy]

Yeah, right now my code isn't too long, about 4-5k lines, but I know I'm going to want to split it up eventually, so why not do it now? But I'm getting strange errors and I feel like I'm doing something wrong. I'm getting close though. It's running without crashing now, but it's not creating the map or objects list correctly. We'll see what happens.

It's a good idea to split code into smaller pieces. They are more manageable, easier to understand and faster to navigate.

Your idea of splitting the code looks sound. src_Classes sounds like a module that might end up being quite big though and quite generic. What would it contain?

I took that out and added it to the bottom of src_Globals. It was just the base classes for everything in the game; Object, Fighter, Item, Equipment and Skillset. So in all actuality it wasn't that long.

I think what is important to understand is that while you can look at source organization as a code organization issue, in many ways it is just as much a tooling issue. What you may want to consider is changing (I hesitate to say 'improving' as that is quite subjective ) your tooling to reflect your growing codebase.

For example, some editors let you have multiple views of a file and some do not. Some editors allow for an outline mode that lets you organize the source in a way completely different than the way the source is actually saved in files.

That'd be very nice. I use notepad ++, and I like it a lot. It allows me to collapse classes and routines, which is helpful. Although when I open a file it automatically 'opens' all of the collapsed sections, so it's only useful for a short period.

Although now that I say that, I bet there's other options under the hood I could play with. Because when it comes down to it, it's just a personal desire for organization. Whether that is achieved via splitting up the source code, or just finding a tool to make it easier to read/follow, is another point.

[TheCreator]

Yeah, right now my code isn't too long, about 4-5k lines, but I know I'm going to want to split it up eventually, so why not do it now?

Very good, it's wise to think about splitting before the code actually grows up. Personally I don't have any KB-wise or LOC-wise limits for source files, some of my modules are well beyond 100 KB and it's okay, while others have hardly 5 KB and I consider them too large . Some files are just begging for being cut to pieces, some are not. The decision whether to split or not should generally be based on maintainability gain, not on file size.

That'd be very nice. I use notepad ++, and I like it a lot. It allows me to collapse classes and routines, which is helpful. Although when I open a file it automatically 'opens' all of the collapsed sections, so it's only useful for a short period.

The same thing happens in Visual Studio, as well as in Eclipse, and I think it's more an ill design than a bug. Certainly it makes the outlining feature completely useless, at least for me. It may be worth a notice that Eclipse has a window titled "Outline" which does similar job, but much better; it allows you to easily navigate through classes / methods / variables / imported declarations that you have in a single file, so that file size does not really matter [that much] any more.

[Shaggy]

The decision whether to split or not should generally be based on maintainability gain, not on file size.

This.

The same thing happens in Visual Studio, as well as in Eclipse, and I think it's more an ill design than a bug. Certainly it makes the outlining feature completely useless, at least for me. It may be worth a notice that Eclipse has a window titled "Outline" which does similar job, but much better; it allows you to easily navigate through classes / methods / variables / imported declarations that you have in a single file, so that file size does not really matter [that much] any more.

Well I just actually read about two other nice features that N++ has. The first is to be able to have multiple views open of the same file. Any changes made will still be present in all views, as it's the same instance of the file. The other is to place 'bookmarks' at any line of your code. Then, using simple keystrokes, you can move to the previous or next bookmark.

[kraflab]

If the code is less than 5k lines I generally just leave it in one big file because it doesn't really bother me.  I don't see any mechanical difference between opening a file by name or clicking on a name within a file to go there.  Once it gets particularly high though it becomes cumbersome to locate what you want within the huge space you have created.  It's definitely a good idea to split things up if you expect to have lots and lots of code.

[Shaggy]

Yeah, I know my source is going to grow, and I already am bothered by trying to find certain parts of code without Find searching for it 

My main issue right now is the globals section apparently re-writing over certain values, like the player's (x,y) position.. Not sure why it's doing that, considering it checks the objects list first to see if there's already a player before it initializes one.. but now that I say that.. If it's writing over the player value, whos to say it isnt writing over the objects list and clearing it, which would mean there wouldn't be a player in the list.

Sorry, just thinking out loud a little. But that's the kind of issues I've been having. I've figured out how to get a workspace set up in N++ but I can't get the code to behave properly.

Also, I updated the github repo today if anyone is interested. There's a few known bugs, which are listed in readme.md. Take a look at how I split everything up, how its structured, or anything really. C&C would be awesome, just send me a PM.

https://github.com/MisterMoxxie/ItsStillInBeta

[tuturto]

I downloaded head of master branch and played around a bit. Looks like a good start to me. I'm watching your repository at GitHub and will try to remember to check back when you have made more progress.

I enjoyed reading through the design document and wouldn't mind visiting the dungeon of blah

I'm not really fond of using globals and try to avoid them as much as possible. But if you can keep track of what is changing what I guess that's just a personal preference.

[Shaggy]

I downloaded head of master branch and played around a bit. Looks like a good start to me. I'm watching your repository at GitHub and will try to remember to check back when you have made more progress.

I enjoyed reading through the design document and wouldn't mind visiting the dungeon of blah

Awesome, I'm glad to see someone taking some interest in the project! And yeah, a lot of the text in the design doc is placeholder stuff for ideas I haven't fully fleshed out quite yet. :p

I'm not really fond of using globals and try to avoid them as much as possible. But if you can keep track of what is changing what I guess that's just a personal preference.

Yeah, I've been having that problem a lot as I started splitting the source into multiple files.. I'm still very new to python, and when I started I was following a tutorial. So I've still got a lot to learn.

How would you suggest?

[tuturto]

I would first try to remove some of the globals and worry splitting code into multiple files only after that.

For example, in module Objects there's function to cast heal:

Code: [Select]

def cast_heal():
    global message
    #heal the player
    if player.fighter.hp == player.fighter.max_hp:
        message('You are already at full health.', libtcod.red)
        return 'cancelled'

       
    message('Your wounds start to feel better!', libtcod.light_violet)
    player.fighter.heal(HEAL_AMOUNT)

player has been declared as global and only player character can be healed by spell. If you pass it as a parameter like below, a monster might be able to use healing spell too.

Code: [Select]

def cast_heal(player):
    #heal the player
    if player.fighter.hp == player.fighter.max_hp:
        message('You are already at full health.', libtcod.red)
        return 'cancelled'
       
    message('Your wounds start to feel better!', libtcod.light_violet)
    player.fighter.heal(HEAL_AMOUNT)

This of course means that you have to modify use function in Item class and pass character using the item as a parameter:

Code: [Select]

    def use(self, player):
        #just call the "use_function" if it is defined
        if self.use_function is None:
            message('The ' + self.owner.name + ' cannot be used that way.')
        else:
            if self.use_function(player) != 'cancelled':
                inventory.remove(self.owner)  #destroy after use, unless it was cancelled for some reason

However, I couldn't find where item's use function is called. You would have to modify that place too, to supply the parameter telling the character who is using the item.

Objects.py is split nicely. It has functions that relate to only one thing, in this case magic. I would probably rename it to magic.py to better reflect this.

Classes.py however, is rather large and has classes and functions that are not closely related. Tile and Rect for example could go to a new module called map.py.