Automated testing

[NON]

I've found myself creating bugs in my game lately, caused by dumb little oversights in the code.
For example:

Code: [Select]

if(OTHER_IS_AWARE == true) isBackStab = true;when It should clearly be

Code: [Select]

if(OTHER_IS_AWARE == false) isBackStab = true;
It's often trivial to find such misstakes, but it sucks when I discover it after a release.

I wonder how others deal with this. How do you prevent it? Do you have automated testing?

I did some reading on unit testing, but it seems like overkill for a one-man roguelike project, and I don't really have the time to implement it.

One idea I have is to have the game print lots of info to a file by using cout. Then I have a separate program that reads the stdout.txt file. It can check for example how many points the player put in various skills, and see if the resulting damage et c match what the skills are supposed to do.

[Darren Grey]

Test, test, test...  Even after minor changes I go into the game and check it's working as I want.  Building in a debug menu helps a lot (so you can check what variables are set to during run-time testing).  Of course you get the odd unexpected thing happening, especially in terms of game balance (which is harder to test), but there's nothing wrong with doing a quick bugfix release - indeed, most people appreciate fast releases.

[NON]

but there's nothing wrong with doing a quick bugfix release - indeed, most people appreciate fast releases.

Good point.

[Kyzrati]

I test every little change I make. And if I find myself making too many additions before I get to something testable/compilable, I start writing a list of all the specific playtests I'll want to do when everything will compile.

I'll also output debugging info to the screen for any features that need more intensive testing over a range of conditions (and leave all previous debugging output code as optionally compilable in case I need it again in the future when something related changes).

Other than closely testing everything yourself, you could also consider giving the more avid players pre-release versions and get feedback / find errors before the public release, but that probably only matters once a game is very popular.

[corremn]

Professionally I use unit tests. They are wonderful, especially when someone else stuffs up your code   For a roguelike, well you need good design....

In the roguelike world I find my self adding a feature, testing it and then thinking it works, unfortunately it often breaks something else.  So I then just play the game, which gets boringplaying the same thing over and over and then I lose interest.

But really the only real testing I do is playtesting. 

[purestrain]

Well yeah; unit tests may be overkill but i write them just for exercise. For example a test which checks that a trigger is activated after the player left the tile and reenters:

Code: [Select]

@Test
public void shouldTriggerIfPlayerStartsThereButMovesBackAndForth() {
Game testGame = Game.newGame(database, new GameBuilderTestImplementation(false, 0));

Entity player = testGame.getPlayer();
CustomController controller = new CustomController();
controller.addAction(new IdleAction());
controller.addAction(new MoveAction(-1, 0));
controller.addAction(new MoveAction(1, 0));

player.setController(controller);

LocationTrigger toAdd = new LocationTrigger(testGame);
toAdd.setLocation(new Point(2, 1));
toAdd.setActivateCallback(new ActivateCallbackImplementation());
testGame.getCurrentZone().addTrigger(toAdd);

testGame.getLoop().rebuild();
while (testGame.step())
;

Assert.assertEquals(true, wasTriggered);
}

They are somewhat essential for persistence because i simply have a unitttest which checks that gamestates are equal after saving/loading. And running a predefined scenario in less then 10ms is worth it instead of manually proving that everything works.

Code: [Select]

@Test
public void shouldBeTheSameGame() throws SQLException {
database.getDatabase().open();
// given
Game toStore = Game.newGame(database, new GameBuilderTestImplementation());

BaseEntity player = toStore.getPlayer();

// when
Game.saveGame(toStore, database);
Game reloaded = Game.loadGame(database, null);
database.getDatabase().close();

// then
Assert.assertEquals(player.getId(), reloaded.getPlayer().getId());
Assert.assertEquals(player, reloaded.getPlayer());
reloaded.testCompareTo(toStore);

}

A neat way to make sure that compatibility and upgrade path for save games work.

[Krice]

I've found myself creating bugs in my game lately, caused by dumb little oversights in the code.
For example:

Code: [Select]

if(OTHER_IS_AWARE == true) isBackStab = true;when It should clearly be

Code: [Select]

if(OTHER_IS_AWARE == false) isBackStab = true;

With bools you can also create logical pairs using verbose code. Something like this:

isBackStab=Are_Others_Looking_Away();

That way you not only make the code shorter, but avoid mismatch in true and false.

I think testing is a waste of time, at least all that source code level stuff. If you need heavy testing to avoid bugs then it's time to look at your programming skills... When the game is ready just play it like you would any other game. Usually big problems can be detected then, but if you miss something, someone will notice it.

[Kyzrati]

I think testing is a waste of time, at least all that source code level stuff.

It's at least important to test things that only someone with knowledge of the inner workings of the source code is likely to understand.

Like corremn says, changing one thing might break something else, and only the dev knows exactly what systems are connected to each other, and should be able to come up with a list of things that are interrelated. That's the stuff that really needs testing: unexpected interactions between multiple systems. I try to think of what parts of a given new system might bork in an existing one, and thus lead to behavior that *wasn't* explicitly coded. Complex games are especially prone to this unwanted type of emergence

Good underlying design and proper encapsulation can code away the range of possible pitfalls, but no matter how good you are you'll never be 100% safe once code starts reaching tens/hundreds of thousands of lines...

[corremn]

Good underlying design and proper encapsulation can code away the range of possible pitfalls, but no matter how good you are you'll never be 100% safe once code starts reaching tens/hundreds of thousands of lines...

Very true. Roguelikes are projects that would indeed benefit from this, more so than a lot of games actually. There is so much complex interaction happening, you never know what might affect something else.  I see it constantly in my RL projects, sometimes to hilarious effects, sometimes not...

Professionally I design a class/component (requirements/use cases/functional design - UML stuff) and then create test cases for it. Before it is ever used in an actual real context.  I escape that rig-moral in my roguelike projects, I just code and play test. I guess that is why my code is full of bugs

If you can write unit tests for your code, go for it. I would imagine that the benefits will eventually out-weight the costs.  
If you are writing tests as you write your code, you will find that you are writing better code, as you are thinking about functional components and code encapsulation. Myself on the other hand might write a 300 line function that does 150 things and it does not really belong in the class I put it in anyway. I could never write a unit test for it.

To sum up: Unit tests are awesome. I am just not bothered with them for my amateur solo projects.  They are essential for any team projects where you want reliable software.

[tuturto]

I like using unit and integration tests and have found they make my life easier and coding less troublesome. I'm not using automated system though, where they would get executed every time I commit a code change.

Advantage of the tests is that I can keep track of what I was doing, even if I get interupted and have to go away from the computer for extended time. When I come back, I just run the tests and see what I have to fix to get the error to go away. If there aren't any errors, changes are that I was either doing something that can't be tested by machine or had already finished what I was doing.

Because of writing tests, I probably code slower than if I skipped them. But I like to think that because of the tests, I end up with prettier code that works more often than not. And since there is no deadlines involved, I can take all the time I want anyway.