Modular monster intelligence system

[kuniqs]

Modularity is a thing not very common among the world of roguelike source code. Most of it is based upon medieval designs such as Nethack or Angband, plus the community itself only few years ago started embracing more open minded languages such as python (how python can survive in a fridge?!) abandoning ye olde senile C. Still, I have seen many outdated things written in Python I would like to forget.
It's bad since many, many roguelike projects get abandoned because the author (who's probably more of a programmer than designer) gets bored by the spagetti.




Neural Nets

What the industry calls "Neural nets" are really pattern recognizing primitives, no sentience involved. We take some inputs, perform something akin to weighted average on them, and if the output is > 0, we have recognized or liked something.

Code: [Select]

        input1       input2      input3
            |               |               |
       weight1   weight2     weight3
            \               |              /
             \              |             /
              \             |            /
               \            |           /
                \           |          /
     
                        Sum

                           |
                           |

                          f()

                           |
                          \|/
                       output
   Standard issue neuron

Schematic of neuron in work is below:

1) Sum = 0
2) for each input <> weight pair
   3) sum = sum + input*weight
4) output = F(Sum)


f() is a special function used almost exclusively during net leatning, so we don't need to concern ourselves with it now. Weights can be of any range, but for net learning purposes they range from -1.0 to 1.0. We ignore that now and use any range we find suitable.

How to understand them intuitively? How do you rank cars, for example? You probably take colour, speed, silhouette, cost and age into consideration. The higher the attribute, the more desirable it is for the 'general' perspective. So, high value for the age input is a young car, high for colour is a red car while a low (negative) value for speed is a slow car.





If you like 'best of the best' models and are a spoiled rich kid that doesn't care about money and  changes his car for newer every year, your net weights might look like this:

Code: [Select]

colour     speed       silhouette        cost       age
10           15             10                   0             20
(weight of 0 means 'we are indifferent')
This guy values most the fastest and youngest cars with good appearance, but doesn't care if the car is expensive.


Or maybe you're an eccentric who likes old, ugly cars bought in retail prices, and..

Code: [Select]



colour       speed       silhouette       cost       age
0                0                 -5               10          -10
..you don't care about car's colour or quality and the older the car, the better.






If we take a average, but expensive car like this:

Code: [Select]



colour       speed       silhouette        cost        age
3                2               5                   10            2
Our millionaire will react to it like this:

Code: [Select]

colour speed silhouette cost age
10 15 10 0 20 weights
* * * * *
3 2 5 10 2 inputs
+ + + + =  150


Since the output is more than zero, we like the car.
For an old, cheap, painted in pink amphibia we get a reaction of:

Code: [Select]



colour speed silhouette cost age
10 15 10 0 20 weights
* * * * *
-6 1 -2 1 -5 inputs
+ + + + =  -165
The vehicle is repulsing.
Important thing in all of this is that inputs can vary a bit and the spoiled kid will still react negatively to .



That technique can be used for making npc's a tad smarter by making them evaluate their targets. Lets say we want a goblin to avoid strong enemies. How do we define 'strong'? Are tigers strong? How strong exactly? If we make all goblins avoid tigers, all goblins will run from all tigers, so this will get old quickly. If the goblins are cunning, they may run away from adult tigers, but pick on baby tigers or even malnourished adult ones if they have good weapons and significant numbers advantage.
Humans (and goblins by extension) can only approximate if somebody is strong enough. Using neurons, we can emulate this behaviour.
Let me see..  Goblins are cowards who dislike attacking anyone taller and with more friends than them. They also know how to backstab - attacking creatures not hostile to them. They're not stupid or sociable enough to sacrifice themselves for the greater good and will value their own health very much. If the output is positive, there's more motivation to attack than not to. Input values are obtained solely by functions, to be more general than direct variable checking.

Code: [Select]



input weight
IsHostile(Enemy) -2
WeaponValue(Goblin) 5
NumberOfFightingAlliesOf(Goblin) 10
HeightDifference(Goblin,Enemy) 4
HPDifference(Goblin,Enemy) 5
Difference(HP,MaxHP) 10
This way, we designed an 'Attack' neuron, that will call a mystic InitiateAttack() function if there are no better actions to take (no other neuron has higher output). What sets this apart from other AI methods is that the goblin can act a little erratic - sometimes it runs away when heavily wounded and surrounded by friends, sometimes it's wounded heavily, but so is the surrounded enemy so the goblin keeps fighting. We can have either species weights (all goblins use the same weight vector) or custom weights (every goblin gets his slightly altered set of weights). Adjusting the weights to change a coward into a berserker is very easy from both the programmer and the monster designer's perspective. Some gobbos are afraid of everything, others are overconfident and there are some dirty cowards mugging on the weak and alone. The player will never be sure.

Those are the most basic things in neural net theory, but enough for us to understand what's going on below.



   Modular decision maker.

Code: [Select]


struct Neuron{
VectorOfInputGettingFunctions I; // those are something like int Func(Creature Self, Object Target)
VectorOfWeights W;
FunctionToCallAfterGettingOutput F; // void Func(Creature Self, Object Target)
int output;
};


struct Brain{
VectorOfNeurons N;
};



void RunNeuron(Neuron N, Creature Owner, Object Target)
{
 call all those Input Getters, building the I vector, multipy I vector by W vector, obtaining output
}

void ContestOfNeurons(Brain B, Creature Owner, Object Target)
{
  call RunNeuron for all Neurons in vector N
choose the neuron with maximum output
return it
}

void UseTheBrain(Brain B, Creature Owner, ListOfTargets L)
{
for every target in L:
call ContestOfNeurons(B, Owner, Target)
store the returned neuron somewhere

choose a neuron from the stored ones with the highest output
call that neuron's F function
Fin
}

There's more pseudocode than C here, but almost everybody will have their own ideas of what a Creature and Object is. In my book, Creatures are Npc's, Objects are Creatures, items and possibly (if we go for very general design) doors, trees, anything we can interact with. Since we use value obtaining inputs instead of directly wiring the input vector to some raw values, the neurons can obtain a sensible value if we try to obtain a weapon value of a door.


   
   Why should I use it?

Remember our goblin? What if we want to teach him something like casting spells? While playing the game? Using states, this requires some magic with flags and ifs which is nice, but a bitch to modify once you've written all the state code.

With the modular design of neuron method, Goblins have Move, Attack, Flee, PickUp, Wear and other neurons. We just add water, or the CastMinorSpell neuron and that's all. Great thing is that we can add and remove those neurons at run time, so we can have a spell named Evolve which grants spellcasting abilities to any humanoid we cast it upon. Or Touch Of Idiocy, which retards everyone into zombie who only StandBy, CloseWith and Attack.

We can also add some unorthodox behaviours like rust monsters hunting for swords in the dungeon, orc pyromaniacs blowing up every powder barrel they spot (if they have too much mana on their hands), kobold skirmishers who keep 1 tile distance to the player when wounded and moving for a strike at full health, or Confusion spell which gives a temporary neuron with a very high output and a function which makes the monster move randomly. Maybe incorporate a phobia system in your game, so every mundane monster has a chance of being blessed with arachnophobia (reacts very negatively to spiders), hydrophobia (hates water elementals) or paranoia (has an insanity counter incremented every time hears a weird noise, starts panic after a certain number of them). Some minor changes in neuron themselves could be done by Fear spell, which alters some kind of EstimetePower function to give much higher value for the caster.

As I said, altering the brain is a trivial task - we can design dozens of neurons and construct many different brains from those blocks. Also storing the weights themselves in a file is nice and clean, so is making minor changes to them.


   Why shouldn't I use it?

Since we have to use indirect function calls and calculate every neuron, it's extremely slow in comparison with most other roguelike AI's. Multipy that by the number of targets we want to check with the brain. Turn time roguelikes (most of them) probably won't even notice since you have the user slowing down the game, but this system is rather overkill for real time. I think the reuse and ease factor far oughtweights time and memory consumption and I intend to use this in a general roguelike engine.

Some neurons can frequently alternate between negative and positive outputs - some monsters may exhibit unbalanced behaviour, quickly changing from escaping to attacking or simillar flavour. Could be a problem when we want monsters who can be relied to run at least N turns when scared.

Plus, having unbalanced weight ranges for various neurons can result in the lower ranged ones being seldomly called, if at all. If we assign CastSpell neuron with min/max weight ranges of -10/10 to brain with neurons of average range of -100/100 will probably exclude that neuron from being useful. Additional tests need to be made, if there's really a problem it is solved by restricting the weights to a global range, -10000/10000 for example with normalizing functions when loading the weights from a file.

There are other methods like this one that are more lightweight, like Need driven AI by Bjorn Bergstrom http://www.roguebasin.com/index.php?title=Need_driven_AI.

[requerent]

You ought to look into component-based entity systems- or something like http://code.google.com/p/awe6/.


The problem with neurons seems to be obfuscation of the design process. It's all well and interesting when our goblin has hundreds of things that s/he is considering whenever s/he makes an action, but in game design all that matters is the frequency with which that will have a genuinely interesting effect on game-play. Creating algorithms that appropriate weights that will result in interesting game-play is more difficult that making an entire game that is both fun and interesting (you've got something like order n! possible combinations of neurons that should generally result in interesting entities for your player to fight against).

It doesn't matter how an AI is coded as long as the observer is duped.

I don't think the problem is time-complexity- there are linear-time solvers for probabilistic knowledge bases (of which you could convert your neurons into a KB for each entity). The problem seems more to me that it's too difficult to ensure that it will be fun unless you make it very very similar to needs-based or a gambit-like system.

Gambit system might be more sensible- you can randomize the values/order/relationships between gambits a little bit for each enemy based upon its abilities to govern what they do in a more coherent way. A rambunctious goblin might have "(HP < 2) => Run|Heal", whereas a careful goblin might have "(HP < 10) => Run|Heal." If a goblin has 'cure light wounds,' that spell can have a property called heal, of which the goblin will pick some form of healing whenever the condition is met (Run and Heal are states- each state offers a different set of overriding gambits, or something of the sort). Generating coherent gambits would also be difficult, but certainly not as difficult as weighting neurons.

[Bosman]

I've had a bit of experience with neural networks so just wanted to add my two cents. First, I don't think this approach will be that slow. If coded right and efficiently I think you can make it work pretty well. Besides, I think it's not necessary to run the UseTheBrain for every creature every single turn. You may want to make creatures reconsider their actions only every x turns (and additionally in case of an event like being stabbed in the back or something   so that only a fraction of creatures uses their brains every turn. This may also make it more natural and avoid situations in which a creature does something like:
Turn 1 = I see a precious ring -> I'm going North. Turn 2 = I'm to close to the enemy -> I'm fleeing South. Turn 3 = I see a precious ring -> I'm going North, etc...

But the most important thing I wanted to stress is that you did not really mention anything about learning the neurons which is actually (in my opinion) the best you can get from a neural network. You could actually create a tool that would use the game engine to, let's say place a Goblin and a Wolf in an enclosed location and make them fight. Based on their actions and the outcomes the learning function would adjust the goblin neurons and wolf neurons. That would allow you to create some pretty clever creatures automatically. What is more, you could make this work during regular games. The game could learn based on player's actions to be generally better but also to learn the player's game style. Let's say you are a melee killer... the word spreads and then you start encountering enemies that are affraid to come close and prefer to throw things at you from afar. That could be pretty interesting.

Generally, I like your idea of making use of neurons.

[kuniqs]

It's unlikely I'll include neuron teaching addition since it incorporates a bit too much math for now and differential calculus neural net learning algorithms are still haunting me in nightmares .
Big time problem with monster net teaching is that we need to know what actually teaches. Survival time? Monsters learn to run all over the place. Damage done vs. damage received? Monsters turn suicidal maniacs and so on. I've never seen a good example of neural nets used in mons evolving over time (maybe Creatures, but this ain't a conventional game).

[jlund3]

I think it's not necessary to run the UseTheBrain for every creature every single turn. You may want to make creatures reconsider their actions only every x turns (and additionally in case of an event like being stabbed in the back or something   so that only a fraction of creatures uses their brains every turn. This may also make it more natural and avoid situations in which a creature does something like:
Turn 1 = I see a precious ring -> I'm going North. Turn 2 = I'm to close to the enemy -> I'm fleeing South. Turn 3 = I see a precious ring -> I'm going North, etc...

The one problem with this approach that I see is that the monster will not be able to react to new inputs very well. Perhaps the goblin is going for the the shiny ring, sees the player, but has to wait x turns to avoid the player, making the goblin essentially perform suicide-by-player. Of course you could hard code all the various events that could trigger the UseTheBrain function, but by the time you have considered all the possible UseTheBrain triggers, you might as well ditch the neural net and go with a more traditional state based or switch based approach.

An alternate approach that might allow you to use a neural network is to consider a very simple recurrent neural network. The basic idea is that the network feeds outputs from one turn back into the network on the next turn. Perhaps the RunAway neuron fired very strongly last turn, causing the goblin to retreat from the shiny ring out of the player line of sight. With the recurrent network, that strong firing from previous turns could continue to influence the goblin so he doesn't forgot about the player and go back for the ring (and certain death).

Of course this adds more complexity to the network, particularly if you are hard coding weights by hand rather than training using whatever technique you learned about in your machine learning class. Still though it shouldn't be too difficult to do if you were willing to hard code the original non-recurrent network in the first place...