From what I gather from your post the player is against squads of enemies. This is a bit short to say the least. Knowing exactly what you want to achieve and the general combat mechanics would help a lot
More importantly the efficiency of an AI is usually depending on the map information / environment. Without knowing about it doesn't help providing any advice (and explaining those can be overwhelming to the reader anyway).
I have not dabbled into planning systems in any projects I have done before. From what I (think to) know they are more suited to strategical game than tactical ones, but I could be wrong. Still, here's a thing for you to consider:
Is the player going to see the difference between this (quite obviously) very long to write, test and debug AI system and a simpler model? As a programmer interested in AI, I perfectly understand that this is quite fun to experiment with, compare to other systems and so on. But if you're planning to publish it as a public/popular game, this may be not the most important factor to look for. Roguelikes (and most fps, tactical games) usually have somewhat modified FSM or Fuzzy Logic systems for combat mechanics, usually because they don't need more and because it's efficient enough.
I realize that this paragraph could be read as "don't do it". I apologize for that. On the contrary, I would be very interested to get updates from you on this game, or even the prototype and general logic you've put in it. If you're doing the project for yourself in priority, go wild with any idea you have, otherwise you may want to consider what users are expecting in priority.
This said, it probably won't help you much, but last time i had a try at squad combat I did something a bit different (that may be reused partially in my own project,
GRA). There's probably an official name for it, but let's call it "dynamic terrain" for now. The concept is the following:
Each square of the map have a bunch of combat related variables that can be assigned to it like the amount of cover, the ability to shoot easily from this square, the level of sound made while walking on it, and so on.
Each time the player moves, all squares in a given radius around him recalculate those variables based on his position, and eventually the position of other NPC (if X is there then my cover will be better because he'll be shooting at the player). For example a square at a distance of 5 from the player with 2 squares occupied with crates or other furniture will have a high cover rating but a low offensive rating. Once calculated, you'll get obvious positions around the player for your NPC to hide (reload and heal), somewhat secure paths to flank the player, positions from where your NPC can get a ranged shot easily before going into cover again, and so on.
So each NPC depending on his stat / needs will choose the most appropriate square to go to without having to rely on anything else than a FSM/Fuzzy mechanism. The NPC look like they are coordinating an attack to the player's eyes when in reality the terrain is dictating where they should go to satisfy their current state.
But it really depends on the combat system here. If it's a regular roguelike with no possibility to run, burst shot, or well do at least 2 actions in a single turn, any advanced AI system is kinda pointless, IMHO.
Cheers,
SK.
