There was an interesting article that Yahtzee Croshaw posted recently which I found raised some salient points:
http://www.escapistmagazine.com/articles/view/columns/extra-punctuation/9276-Context-Challenge-and-GratificationEssentially it's saying a game is made of a mix of story/setting, gameplay and sheer fun. Applying these archetypes to roguelikes and you can see some clear ways that the traditionals fit along these lines. I might make a separate post on it some time since I think it raises some interesting points. It's a little different from this discussion of gamer types, but is still a good read.
I don't like all of your groupings I must say. There's no such thing as a "Smooth Operator" that plays a game purely for interface. Instead there's the opposite, the person who is willing to go to pains to play a game in spite of a bad interface. It's a commendable trait in a gamer, but a terrible trait in a developer to target that audience alone, since it inherently limits your audience immensely. What you class as "Smooth Operator" is more merged in with "Quick Mixers", since a quick game with a complex interface is pretty much doomed to failure. That's not to say bigger games can't have smooth interfaces; rather there are a few that can amass enough fans who don't care. Those that have a good interface will innately have wider appeal.
On roguelike radio we rant on about UI a lot, and sometimes it feels a bit repetitive, but in the world if roguelikes I think it stands to be repeated. Too many developers are complacent with the idea of sticking to a "traditional" bad UI. If they're only coding for themselves and some die-hards then fine. But it's ignoring a strata of gamers that could be enjoying and contributing to their game too. I say a strata since it's not a separate group - plenty of people who like die-hard gameplay or sandbox or story or whatever mixture still demand an accessible interface. To use the coffee example, UI is more the mug it comes in than the coffee itself, but if it's too difficult to grasp it inhibits the enjoyment. For roguelikes it doesn't have to be pretty graphics, or graphics at all, it just means that if players get frustrated by the controls in the first few minutes they'll move on to another game that won't frustrate them. In the modern roguelike scene they have plenty more accessible choices.
Oh, and I really resent the idea that there should be any distinction between diehard fans and those who like UIs. This implies there is some elite rank, which is very false. Plenty of diehard fans care very much about UI, and I see many start to care when they notice how other games have made leaps and bounds beyond the classics without sacrificing any complexity. There are a few curmudgeony sorts, but these are rare, and are generally the sort of bitter elitist that hates everything. These are found in many nerdy communities and are best ignored, since they are oft little more than pathetic trolls that desperately want to appear superior to others by clinging to their illogical ideals. Uh, not that we have anyone around here like that...
"Story Lovers" is a bit of a misnomer, since it's not necessarily about a linear story - well, not in this genre anyway. Linear stories don't work well in roguelikes, and perhaps are a bad thing in games in general. Atmosphere is important though, and this is what many crave in a game. It doesn't need a lot of text to attain. Nethack has more text in it than Frozen Depths, but the latter is far more engaging because it has a coherent theme and gameplay mechanics that support the atmosphere of the game. Note that "humour" can also be a theme, which is one thing Dredmor uses to engage with players. A good atmosphere can help one overlook other flaws in the game, as is evident in many AAA commercial games. A game like Skyrim can have an epic feeling without a linear story, and I think it would be great to see a roguelike do the same.
If ignoring story/setting and any individual feel to the game then it's best to go with generic fantasy tropes as these have the benefit of familiarity to the player. It'll be off-putting to some, but it'll require less focus on imaginative elements and long descriptions so you can instead put all effort into gameplay details. Every player knows how goblins, orcs and trolls rank against each other, and how pyromancers will behave - it's an easy setting to get into and straight away offers many design opportunities. The only downside is if you have some very individual design ideas, and you might need an individual setting to truly support your goals.
Some other elements to consider:
"Tight Tacticians" - ToME4 and DCSS are the best examples here. No room for scumming, every decision can be important, spoilers are less useful. Your actions mean a lot, and there is a high density of meaningful choices. You can't blindly hold the direction key and win. Some people misinterpret this as difficulty - it's not, it just demands care and attention, and a game appealing to this group should have a very high success rate amongst those who play perfectly. The caveat is that no one is perfect. Some players don't like this - they enjoy being able to scum, and they like getting their characters to invincible states.
"Gimme more"s - Tie in with sandbox a bit, but also with very complex games like Nethack. These games are characterised by numerous item and enemy types (oft very similar) and a wealth of interaction opportunities, leading to innumerable possibilities in every playthrough. This is one of the joys of roguelikes, but leads to the problem of being hard to maintain balance, coherence and of course a decent interface. Also encourages the use of spoilers. The historic games can get away with this by reputation (though many have now died or are dying) but very few new games can pull this off without some very careful thought and design. Still, it's a big attraction for many roguelikes, so whether to appeal to this or not is an immensely important design decision. It doesn't work well with Tight Tacticians due to balance issues.
"Simulationists" - Some players like games that represent real world mechanics very well, even to the point of including mechanics that others consider painful to play with. They tend to like crafting systems, weapon modding, building in-game and very detailed combat mechanics. This isn't just sandboxness, it's down to wanting exceptional detail in every area. Can crossover with Gimme Mores a bit, but can also be wholly exclusive, and perhaps works best when separated from grander designs. A very small but focused game can maybe do achieve this to best effect.
No matter what do not listen to those that say the classic style of game play is dead. This was postulated by one of the commentators on Roguelike Radio. Was it Ido? It will only die when the players die.
I think I said that some of the classics were doomed unless they get with the times. To those that say the classics won't I die I ask, where are all the Moria players? Moria is still a good game, but it no longer gets significantly played or discussed. It is, in essence, a dead game. Angband and Nethack run risk of following the same route. New players would rather go to ToME4, DCSS or Brogue, and the existing communities are visibly dwindling. There will be a few diehards for many years of course, but why should the great classics put up with this when they are open source and ripe for improvement?
