As I see it, there are two objectives in deciding a combat system- coherence and fun. Too much random is unfun while complicated algorithms are incoherent, resulting in incoherent player decision-making. Too much determinism, however, can also be unfun- as it produces a bit too much tedium and chess-like stress.
I suspect that many use a probability distribution in combat to ensure that things are mostly predictable, but allow for the occasional random win/fail.
I've been fooling around with the source code in Brogue, which is a very mechanically transparent game, and realized that the way the combat actually works isn't particularly communicated well to the player. Specifically- how much Armor, Life, Damage output, and Accuracy are necessary to be successful at any given depth. Brogue diminishes the importance of knowing the precise numbers with the various staves/wands/consumables you find- but it can be difficult to know what trade-off is actually being made when you use a Scroll of Enchanting. If I put an enchant into my Broadsword over my Plate Armor- how does that effect my ability to survive?
I particularly enjoy Brogue's system, which is why I've been looking into it. I don't want to talk about Brogue specifically in this, but rather what systems, in a general sense, make good sense. One issue I see with Brogue is that it uses a number of internal mechanics that the player doesn't really know about- particularly in how hit probability is determined. Defense, for example, has diminishing returns, while attack (the relationship between accuracy and damage) has increasing returns.
The numbers for the individual monsters, however, are so finely tuned to the system, that it really isn't clear which is best to enchant. And in fact, against the most dangerous monster (Tentacle Horror) with top-tier vanilla gear and 10 enchants to go around, it makes very little difference where you distribute enchants- you'll win by ~1 turn worth of damage (1.5-.5, favoring armor) assuming average damage relative to hit-probability is dealt each turn (in other words, if you do win, you'll die if anything breathes on you shortly after). In a balance sense, this is problematic because it leaves much less flexibility to deal with long-range flame attacks. In Brogue, there are essentially 3 problems you have to solve to succeed- A solution for dealing with Revenants (immune to physical damage- not really a big problem but still pertinent), a solution for surviving Fire-breathing (from dragons and other deep-dwelling denizens), and a solution for horrors. So we run into other balancing issues for solving each problem as each problem demands a certain number of enchants in different categories of gear.
There are two real issues here-- One is affordance. How can we communicate to the player that what they're doing actually improves their odds for surviving, and two, how can we produce a game experience that doesn't appear to be tailored for the player but still provide opportunities to advance without compromising a diverse experience?
So-- A sort of general question-- What makes sense? What sort of combat system is simple enough to grasp but allows for interesting and emergent scenarios for the player to deal with without promoting a 'hunt' for particular gear for particular solutions? What examples do you think are most successful, equitable, interesting, and diverse?
I suspect that the D20 system is a popular answer to this problem because of how familiar it is- but at the extremes (high and low levels), the system is wholly uninteresting- too random or not random enough.