If they do actually die you could save info about that event in a config file for next time they play. The game would keep on spiking the player trying to equal but not exceed their ability threshold.
That sounds like just about the worst thing you could ever possibly do. I'll come back to why in a moment.
Games, like stories should have a narrative, development, progression.
Well, that depends on what you mean by 'games'. Games can be
like stories, but games are not stories. Let's delve into that a little bit because it might make it a bit clearer why we're so sceptical about this.
Imagine a line. At one end we'll put 'gamey games' - games which are totally abstract and are basically rigid rulesets within which the player(s) have to optimise their actions as best they can. Things like Go, Tetris, Football etc. The appeal of these comes from their challenge and a sense of personal progression as you gradually increase your skill and knowledge.
At the other end, we'll put 'narrative experiences' - games which some people might not even call 'games' but which are still very much 'play'. These have no fixed rules - everything that happens is controlled either by a god-like author or by the players themselves. The purest form of this would be something like a playground game of 'let's pretend' or just simple bog-standard fantasising. The appeal here comes from a number of things, but mainly wanting to see how the narrative develops and a bit of wish-fulfillment.
In between these two extremes you have most actual games. Chess is more towards the gamey end, but with a slight aspect of fictional flavour in that the pieces represent kings, knights etc. Art games like
Dear Esther go more towards the other extreme. D&D would be more towards the fantasy end, but can shift along the spectrum depending on how you decide to play it - you can choose to follow the rules exactly or to bend them to suit the narrative you're trying to create.
Roguelikes traditionally fall very much on the gamey-game end - they have clear and rigid rules with strong penalties for failing to observe them. Their stories are usually minimal or non-existant and more often than not their fictional universe is cribbed directly from some other work. (To complicate things; they do have their own kind of narrative experience, but it is one that arises directly from the rules of the game and not one imposed by an author or the player - it's a subtle but important difference).
It sounds like what you want to create is a game that is far more towards the fantasy/role-play/make-up-your-own-adventure/anything-goes end of the spectrum. You want the player to be able to play the game the way they want to and for the rules of the game not to get in the way of that. I don't, for the record, think there is anything wrong with that.
But what's important to understand is that the reason there is a spectrum at all is that there is a tension between these two poles. They are two very different masters and serving them both at the same time is difficult - design measures that you take to enhance one can often (not always, but often) harm the other. Consequently, trying to take a set of design characteristics engineered towards one end (such as the basic roguelike formula) and trying to drag it down wholesale towards the other is probably not the best way to approach the problem - the end result is likely to come out a little bit confused about what it's trying to be.
Which brings me back to why
If they do actually die you could save info about that event in a config file for next time they play. The game would keep on spiking the player trying to equal but not exceed their ability threshold.
is a bad idea. Well, there are several reasons; people don't tend to enjoy sudden difficulty spikes, for one. But putting that aside, the chief reason is that it undermines the whole purpose behind death (and more specifically, permadeath). In gamey-games, death is a lesson. It's the game saying 'don't do that'. Ideally, every death should teach the player something and leave them a little better prepared to face similar challenges in the future.
But your system would poop all over that. In your case, the player might learn something from death, but it would be the wrong lesson, because the aim of the whole thing isn't to teach the player how to adapt to the game, it's to teach the game how to adapt to the player. They could make exactly the same mistake next time and come out OK because the game has shifted the goalposts. Worse, the player isn't even being punished this time because they did something wrong - they're being punished because the
game did something wrong.
From a narrative perspective, player death is usually a bad thing because it interrupts - and perhaps ends forever - that narrative. A lot of adventure games, for example, have no death states because they don't want to have to pull the player out of the story by cutting it off and making them go back a bit. The 2008 Prince of Persia also - rather bravely - did away with death altogether (which I personally hated, but then I'm a gamey-game kinda guy).
So your system ends up trying to be all things to all people but succeeding in being nothing to anybody. Gamey-gamers won't like feeling like their actions have no consequence and narrative-role-playey people will just be annoyed that they've died a death that served no purpose and don't have that unicorn they wished for.
Rogue was a very difficult game because all games were like that then. Go and play even a Disney game from the 1980s. They were hard as hell. Certain people, like me, were prepared to play them even so. Why? Because I was a kid, because I had lots of free time and not much to do but fight with my brother. Because getting to the end of Black belt would be the ultimate in bragging rights. But these days things are different. Who's got time to play a game with only 10 minutes of actual game play, but which takes months of hours a day practice to beat?
Sure, a difficult game will force you to improve so you can beat it. You can't just coast through. But there has to be a better way than save die reload or die restart infinite loop. At the end of the day we want people to play our games (I assume) and there aren't many people with the patience to put up with that kind of thing any more.
None of this is really relevant, though - we're not talking about absolute difficulty, we're talking about auto-adjusting difficulty, which is a separate issue. If you want to make a game easier then you can just go ahead and balance it so it's easier. Likewise, if you want to make classes other than clerics viable then you can balance the classes to make them all equally difficult. If you want a game with more than 10 minutes of actual game-play then make a game with more than 10 minutes of actual game-play. There are other solutions to these problems which don't involve compromising a core mechanic of the game. By far and away the best option would seem to be to provide a suitable range of difficulty levels so that the players themselves can choose what it is they want to get out of the game.