Somehow I managed to miss this thread all along...
*SPOILERS*
You can safely ditch Scrolls of Random Uselessness, Curse Weapon/Armour, and Paper. Scrolls of Noise and Fear are almost useless, too. If you're carrying Remove Curse, you can stash your Detect Curse scrolls. Scrolls of Fog, Vulnerability, Immolation, and Torment have their uses, but they involve some risk to your character, so I usually stash or ditch those as well.
Some scrolls can be used as soon as you find them. This goes for Scrolls of Enchant Weapon/Armour, Recharging, Acquirement, and Vorpalize Weapon. You may be tempted to save these for 'later', but in my experience, 'later' usually turns out to be right after you were blasted by a fireball.
You can safely ditch Potions of Confusion, Slowing, Paralysis, (Strong) Poison, Degeneration, and Decay (unless of course you have the Evaporate spell). Potions of Porridge and Water are actually food, and Water is nearly useless. Potions of Experience or Gain Str/Int/Dex can be taken right away; the same goes for Mutation if you're feeling lucky. Potions of Berserk Rage may be redundant for Berserkers, Potions of Magic are useless to non-spellcasters, and potions of (Coagulated) Blood are useless to non-vampires as far as I know.
Safe places for stashing are the Temple, Lair:2, and the bottom level of the Hive. It's not necessary to collect and stash anything and everything just to remove threats. Do remove curare needles, most wands (especially draining), weapons of distortion or pain, vampric weapons, weapons of venom if you don't have poison resistance, weapons of electrocution if you don't have insulation, orc slaying if you're an orc, dragon slaying if you're a draconian, and holy wrath if you're undead.
If you're absolutely sure you don't need an item, you don't want anything to wander off with it, and can't stash it, you can throw it into deep water or lava. You can in fact throw any item with the "f" command, if you press "f*" to select any item from your inventory.
This all assumes you've somehow been able to identify everything you find, which isn't going to happen. It's probably wise to take unidentified wands, and weapons/missiles/armour with special descriptions, to stash them somewhere safe.
Oh, and another good way to save inventory space is to use darts and rocks for throwing in stead of daggers, spears, or hand axes. To avoid being burdened, try dropping some items off a heap in your inventory; use "d#i" where "d" is the Drop command, "#" is a number, and "i" is the letter of the item to drop.