That does not look like a question anymore. If you already have a solution and proud of it and it even works, okay, use it. If you want to discuss something instead, please pay attention. (Well, who am I kidding here.)
So the situation is, objects may disappear abruptly so you made a list of reverse references back to actors to inform those actors about disappearences, right? And now you wonder if it is efficient to keep these references thing in every object and asking whether there is a more efficient way to notify actors that their target has expired.
I suggest to take a step back and take another look at the source of the problem: actors noticing that object they are interested in has disappeared. Reverse references is not the only way. If you keep a simple forward reference in a form of a weak_ptr in the actor to the target, your actor NPC will know that the object has disappeared right at its next update step. Without any extra machinery involved.
The passage about IRL was to hint that there is no need to anything inform anyone to notice changes. You just notice changes next time you pay attention to something and see that weak_ptr has expired the object is not where it was.