Perhaps it has something to do with my gender (although I'm a measly 5'8" and no muscle to speak of), but I do not equate isolation with vulnerability. If no one else is around to see me camping, there's no one to do me harm either.
But even if they see me and in the unlikely off-chance they wish me harm, there's the .45 by the bed or the .32 in the pocket. Sometimes bear spray on the belt too. Why feel vulnerable unnecessarily? The right tool for the right job, that's all it takes. While caution is prudent, needless fear can rob us of some good experiences.