It's not as bad as you think it is. Actually, just to freak you out a little more, it is all over your entire body except for your palms and soles. Basically, anything you touch with any part of your body, other than your palms and soles of your feet, you're transferring sebum. Sweat carries it as well.
You know that smear on your bass finish? You may want to wash that off if your a germaphobe. Just don't become too much of a germaphobe, because without sebum, microorganisms will get right into your skin. Just a little food for thought.