Are you improvising or singing on cover tunes? I've found that when improvising, start by singing 4ths or 5ths that will make sure you don't hit some non-diatonic note that will stand out like a bull in a China shop!

You can work from there.
In cover tunes though, if there is a harmony there you should hear it on the recording. Sometimes if you add it in when its not the tune can sound "wrong" That all depends on the song, the style, where you're harmonizing and how much of it you're adding. Tastefully done it can work. just don't overdo it. As far as what notes, any notes out of the chords should work as long as they are fairly close to the melody. Just use your ears.
