And yes, absolutely check the bridge saddle under the E string. Sometimes a subtle half turn of one or the other saddle-height-screws can solve a problem where one of the feet (screws) isn't just quite perfectly seated. And the nut... could be badly cut or worn in the E string slot, allowing the vibrating part of the string to rub against the edges or bottom of the slot, rather than having the playing-edge of the nut slot be the firm pivot point that the string rests upon.
This is all assuming the string is ok, and also assuming there isn't some weird but unfortunate thing going on with the resonances of the instrument itself. I think the bass itself being the cause (in a "just how this piece of wood is" kinda way) is pretty unlikely though.