I think I could adapt, and I do like the idea of having a more versatile instrument if having only one. But yeah your logic makes sense.
The more I play and the older I get, the more I distill things down to thinking the regular old one pickup P is the most versatile instrument there is. I have several P-Js, one with a double jazz in the bridge as you have. It can make some different sounds, but honestly, 75% of the time I'm on the P pickup only anyway; the other 25% is just adding a little of the J pickup to add some midrange. A good amp, and the P alone will growl for days. It's just the perfect bass sound.
I have also recently discovered, in going back from 5 str to 4, that the 4 is not only easier, but more fun to play. For what I do, the B string is a very nice thumbrest.
And from what you say, the MIJ is your love. So I'd flip the 5 as you had planned; they are indeed hard to come by so you might make some moola.
But OTOH, your guilt might be talking, what with a new babe on the way. That 5 string you have is a good one, as we have said, somewhat rare, and if it makes you happy, you should perhaps try to put aside the guilt and keep them both. Always have a backup, too. Not kidding about the backup.
good luck!