The only downside to a PJ setup is that if you have the pickups mismatched, the P will be much stronger than the J, so if you want to pan to your J p'up, you'll have a drop off in output (which is why it's important to remember this when you're buying p'ups...
The opposite is true as well....if you solo the P pickup, you may get a big volume increase....which is cool, until you turn the J pickup back again and your sound disappears.
When I had a P/J in the past, I used the P pickup only for the majority of the time, only using the J pickup when I wanted slap or have some more bite on my fingerstyle tone. However, I was actually harder to hear during those parts where I most wanted/needed to be heard, so I had to compensate by increasing the volume at the amp. Plus, the J pickup always had hum since there was no RWRP neck pickup to cancel it.
The answer IMO, if you want to stay passive, is to find a vintage-type P-bass pickup (Fender Vintage, Duncan SPB-1, etc.) and pair it with a hotter Jazz pickup. The P/J never really sounds like a true Jazz anyways, so there is no reason not to get an overwound, hum-cancelling Jazz pickup that will not be drowned out by the P pickup.