You would need a third coil on your P pickup to be able to do that.
Honestly, from the bottom of my heart, cross my heart and hope to die, what you are suggesting would only be done by someone whose primary goal is tinkering, not canceling hum. There's nothing wrong with tinkering, but if your goal is to cancel hum on a PJ bass, don't mess with the pickup that already cancels hum. Just get yourself a J pickup that also cancels hum. There are a lot of them out there. You can spend anything from about $70 to $200+ to get a new one, you might be able to find used ones for less. Trying to do what you described would cost a helluva lot more, and there's no guarantee it would even work.
My favorite types of hum canceling J pickups are the split-coils -- they have two coils in line wired in series, with each half covering two strings like the P pickup. There are stacks, with two coils, one wound on top of the other. Less common and more expensive are the parallel coils, with two very thin coils wrapped side-to-side. There are others, I'll let you do your own research if you're interested.