There is a lot of thought put into pickups, and if there were zero difference, then there would only be one or two pickup manufacturers, period.
Like anything else, there is of course a limit to differences, starting with P bass pickups. One configuration, and all you can really do is vary windings and pole pieces. I find, that you can open up a P bass a bit by changing the pickups (I just did that on one bass, going from stock to Dimarzios) but at best you will gain a bit more high and definition. I use La Bella Flexible flats, so the differences I will notice are not as much as if I used round wounds. But I definitely have some more frequency range.
For J bass pickups, well, you can get a wide range of differences, including from single coil to noisless. And, you have large poles, angled poles, two poles per string, ceramic, not ceramic, and heck, even blade style. Again, remember, this is a bass, so at some point it will still be variations on a theme, that thing called bass.
Swapping electronics and wiring I have found makes a slightly better difference. Using high quality pots, getting good wire, is as refreshing for a bass as changing pickups or strings. Its just like cars. They all run, and a Ferrari is about as fast as a Corvette, but its the little things that make a huge difference.
Also, if you are looking for a more vintage tone, then yes, replacing pickups with a high quality, specially wound pickup will get you closer to the tone you want. If you want to sound like Jamerson, then you need a '62 Pbass wound pickup with Jamerson gauge La Bella flats, dip them in all kinds of sweat, grease and funk, and then set your action high, and I don't mean your high, I mean old school, serious high.
Pickups are one of the best and most cost effective upgrades around. So if you think you like it, go for it. You can always have the others put back in if you don't like it.