Yep, some buzz on one coil's mags is normal, but the other coil should be silent.
If you notice a little buzz when you touch one coil, but a louder buzz on the other coil, then reversing the phase of the pickup should leave you with one coil that buzzes a little, and the other that is silent. This is because the coil that is silent now has its inner windings at earth.
The trick then is to swap the positions of the d/g bobbin with the e/a bobbin and vice versa, so that your thumb only touches the silent bobbin.
This all sounds a bit complex, but really, its very simple, especially since you can solder.
There's three other ways you can overcome this.
you can paint a little clear nail polish or super glue or something on the tops of the magnets.
You can replace the pickup covers with ones that have no holes. You'll have to add shims next to the magnets on the tops of the bobbins so the cover won't distort.
and, thirdly you can earth the magnets. This is usually done with copper tape that has conductive adhesive, underneath if the mags are alnico, or on top if you don't fancy prying off the ceramic mags then replacing them. FWIW, if you earth the magnets on a vintage fender pickup you can often short it and stop it from working. These old pickups often have corroded magnets and the inner windings will be short to them. No biggy though, as a pickup can keep working for many years like this before needing a rewind, so just go with one of the other options.