It is a floppy E string that is too loose and slapping either the top of the pickup or slapping the fret at the end of the fretboard. A bandaid is lowering your pickup if it is the string hitting it. raising you string height will work if it the fretboard.
You can also play around with the EQ right around the 400 hz range, should get rid of some of it.
But the real problem is your string tension is to low/ not tight enough. You are asking a string never designed for that frequency to go down to D. Only way to do that is to detune/ detension the string. Thus the flop and clack.
You can either buy a set of strings for a five string, detune the entire instrument to BEAD, and hope that your four string neck can handle the tension. Or you can go ahead and make the jump to five string, that you are already on the path to doing with you drop D tuning.
That is the route I would/ have already gone. You would then have your D on it's own string, plus a couple lower notes. Be a lot easier to play a low E at the 5th fret of the B string, and sound better than an open E.
Had similar problem many years ago, switched to 5/6/7 string basses, was still annoyed by the clack noise. I have a very heavy finger style, fast and hard. lol. But I eventually got rid of my fret noise completely, by getting rid of the frets. lol.
I have been playing heavy prog rock fretless for many years, nothing sounds better in my opinion.
Hope this helps you find your way to better sound.