First (position) fret buzz

[blipndub]

Old workhorse warmoth pbass. Banjo frets, never had a problem in about 8 years, suddenly I'm getting fret out at the first position on the D string. I packed the nut slot with baking powder and CA raising the string noticeably but the buzz is still there, not a little bit either but a note killing kind of buzz, happens on the G string too, but not as bad. Relief is medium-flat and I've raised the saddle a little bit too. Thinking of increasing relief but it's a warmoth neck so it's at P.I.T.A with the nut at the heel.

Kind of at a loss, any suggestions?

[blipndub]

I polished up the frets before putting on a new set of strings (fretted out with old strings too) using steel wool and a Gordomite cloth, but i do see a little fret wear, wonder if I need to take it to the leveling level although i'm hesitant because they are tiny vintage steel frets.

[blipndub]

Update, filed down the first three frets and i seem to be getting closer. Will let it settle for the night and see what the dawn brings!

[JLS]

It's going to be more of a PITA, if you keep going after the frets, and don't do what you know you should have done, which is try a little bit more relief in the neck.

"Suddenly", fret out? Sounds likes seasonal change, to me. And that would be a trussrod issue.