Bridge saddles too high & fret buzz struggles

Hey all, I am trying to troubleshoot a few issues on my Jazz bass with a Hipshot Kickass bridge, and nothing I have tried has worked. Here are my current issues:

1. To eliminate fret buzz completely, the bridge saddles have to be raised to the point where they no longer sit in the saddle guides. As it's pictured below, I am still getting fret buzz the whole way down the neck. They do ring out, but buzz for a second after plucking at any fret.
   
   
2. Once I raise the saddles enough to eliminate buzz on frets 1-12, the buzz persists on 12-20. Once I have raised the saddles and ALL fret buzz is gone, the D string still has buzz from frets 12-20. It has to be raised significantly higher than the A string. (As it's pictured, I'd have to turn the D saddle screws two turns higher to eliminate buzz compared to the A string).

Current action at the 20th fret with no strings depressed is between 1.5 and 1.75mm (0.060 to 0.068 inches). Relief at the 8th fret is about one business card.

Can anyone give me some guidance here? Is there something obvious I'm missing that I can adjust to bring the saddle height down, keep low action, and not have an excess neck relief? I can take as many pictures and measurements as y'all need. Thanks in advance!

Edit: I am trying to find a straight edge to see if the frets lie even across the board, and if there are any tall ones. But since it happens the whole way down the neck, I'm not inclined to believe there are any.

 

No shims - I considered trying one but wasn't sure if that would solve or just perpetuate the issue.

 

Perfect, I'll try out the neck shim first. I replaced the original bridge with the Hipshot because it was taller, but that didn't really seem to resolve any issues. I'm hoping I don't have to raise the bridge itself.

For relief, I put a capo at the first fret and press where the neck meets body (17th fret I think). I've always set relief to about the thickness of a business card on all my basses, but I don't mind adding a little more here if needed.

 

@JKos and @Zooberwerx thank you both for the help. I stuck a shim in the headstock end of the neck pocket, which brought the saddles WAY down and got rid of the 12-20 buzz. Added a little more relief, threw on new strings, and set my witness points and now it plays like a dream! I really appreciate the advice, thanks!