I used to get so pissed off by the same problem- being thrown off by really advanced drummers or those considering themselves to be... The latter is of course more irritating, but you better be damn sure of your time before you blame the drummer, I always told myself.
What I experienced, everything I practiced that touched on the subject "rhythm", made me more stable and secure over time. For example, I often practice subdividing very slow metronome clicks in different subdivisions, e.g. click is 50 per minute, so play 2 notes per click, 3, 4, 5, 6, 7, 8. Or practice 5 notes over the duration of 4 clicks, then take those 5 and consider them 1/4-notes- take their 1/8-notes (which is 5 notes over 2), their triplets (scary), etc. pp. Same with 7 over 4.
Or, on the other end of complicated, play walking lines in a form (rhythm changes, blues, giant steps, whatever) with metronome on 40. Play every click is a "one", then twice as fast a that, and so forth.
It really, to me, seems it does not matter what specifically you practice, as long you practice it thoroughly, your own sense of time will improve just like your ear will improve whenever you practice anything harmony- or melodywise.
In the last few years, my time has improved a lot. A few weeks ago, I was on a jam when a drummer showed up playing really "all over the place", to the point where I absolutely felt "I am SO not getting this!". The really good thing was, though, that keeping steady time and pulse in itself was not too much of a problem anymore (as compared to, say, 2 years ago, where this would have killed me, and consequently I would have killed the music), so I could listen analytically while playing, and try to figure it out. Eventually, I understood some aspects of what he was doing, which made it a little easier to play with it. (Because, practicing all this overlay/displacement/subdivision stuff, once you KNOW it, you can identify it) It still felt weird, but afterwards I felt a lot more like the solution than the problem, if you know what I mean.