You should also determine if the drummer can't consistently keep the initial tempo OR if they are pushing the song because they think it needs to go faster.
BTW...99% of us think we have great time.
I think I have horrible time & a horrible sense of pitch, which is why I put effort into these things. This is the kind of insecurity you develop when you play with musicians who are better than you are... But then, like the Karate Kid, one day you see someone who really has bad time or a bad sense of pitch and you realize that you've actually been learning from them all this time. (Though I still can't figure out why my music teacher thinks vacuuming his car is essential for my skill progression...)
Make your drummer get one of these. Just be like "can I borrow $300?" and instead of paying him back, hand him this box. If he won't lend you $300, then borrow $20 and get him a $20 metronome.♪
♪ note: this is a joke, I take no responsibility for whether or not you get kicked in the nuts after doing this.
I'm an OLD guy with thousands of gigs and sessions behind me, and I can tell you one thing I've learned: Bandmates CANNOT (in 99% of cases) improve the performance/ability of ANY other musician in any band. Pride may be a sin, but NONE of us - playing ANY instrument - is totally down with being told by bandmates that play a DIFFERENT instrument, that we're not playing ours correctly.
It's a stone fact. Do your arm hairs not prickle when the guitarist tries to correct your bass playing? Don't you want to shove the mic up his presumptuous ass when the singer tells you you're not playing the part right? No drummer wants to hear from the bass player that his meter sucks. I've told many drummers this, (and WANTED to tell many more), but by then I'd learned: You can't improve a bandmates' performance. If you can't work with your drummer, tell your other bandmates and try to get them to agree to look for another guy.
On the other hand, if your drummer is basically pretty good and you get along great, then maybe a few more months of practice will kinda 'teach' you where he speeds up and slows down. For instance, I always KNOW that my drummer speeds up whenever he takes an occasional lead vocal, so I can anticipate that. It made life in my band a lot easier.
+1. Except for the being an old guy bit. Stop being an uptight human metronome and either learn to love it, or find someone else who's also an uptight human metronome.♪
♪ note: if you find being called an uptight human metronome insulting, just imagine how he would feel being told his sense of rhythm sucks by an uptight human metronome.