This is something I also have a hard time with. I've got a bad habit of speeding up when I make a mistake, but I'm making mistakes because I'm trying to go slow...
What seems to work best for me is thinking "horizontally" instead of hyper-focusing on the downbeat. If I try to sing out the line...