One thing I have learned over the years is that not everyone has a recording device implanted in their brain. In fact, few people do. I can sit with my drummer and no one else, count off a song, and we will both hit every section in the correct place with every fill where it's supposed to be, because he and I are both hearing the song in our heads as we play along. But the VAST majority of players I have worked with cannot seem to do this and need an anchor to the song structure, such that if we're not rehearsing with our vocalist, for example, someone needs to sing a scratch vocal to keep everyone on track.
I don't know if this is a learnable skill or not; I've always had it. So I'd like to hear from someone who has developed this ability over time because it would give me some reassurance that it should be added to a list of goals. Otherwise it seems like it could be a bit of unobtainium that might be discouraging.