I play almost exclusively in places where tables have to be moved to create "the stage." I'm mildly irked when there are still diners/drinkers at the table when we need to set up, but I can live with that. In these situations, the goal is "make the bar/restaurant money through sales" - these are paying customers and they win the priority-battle.
BUT - when they've paid the check, are done and still sit there, loitering for another hour (and past our start time for playing), we've blown past "mild irritation."
"Oh, a band! When do you start playing?"
"As soon as y'all clear the table so we can set up."
I've only skirted the edge of rudeness once, but I can get passive-aggressive with wheeling the gear in and waiting nearby.