To be honest...you don't look like you are trying to hard. I don't know you, nor have I seen you on stage...but your audience does and you are getting feedback that you can up your game.
When I see a band in shorts and flip flops - they don't give a poopie, and therefore earn the criticism...