Having lived in South Jersey, and Jersey just being one of those states where you have to take the Garden state to get anywhere, I wouldn't do it. Are you in South Jersey? Because sadly in that area, to get anywhere interesting (Ignoring Philly of course which is cool!) you have to do hours long drives and North is where a lot of the cool stuff is.
And if not and the band is just far, I feel sad saying that I agree with others on here I wouldn't do it. It sucks I been in those shoes where you wanna do something but it is a long commute away, I mean some hobbies yes I seen people commute that long, but it would suck for you have an added stress of driving for something you are looking forward to. If you are taking the Garden State all that way, while that road is hated isn't the worse, (Beats most main roads in NY let me tell you that imo!)
I always tried to say, when we lose an opportunity a better one will always pop up... but if you do go through with it, stay safe and look on the positives!
Also wanna add follow your heart and do what you feel. Everyone is just giving an opinion and view. Sometimes doing the crazy is rewarding and other times finding a new adventure is just as rewarding... best of luck again!