spade, you're not a bad looking guy. Quit stressing about your looks. From a woman's POV, we're far less hung up on your looks than you think we are. Make us laugh. Treat us with respect. Listen when we talk. Talk to us about yourself. Be confident. Those are HUGE to us.
In response to your comment above about bigger girls not being keen on you, as a bigger girl, I will tell you that IME, it's not how a guy looks that makes us get all defensive about our self-image, it's what he says and the way he looks at us. If your discussions with them are all about your diet, your exercise, your weight, your looks; you're sending the message that appearance is what's important to you.
I've been on dates with guys that ended with me feeling like a million bucks despite the fact that I was obese and I've been on dates with guys who left me feeling like I didn't deserve to live because my fingernail polish was smudged. Neither of these experiences had a thing to do with what the guy looked like. It was all about what he said, how he said it, and the look on his face when he said it.