Hi Mike, a fellow newbie here just a wee bit further down the road. A few things you mentioned were things I really affected me too, so here's what I found.
First: soreness/pain.
When I started last winter, I tried really hard to hold the left hand positions recommended by various books and skilled friends, practiced despite tension, and ended up with some tendonitis that made me take a few months off; a period with frequent applications of ice and reading up everything I could on left hand position (need the wrist in this position, thumb in that, Stu's hand looks like this, etc.). These various tips and positions I'd periodically try, with resulting despair since inevitably there would be some pain.
The video posted earlier "Developing Safe Left Hand Technique for Bass Guitar" (
Developing Safe Left Hand Technique for Bass Guitar - YouTube) really is great, so that's why I'm posting it again. It really resonated with what I know of biomechanics from piano and various sports - minimizing tension is a key to minimizing injury and maximizing fluidity, and simply finding the position where my left hand has the least tension and fitting the bass to that was quite eye opening (in my case, the neck was closer to vertical than most pictures, but nothing bizarre). Then, I started practicing with the mind not of "does my hand look right?" but rather "do my hand, fingers, arm, body feel right?" That made a HUGE difference for me.
For me, it turned out, the biggest killer was pressing too hard with my left thumb (a common beginner mistake), so I try to pay particular attention to that (sometimes playing without the thumb, sometimes wiggling it around or tapping while playing, trying to feel whether it's appropriately relaxed and only squeezing when and how it's needed to, turns out, a lot less force is needed than my muscles originally thought).
Paying attention to my tension and comfort really brought out other interesting learnings - for example my left hand is much more tense when I'm trying a new piece the first time at speed, and gets more relaxed once I've learned it. After noticing that, I was surprised to discover the same pattern in my piano playing -- something I'd never noticed for decades!
Other commonly discussed exercises are to practice pressing strings with the absolute minimum of force to train ourselves what it feels like. Once I was able to play with a relaxed hand and my body learned what that felt like, I found I was able to get a lot more fluid, and sounded a lot better with less effort.
Different folks will have different patterns of biomechanics and tension - focusing on listening to our body's quiet attempts to tell us what's not working and letting it guide us to what does is powerfully useful, and every body will lead a slightly different way.
Second: that pesky pinky finger.
When I was starting, my pinky went everywhere (and in a tense way too!), and I tried focusing on making it not fly around. That didn't work - no amount of telling that poor finger not to move in a bad way worked. Once I found a good set of material to practice where I was instead focusing on putting my fingers in the *right* place, along with trying to be aware of muscle tension, I forgot all about trying to restrain my poor pinky, and was quite astonished last week to notice that it was staying nicely down by the strings ready for action without me even thinking about it.
For me, that material was Ed Friedland's book "Bass Method" published by Hal Leonard. GREAT BOOK! Good material that gradually and strategically introduces new positions and movements in a series of exercises that pretty quickly become rather fun to play (especially if you get the version with the CDs), written by a great teacher who really pays attention to not overloading us newbs. I've also tried some video lessons (Stu Hamm's and Roy Vogt's are both great) - but found at the beginning that the self-induced pressure of keeping up with a video triggered me to be too tense and practice too long. Many threads here advocate beginners start off with 5-15 minutes a day for the first while -- and I think in hindsight their advice is quite right. I'll go back to Stu and Roy after finishing up this wonderful book from Ed.
Third: stretching and exercises.
An another recent thread, someone made the great observation that a ten year old has enough strength to press those strings; so what we need is flexibility and speed, not strength. Totally makes sense. I'll add to that the advice I got from my organ professor grandpa who taught me piano -- he had many stories about pianists in the early 20th century who got really into trying out various hand strengthening exercises and gadgets (guess it was the whole 'scientific modern age' thing) and ruined their hands. Our hands are very precise instruments, which we have a huge amount of practice using on a day to day basis -- they're actually pretty strong and flexible already. The focus should be on learning how to fluidly and precisely apply the strength and flexibility we already have, rather than on building up more.
So, stretching on the fretboard might not mean the same thing as when we're stretching out our legs while working out -- we don't need to *stretch* our hands out to new awkward positions in an attempt to meet some goal of making it from some fret to another (there lies the path to tension and injury), rather, we can put the hand in some particular position (or in motion switching between positions) and learn how to, with minimal tension and force, have our fingers do what we want while in it.
Similarly, let's look at the stress squeeze ball PerlNinja recommended -- many like to use those to strengthen the grip (or even recommend to us beginners that we do) -- but note he didn't say that and recommends using it to help relax. A really interesting distinction. We can use them to practice relaxing and being conscious of *how* it feels when we press this finger that way, relax that finger that way, how it differs to press down from the finger tip vs. the base of the finger, etc.
Finally: bass setup
I've read a lot of stories on this board from people who started off with basses that had bad setups, and never new for years that they were playing something with bad actions, tension, or intonation. Know way of knowing how your bass is set up, but it probably can't hurt to have a pro look at it to ensure there's nothing silly that might make you have to work harder than need to, and that the intonation is right so that everything is in tune across the board.
Don't know if any of this is useful, or if even most of it is correct, but they are things I've been gradually thinking through as a result of being in the same situation you're in, and maybe they (and any helpful subsequent critiques of them by those more knowledgable than I) can help.
Keep having fun! It's a blast, I'm really glad I started.
Get one that's comfortable. After all you will be supporting the weight of the bass on it; I had a narrow strap myself and after 30 minutes that sucker starts digging into your shoulder pretty badly, so got a wide one and it's comfortable now. 
