Been messing around a bit more and thought I'd add a few more songs suitable for beginners.
"Where's Summer B?" by Ben Folds Five is rather basic, it's all quarter notes and shifts between simple patterns in first and second position. Plus (and aside from being a fan, this is the reason I was looking into their music) since it's piano/bass/drum it's very easy to clearly make out the bass line.
I prefer this
tab, but there's another one by Tim Swan that's floating around the Internet that I really ought to try out.
As well, the Pixies have a number of great songs with very simple basslines. "Gigantic" is just:
------------------|----------------------
------------------|----------------------
--------5-5-7-7-2|-2-2-2-3-3-5-5--------
-3-3-3------------|---------------3------
played as straight eighths at 110 bpm and repeated for the entire song (with the slight exception of the guitar break). Technically the shift to the 7th fret isn't necessary (you can play it all in 2nd position if you wish), but that's the way Kim Deal plays it.
"Where Is My Mind?" isn't really any harder, but is structured in a way that you're better off looking for some tabs.