Hey Lazy,
Welcome to the wonderful world of bass. Glad to see you converted!

I'm not going to sweat you about thinking that the bass was "easier." A lot of people think this. My dad still does. At least a little bit.
Quote:
| My question is, is the solution to my problem going to come from more listening to others, fretboard memory, or more understanding of musical theory? |
Well, all those things that you described are going to help you get better. You can't downplay, I believe, the importance of listening to others, whether it be recordings or live performances. Art comes from inspiration, and often that inspiration is by a fellow artist. Not only listening though, but if you find a particular passage that you like,
transcribe it. Once you transcribe it, you can analyze what this or that person did. Besides which, you're also improving your ear. Take it a step further though, transcribe what the bass player is doing, but analyze against what the rest of the band is doing. Heck, transcribe the drum pattern, so you see how the bass is playing along with the bass drum, snare, or cymbals. See how it interacts with other instrumentalists, and also with the vocalists.
Fretboard memory certainly can't help, but I would probably encourage more of an aural fretboard memory, than a visual or kinesthetic one. In other words, it's important than your fingers will take you where your head or ears hear that dominant 7th, or that b3. Scales scales scales scales scales scales scales.
Theory will certainly help you wrap up all this information into a tight little package, but I very much believe, the most critical part to your development right now, is to
GET A TEACHER!
A teacher can show you if your right hand or left hand technique is correct. They can help organize your practice routine. They can provide an objective ear to what you're playing. They can provide encouragement. They can explain things to suit your learning abilities.
Also,
record your rehearsals and lessons. The tape doesn't lie. You'll hear how you sound to others, which often can be an enlightening experience.
These things take time and money. It's not going to come quickly, but the pursuit is a lot of fun and the rewards self-evident.
Either way, good luck.