I have know idea how long you have been playing. By the time I was your age I had been playing for 12 years (gigging for more than ten). I can tell you that for me it just came with time, and YEARS of playing (both practice and live). I go so long without looking at the fretboard that sometimes I forget which bass I'm playing! (reaching down for a knob that isn't there and whatnot) But really who cares? I can do it, but it's not ALL that cool. I don't watch other bassists to see if they are looking at the neck. If they fit in the pocket, and pay attention (musically) to what's going on around them, then I DIG It! I wouldn't stress too much over it. Some of the best (most famous) players just stare at the thing all night.
But to answer your question, I do boring finger stuff (scales and patterns) a LOT just to keep my chops up. Try this. Number your finger on your fret hand 1,2,3 and4 (index is 1, middle is 2, etc). Then just mix them up and go back and forth over the strings. Start off like this. Place your index finger on the 5th fret of your lowest string, then play four notes in a row using your four fingers.
1,2,3,4 E string
1,2,3,4 A string
D string etc.
Start with the 5th fret on each string. Then move to the 7th, the 9th, the 11th and so on until you have run the range of the fretboard. Then the FUN starts. Go back to the 5th fret and mix it up. This time do:
1,3,2,4 E string (notice I switched the order of the fingers)
1,3,2,4 A string yadayadayada (Hopefu;;u you get the picture)
Then do 1,4,3,2
Then 4,2,3,1 etc. etc.
But here's the kicker! Alternate your plucking fingers every other note! Period! If you have to start off slow to do this, so be it! (if you like a pick, no biggie, alternate up and down strokes every other note).
By the time you work a lot of combinations up and down (top to bottom AND left to right) you will probably have a cramp the first few times. But the point is you will KNOW your fretboard up and down top to bottom. Do this (or at least a few combinations) every other day for a couple of months and I guarentee you will look less and jam more!
P.S. I think the tech term for this stuff is "permutaions" so you can probably look up some better explanations online. Good luck! Sorry for the long response but I hope it helps. It really helped a bunch of my students (even though they HATED me for it...haha!)