I love fretboard nerdery! I have examples of shapes and visuals in my TED Talk, so I won't go into all that here, but a few thoughts:
Knowing where every single instance of each note is located is SUPER important. You should be able to close your eyes, grab a random 4-fret span, and know how many instances of each note are within reach. It's important to recognize the difference between enharmonic notes and actual different pitches, for example, the E string at the 12th fret and the D string at the 14th fret are two different pitches of E, while E at the 12th fret and the A string at the 7th fret are both E but produce the same pitch.
Next, mapping out your fretboard into intervals. Sure, you know there's a C at the 10th fret on your D string, but if you play it with your first finger, how many major thirds can you access without shifting? What if you do the same exercise playing it with your pinky? What if you allow yourself a single shift up or down, how many major thirds can you reach? Repeat this for every interval.
Since the bass is all patterns, every interval has a pattern that repeats everywhere on the instrument. Up one string and up two frets from a given note gives you a perfect fifth, going down one string on the same fret gives a perfect fifth. Up 3 strings (from E to G) and four frets also gives you a perfect fifth. Recognizing these patterns up and down is important, some bass players only work on ascending intervals and their walking lines really suffer as a result.
To really open up the entire fretboard I find it very useful to practice shifting exercises. For example, I might play my major scale in C starting with my 2nd finger on the E string 8th fret, play up to the fifth (with the pinky) and then shift my entire hand up a whole step and continue the scale. This allows me to easily reach the 12th fret on G, the fifth, without having to shift up and down at the very end when I run out of space. The next question, which is really the big one, is: If I want to extend this scale, from the 8th fret on E to the 2nd octave root on the G string 17th fret, what is every possible shift I could make to get me there? Sure, I could wait until I run out of fingers and play the last 4 or 5 notes with my pinky on the G string, but there are way better ways that allow me to play fluidly and incorporate more interesting ideas like arpeggios or sequences.
Mastering the fretboard is knowing your notes, intervals, scales, and having systems for shifting fluidly so you can access any note at any time in an efficient manner.
I've had every possible reason to use these tools: broken strings, songs transposed on the fly (sometimes without any advance notice), reading charts, playing over changes where you can't easily rely on basic patterns, etc. If I'm reading a chart, or watching for cues, or watching a guitarist or keyboardist's hands to see what the heck the chords are, I can't spare any brain power to find my own notes or tell my fingers where they are. It has to be automatic.