I've been working through resolving similar issues on my fretless.
It's seems to come down to a combination of right hand floating thumb and general left hand technique.
Though floating thumb helps for ascending across the neck, coming back down is really all about the left hand.
I've found that how I release/transition to the next note is what matters. Remembering that the fingerboard is not a keyboard. So keeping those fingers as close to the strings as I can. Additionally right hand attack (especially the E string on my bass) is a big factor.
I can't imagine why these lessons wouldn't be applicable to a fretted bass as well.
Like an earlier poster mentioned , you just get a feel for it. The biggest part is first recognizing the things you don't like and accepting that it needs to change. That's a big one I thing as I think a lot of players either don't realize or just live with aspects of their technique that could improve.
Rob Allen Mouse 30, Acoustic Image Coda R Series II
Mac Book Pro > Roland Quad Capture