When the music calls for it. Which is the same reason to "transition" to a 6 string or a fretless. If you are into "old-school" rock you may never need to "transition". I personally find slappy funk easier on a 4 but some people play it just fine on a 5.
In my case I musically like to look forward rather than backward. So after playing 4s forever, I started to see the need for low synth-style notes, for eliminating the hassle of playing in horn keys (Eb), and some other points. At the time I bought my first 5er I didn't realize the advantage of the extra range at one spot on the neck.
Soon after I got a 5 I bought a 6 and that was what really opened my eyes. A 6 forces you to do it all correctly. You have to damp strings, you have to have a correct hand position to play the super wide neck etc. once you get that down, a 5er is a snap. Plus the 6 really shows you the range in one spot thing.
So now I almost never play my 4s (except for the 8 (doubled) string). Usually it's a 5 unless I need the extended high end range (solos, chords, double stops etc.) Which I play makes little difference to me except the 5s are a bit lighter than the 6s and the look of a 5 flies better under the radar for people who get upset about that sort of thing. But I'm not into the kind of music that needs a 4 string.
So I say look to the music you are trying to play. What do the bassists who play that style use? If they use 5 or 6 strings watch what they do with them and ask if you might want to do that (or something like it) too? And that will tell you if it's time to get another instrument.
