Man, this is a tough question and I'm the wrong guy to be answering it. But I'll take a stab, especially since I have a fretless 5 on order with The Perfect Bass.
I'm of the opinion that everyone should at least give fretless a whirl - and a good try. Not just a few minutes, but a few months dedicated to it to see what it does, or doesn't, do for you. Of course, you're not going to want to start with a $1k or $2k+ bass for something you may not stick with... so I'd have to advocate used or low end to start.
Personally, once I picked up a fretless, I knew I had to have one. Not to replace fretted playing, but to augment and have options. I'm also playing electric upright (and soon, double bass), so having no speed bumps feels pretty natural at this point. But, a new person to it is going to have issues - the biggest likely being terrible intonation. Practice will get you over that.
I wish more places would stock fretless, but man is it a hard sell. The overwhelming vast majority of bassists want fretted. That's fine. It just makes it hard to go to a store to try fretless. Stores can't afford to have basses sit there for long periods of time.
Also, I'd shy away from making two big changes - to 6 string & fretless at the same time. You're signing up for double trouble. It might not be too much of a jump for you, but it's adding a lot of new factors at the same time.
Anyway, I don't think I helped, but hopefully gave you something to think about.