Just to be sure we're discussing the same thing, I'm assuming that you're asking about the Roto TruBass 88's and the Fender 9120's. I've used both on fretless.
To me, the Rotos are so smooth, they actually feel like they are dipped rather than wrapped in nylon. The Fenders are more like rounds in my book. The Fender wraps are quite rounded and are very visible and noticeable to my fingers (but quiet, unlike many metal rounds). The Fenders have much more "snap" than the Rotos, and overall, the tone of the Fenders is much better from a broad sonic range perspective.
The Rotos provide more of what many bass players think of when they consider flats. The Fenders are much more aggressive. Both are good strings, but for different applications.
Fretlessguy is correct about the need for compression/EQ. Both of these sets will absolutely kill in the low end and tend to get weak in the D and G strings (the Fender set perhaps a bit moreso). I always use a small amount of compression, and that helps, but I also adjust my pickups (mainly the neck pickup) to slant substantially upward from the E side to the G side to even out the response.
Edited to add: I'm actually having adjustable-pole diMarzios installed in my CIJ Fender Jazz fretless as we speak to aid in this balancing (this bass has the 9120's on it). In the past, I've used the Rotos on this same bass. I also have the 9120's on a fretted Jaguar and the edgy tone is just ridiculous.