Try playing downstrokes with the back of your index fingernail, supported by your thumb. Gets you real close and it's surprising how quick you can play using just downstrokes with a bit of practice.
For faster down and up strokes you can use this technique for the down strokes and then the very tip of your index finger for the up. You lose a little bit of the "pick" tone on the upstrokes, but at that speed it's not that noticeable and in a mix hardly anyone would hear much difference between doing that and using a pick.
I do both and find no difference with speed. I do find that I get more of a pick sound with up stroke. The trick is keeping the nails shaped right and adjusting your wrist to get what sound you need for that moment. But this is 30 years of doing it this way too. I have had ones come up and ask how I got the sound because they expect a pick but just the nails doing the work. Using three fingers with nails I know I can out pace a picker for speed since I can cover more rel-estate at one time. Trick also is not beating the crap out of the strings, just catch them with the tips.
And as a proficient self-proclaimed "bassist", you should be able to do both. It just makes you less well-rounded than everyone else.