I think it is worth looking into modes because they help you to know what scale tones are available within the context of the chord you are playing. The key thing though is that you do still want to make sure that that you keep the chord tones at the center of your playing (See
Chord Tones Are Primary | Bass Chord Patterns | StudyBass for a good explanation of this. If you aren't already familiar.)
Like you I play in a church context and am just starting to look more into modes myself. Yes, you're extremely rarely going to have a bass solo, but as you say modes are useful for fills, which tastefully used can help support the harmony of the song, add to the momentum of the chord progression and support transitions between different sections of the song. As a simple example of this, for a I-IV transition (e.g. C to F when in the key of C) I might perform a litte run up of C D E F, i.e. a 1,2,3,4 run up the major scale. I find this natural to do because I'm familiar with the basic major scale. However for different chords I would need to use the a different scale in order to know what notes I'd be "allowed to use" for similar runs, and so learning the modes would help me out here. (Or rather, really it's still the major scale, just starting and finishing in a different position - but that's precisely what the modes are!)
So by all means learn the modes - there's a big difference knowing the scales and how you make use them. Sadly, a lot of discussion of the application of modes forgets the importance of chord tones.