it would help us if
you offered examples for reference.
(preferably a clip of you playing the song)
but in general when playing slow tempos try to feel the subdivision while executing the downbeat... for example playing a slow 4/4 could be the heard/felt as a slow 4, or 12/8 if focusing on the triple subdivision.
If you have a drummer, focus on the bass drum/snare (downbeat) and ride (subdivision)
moreover, if its duo (no drums) that complicates things more;
- How confident are you with the beat?
- Can you trust your that your partner is strong on the beat?
- Then there are players that don't "come to the center of the beat" but ALWAYS playing ahead (or behind) the beat.
there are many things to examine but the short answer is; you need to be solid
on the beat, and ignore fills and 'over-emphasizing' /stressing the beat. A slow tempo is a master class in control.
For SLOW TEMPO practice, try using an app to play along with a drum pattern:
- Start with working the slowest tempo you feel confident in. (maybe 50-60bpm?)
- play a few choruses to solidify that feel.
- now slow the tempo by 4-6bpm, repeat these steps till you reach your target.
once you get the feel of this exercise try moving to a simple metronome click.
go another step deeper and put the metronome on beats 2&4 —forcing you to properly place the downbeats.
all of this is process and not mastered in a day

For FILLS:
hard to say without an example. if you have 'beat 4' to play a fill and in a duo, I'd focus on the beat and let "the song happen", you can't force the music—especially something as delicate as a 20bpm groove.
You can also work with a teacher,
and don't discount studying with a drummer who specializes in the style you want to play. They will give you insight about the drum—bass connection you may not have thought of, as wells being able to play with a skilled drummer at least once a week.
hope to hear some examples for reference, further discussion.