It takes time, there's no way around that. But if you work on it consistently each day, even if only for 10 or 15 minutes minimum, and find a way to make it fun and not over analyze how hard it seems or how much progress you are making or how smart you think you are, it will work itself out.
Going through arpeggios of harmonized major scales as 7th chords in all inversion in all keys is a really good way to internalize this and lead into other areas of theory... but it takes a certain amount of familiarity to be able to do this.
1. Memorize "BEADGC." BEADGC is your friend. The four open strings are right in the middle--"EADG." If you had a string below your open E, it would be B. If you had a string above your open G, it would be a C. Now you now the open strings on a 6-er, if you ever get one!
Better yet, make that BEADGCF.
Not only does this ascending series of fourths (or descending series of fifths) correspond to the (extended) open tuning of the bass, it also happens to be the order that flats are introduced in keys that use flats, and in reverse, the order that sharps are introduced in keys that have sharps.
So, the key that has 3 flats has Bb, Eb, and Ab, with the rest being natural. The key that has two sharps has F# and C# with the rest of the notes natural... etc.
You can also use this information subtractively, because once you are dealing with 4 or more accidentals in a key, it can be simpler or advantageous to think of the notes that are natural. So for example, in the key with 5 sharps (the key of B), only the B and E are natural, because they are the last two in the series when you start in the top as you do for keys with sharps.
I use somewhat related mnemonic devices for the keys... in that same series, if you start at the key of C which has no sharps or flats and look at it as if it were an open string, going down from there are strings that correspond to the keys that have sharp notes. when you get down to B you start going up in fifths instead of continuing to go down in fourths (unless you prefer to think of descending further in fourths). G D A E B F# C# are the keys with sharps, from 1 to 7 in that order.
For the keys with flats, visualize the B E A D G C F series as if it were a fingerboard where you are going across it at the first fret. You start with C that has no sharps or flats... then F Bb Eb Ab Db Gb Cb (the last of which you'll have to imagine at the 6th fret of the F string) which is the order of keys with flats from 1 to 7.
Using these ideas and the harmonized scales as 7th chords in all keys, you will gradually eradicate any preference or difficulties that make some keys easier or harder than others, and you'll get rid of any mental blocks for the enharmonic equivalents which cause some players to have to slow down and think (or just use the wrong name) when they encounter an E# or a Cb or whatever.