Which gauge for BEAD?

If you’re already happy with a .130, the easiest thing is just stick with medium strings, like .105, .085 and .065. It’s basically a 5st medium set without the G.

Start there, and then you can adjust as necessary.

 

Stewart-MacDonald has bass specific nut files. Expensive but they'll last several lifetimes. Amazon sells some Stew-Mac items and may have what you need.
B.

 

Surely, assuming you are currrently tuned EADG, you already know what gauges you like for EAD? Just use those gauges.
What do you mean by 'a good tension'? Everyone's taste in tension is different so what others like is irrelevant.
You can judge this based on how you find your current EADG tensions. Raise or lower the gauge to raise or lower the tension.

 

A standard 5-string set without the G string?

Ironically, that is how the standard electric guitar 10-46 light gauge set came to be: electric guitar players would purchase a Mapes, Black Diamond, Gibson, or other set of "heavy" strings, 13-56, discard the 56, purchase a banjo d string, which was about a 10, and shift everything over a string. That is how Ernie Ball came about with the original commercially available Slinky set, because the 10-46 (and later other sets) named for how they felt compared to the standard 12's and 13's sets of the day.

 

Specialist nut files are really expensive, I think I paid well over £100 for a set of 4. For a one-off use you'll probably get away with a small diamond file, I got a cheap set in Aldi not long ago with various different profiles for less than a tenner, or chainsaw files would also do the job in a pinch. Either that or buy a new nut and get your local Luthier to file the slots for you, you then can use the old nut if you decide to go back to EADG.