Originally Posted by unclebass
just take your time, once you go too far, there's no going back.
That's it right there. Though if it is anything but a brass nut, you can fill it back in with epoxy and re-file, though that takes a lot of patience and even more skill. It can be done though it's a lot easier to file it right the first time.
My bass has a brass nut that isn't permanently attached, so if I ever had this issue, I'd probably get a nut specifically for what I'm using it for so I can switch back and forth if I had to (though I'm currently using fat beams, and they don't get a whole lot thicker than that, so I'll probably never run into this)