These guys I'm jamming with are playing in Drop A, so I ordered a set of Circle K strings, the thickest being a .166. The issue is I have a Warwick Vampyre with a brass nut. Does anyone have any suggestions on how to file the brass to make this string fit?
Find a tech with the proper nut files. They are the proper width, and they are curved at the bottom to match the string. Doing it yourself is inviting problems - cutting it too far and getting 1st fret buzz and making it not fit the string right so it rattles in the nut. The brass nut will be expensive to replace if it gets done improperly.
Definitely go to a pro if you've got a Warwick Just-A-Nut, those are like $120 to replace if you screw it up.
I'd recommend having a tech install a new nut for the specific purpose of using a string of that gauge. So if and when you go back to standard tuning the original brass nut is not destroyed.