Hello Old Fart;
What you are doing is the classic method of checking the nut slot height; pressing the string down on top of the 2nd fret and checking the clearance between the underside of the string and the 1st fret. A better way to measure using that same technique is to use a thin 6" steel ruler up on edge. Set the edge of the ruler in the nut slot and on the 2nd fret. Then check the gap between the ruler and the 1st fret with a feeler gauge. You can get a better reading because the ruler won't flex like the string.
Using that method of measuring, you should cut the nut slot depth to get a gap of 0.003" to 0.004", under the ruler at the 1st fret. That's the standard number for electric basses, regardless of action height, string gauge or string type. Lower than 0.003" and the string will buzz too easily when played open. Higher than 0.004" and it will become stiffer to press the string down at the 1st than at the 2nd and beyond. That range of 0.003" to 0.004" puts the 1st fret pressure about equal to the others.
If you've read through some of the many threads around here about setting nut slots, and height of Zero frets, some of us will talk about a height of 0.010", the height of the nut slot (or Zero fret) above the 1st fret. But don't get confused, because that's using a different measuring method. Using a longer straightedge set on top of all the frets, the nut slot should be 0.010" higher than the bottom of the straightedge.
That 0.010" number gets you to the same place as the 0.003"-0.004" number, just measured in a different way. The 0.010" number applies to a straight line across the tops of all the frets, while the 0.003" - 0.004" number is for the gap at the 1st when measuring with the short tilted ruler touching the 2nd. Don't get confused by those two numbers. It's a common point of argument in any thread about nut slots.
The reason why some of us use that 0.010" number is because of a Secret Pro Trick method for filing nut slots quickly and accurately:
- Measure the height of the 1st fret above the fingerboard surface, after the frets have been leveled. Say it's 0.035". You want the bottoms of the nut slots to be exactly 0.010" higher than that, which would be 0.045" above the fingerboard surface.
- Find several small rectangles of thin aluminum sheet, around 2" x 1/2", which measure 0.045" thick total when stacked up. Hand bend them slightly to approximately fit the radius of the fingerboard.
- Set the stack of aluminum shims on the fingerboard, right up against the nut. Hold them there.
- File the nut slot, holding the file so the vertical angle is aiming toward the tuner post, creating the break angle in the slot.
- File the slot down until the file just touches the edge of the aluminum shim stack. You see a tiny shiny spot at the edge against the nut. That point sets the front edge of the bottom of the slot at 0.045" above the fingerboard and 0.010" above the 1st fret. It will also measure 0.003"-0.004" at the 1st using the tilted ruler method.
- Don't use steel feeler gauges up against the nut for this, because they will dull your precious nut files. Use aluminum or brass for the shim stack.
That's the pro trick for filing all the slots accurately without all the trial and measure stuff with the feeler gauges.
When I'm trimming the height of Zero frets or fretless Zero Blocks, I use a special tool that I made, a dial indicator on a beam. That's also where that 0.010" number comes in.
View attachment 7045965
View attachment 7045967