I'm a "wood matters somewhat" guy, and I do think that solid rosewood to solid maple should be enough of a contrast to be noticeable. Still, it shouldn't have been an earth-shattering difference, assuming that the neck contour, neck construction, fret size, truss rod type, etc. were truly identical.
I'm wondering if you used feeler gauges to double-check your setup during the neck swap test.
In my own experiments, I've found that a mm or two difference in geometry (specifically, neck angle and fretboard height, action height and pickup adjustment), can make a big difference: enough to influence the voting patterns of myself and my friends, even when one of the "variations" is actually the same neck.
Yep, good points on the set up. They we're set up to the same action and both had the same clearance at he 12th fret and on both pickups. Both necks were made by Warmoth with the same fret wire and nut specs. The differences between the two were one was a rosewood/rosewood neck with no sealer and the other was a maple/maple neck that was blocked and bound and had a vintage tint.
I didn't buy the necks with the intent to compare the two. I was undecided on the color I wanted for the body and I was concerned about how neck heavy a rosewood 5 string neck would be so I bought a maple black bound and blocked neck as well as the rosewood.
I was under the impression at the time of purchase that the neck wood would not have any impact on the tone of the bass based on numerous strong opinions of a large portion of the TB community as well as some maple vs rosewood YouTube videos that indicated no difference.
All I can say is, I was wrong. In my personal first hand playing experience I found the rosewood neck to have a much more mellow tone, less attack, slower decay, more fundamental and overall richer more complex notes. The maple neck had a faster attack, with crisp notes, less low end and more high end - almost stingrayish.
I recorded open strings and a walking bass line with each neck and played the recordings for a number of my friends and relatives - some of which are bass players and some who aren't. Everyone as able to spot the different neck. Some preferred the bright crisp sound of the maple neck and some preferred the dark mellow sound of the rosewood neck. I liked both and decided to install the rosewood neck on the bass because it looked better with the vintage white body. I later decided to have brass frets and brass nut installed to make it a little brighter sounding, which it did.
With he exception of the necks, every thing was as identical as can possibly be expected. Strings, tuners, pickups, preamp, bridge, setup, strap locks, screws - you name it - we're unchanged. The necks were built to the same spec except for the woods and blocks/bindings and finish.