Random listeners ( not "experts"who might claim to hear a difference, online listening.Correct, the null hypothesis is that guitars made from "traditional" woods do not sound different from guitars made from non-traditional woods. They used a valid basic experimental design. The problem I see is that the particulars of the methodology are messy. Both the traditional and non-traditional groups used different combinations of woods. The intra vs. inter group variation is not parsed out (i.e., were the sonic differences between the various traditional guitars or between the various non-traditional guitars more or less than the differences between tradition and nontraditional?). Also the listeners were random people recruited via social media who responded via survey online - not exactly a well controlled test group. I'm just not sure how much you can take away from this study.
Two major issues, + the group variation mentioned above.
Agree. Not much can be taken from that exercise. Perhaps comparing the accuracy rate of random people vs experts might provide some interesting tone-cannon fodder
I didn't see any p values( did not look hard) but the validity of the 0.05 p has come into question recently and will probably change before my next "competency assessment" at work, shattering decades of statistically insignificant knowledge