Help Me Design My Next Test!

[mikedg18]

Agathis can't be any more crap than the random chunk of pine(?) that he used in his previous experiment!

Every time people mention Agathis is to say "it's worse than plywood".
Plus I'd like to see how a cheap bass, doesn't necessarily have to be Agathis, compares with an expensive one when they have the same pickups/electronics, something like the ash/alder thread but with a cheaper model.

As for his piece of pine, unfortunately we can't buy that (you know what I mean) so I'd rather it was an existing model.

 

[Headless Llama]

How about you test the same exact bass. But for one sample you eat a snickers bar first. Then we can try to decide which one is which.

How about for one sample you can play a Jazz bass and see how many people catch on.

 

[Bongolation]

Also, I tend to prefer "real world" samples where the bass is a part of a mix, but what do you think? Etc.

From a scientific standpoint, it's meaningless.

Why are you wasting your time and ours?

Hey, you asked.

You want a meaningful test protocol, you eliminate all extraneous variables.

If you are doing comparisons, you record direct in, no amp, no effect, no EQ, no band, no VSTs -- nothing but raw signal with bass controls nulled.

Any other way is corrupt and pointless.

 

[GlennW]

If you are doing comparisons, you record direct in, no amp, no effect, no EQ, no band, no VSTs -- nothing but raw signal with bass controls nulled.

I can't stand the sound of a bass going straight through the jack without a cap...Clackety City. I'd rather hear just the bass with cap or cap/pot, and have it adjusted the same for all tests. More of a practical comparison imo.

 

[GlennW]

Every time people mention Agathis is to say "it's worse than plywood".

Anyone who says that doesn't know a gd thing about agathis or plywood. With all due respect, IMO, ymmv, etc.

 

[GBassNorth]

I think it's important to first nail down very specifically the question you're trying to answer. Are you trying to determine if there exists a difference between woods? Or if people can detect a difference between these samples? Those are two very different questions. Or is it something completely different?

You might start even a step back from that by defining the tonewood hypothesis very specifically first. Then devise your questions, and only then design your test.

BINGO!
Dan, this is THE most important step in structuring any kind of meaningful test. Clearly articulate your hypothesis first. Then set out to prove or disprove it with test protocols that isolate as many of the variables as possible or in this case as your limited resources (being the two test basses) allow. I think you'll find that the results obtained from a well thought out experiment will stand up to any reasonable critic. Until you've clearly stated your hypothesis we're all just shooting in the dark suggesting test methods. The only other general comment I'd add at this point is - keep it simple, there are enough variables at play already, adding any additional ones only undermines the credibility of the conclusion.

 

[GBassNorth]

From a scientific standpoint, it's meaningless.

Why are you wasting your time and ours?

Hey, you asked.

You want a meaningful test protocol, you eliminate all extraneous variables.

If you are doing comparisons, you record direct in, no amp, no effect, no EQ, no band, no VSTs -- nothing but raw signal with bass controls nulled.

Any other way is corrupt and pointless.

Also spot on.
Eliminate all reasonable variables - we dont have the resources of NASA so we'll never get rid of all the variables in this type of test - but try hard not to add any additional variables that don't have to be there. As entertaining as the poll process is, or as "real world" as a bass in the mix idea sounds they do add significant variables to the test and may not actually help the discussion at all depending on your hypothesis.

 

[arsie]

It depends on whether you want to continue with the social experiment aspect of it. Which is quite hard, considering that Poll results and forum posts are open to be seen/read. But where would the fun be in that?

None of the critics of your tests have stepped up with their own tests based on what they consider more scientific methods.

My only suggestion is to have one additional person playing the basses/configurations. So we get to hear a Dan Atkinson played version of the choices, and another person's played version.

 

[-=DanAtkinson=-]

I think it's important to first nail down very specifically the question you're trying to answer. Are you trying to determine if there exists a difference between woods? Or if people can detect a difference between these samples? Those are two very different questions. Or is it something completely different?

You might start even a step back from that by defining the tonewood hypothesis very specifically first. Then devise your questions, and only then design your test.

Good point.

Let's put it this way: I'm 99% confident there exists a difference in tone between these woods. What I'd like to determine is, how significant is it? What percentage of us can hear it?

There may be such a subtle difference in tone that the human ear can't identify it. But that type of tone variation is completely useless, unless you're playing for an audience of androids.

But if the tone is drastically different, we'll learn a bit about what to expect should we opt to buy or build a bass from any of the same wood combinations.

 

[GBassNorth]

Good point.

Let's put it this way: I'm 99% confident there exists a difference in tone between these woods. What I'd like to determine is, how significant is it? What percentage of us can hear it?

There may be such a subtle difference in tone that the human ear can't identify it. But that type of tone variation is completely useless, unless you're playing for an audience of androids.

But if the tone is drastically different, we'll learn a bit about what to expect should we opt to buy or build a bass from any of the same wood combinations.

Dan,
You have the beginings of a great hypothesis. If it were stated this way does it get at what you're trying to address:

Wood types in a bass build effect tone to varying degrees, I hypothesize that the degree of tonal impact attributed to the wood selection is statistically insignificant to a listener and becomes even more so when the bass is played in a mix.

Not trying to put words in your mouth, just trying to help. Is this close to where you wanted to go?

 

[audiomitch]

How about another material than wood? Metal, paper mache, or ?

 

[ehque]

Hypothesis: the tone is in the fingers / player.

To support or counter statements like:"they all sound like dan".

Method: same bass, same strings.
Get a bassist friend, and record 3 tracks between the 2 of you. Use the same track or a metronome. Equalise levels and post.

The poll should have 3 options:

Track A is different
Track B is different
Track C is different

 

[DTF]

push back cloth wire vs normal plastic insulation wire

 

[knucklehead G]

How about another material than wood? Metal, paper mache, or ?

If you're up for some screwing / unscrewing, it would be fun to see the effect of the necks both on the basses you mentioned, as well as a third one..


..with a body of MDF. Its easy to work with, won't hold up long but you should be able to tune it up and play a bit. Hope you don't sweat while you play.

 

[Jeffrey Wash]

Dan - My hat's off to you for reaching out for ideas on this - it certainly helps counter the notion that you are pushing some sort of "agenda".

I would add my voice to those who are asking for sustained notes and perhaps a clip that encompasses a wider pitch/dynamic range. The clips in the previous experiment only contained 5 bass notes. I understand wanting to keep the experiment simple and I also understand that very often those 5 good notes are all that's expected of a bassist in many situations, (and I don't mean anything negative by that at all - I love a simple groove with a great feel). But more to the point: I'm sure that I could walk into any music store and grab any bass off a hook and lay a similar sounding track. (And maybe that was the point you were trying to make?)

For myself, being a full time "doing it for a living bassist", the proof of the pudding is always in how an instrument resonates and responds to dynamics when I play it. If it doesn't pass the "feel" test - resonance and dynamic response - then I won't bother asking the guy at the music store for a cable to plug it in. I've always viewed the electronics and hardware to be merely ancillary.

Over the years I have come to associate certain types of resonance/response with certain types of wood. But just about the time that I think I've got it all figured out, that's usually about the time I happen to pick up a bass that completely contradicts my preconceptions! And that unpredictability is one of the things I enjoy about playing music.

The point you made in the last thread about tone originating with the player rather than the instrument - that's the thing that really resonates with me (and many others I suspect).

Whether it's a $200 Fender Squire or a $4000 Sadowsky, it seems to me that what's required is for the bassist to adapt his or her technique in order to bring out the best in that particular instrument. I only wish it hadn't taken me so many years to figure that out (but that, perhaps, is for another thread...)

 

[Kwesi]

I can't stand the sound of a bass going straight through the jack without a cap...Clackety City. I'd rather hear just the bass with cap or cap/pot, and have it adjusted the same for all tests. More of a practical comparison imo.

That's how I do all my videos, lol. They seem to lack this "clack" you speak of .

 

[playerunk]

max science FTW:

The basses should be equipped with same hardware [bridge and nut since they come in contact with the strings] and tuners [i heard that extra weight on the headstock adds..something..something stuff idk]

Same brand of strings, new sets on both basses.

Same pickups/electronics [maybe the same] and the same shielding job.

Same or similar setup [maybe you can't get the strings the same height on both basses]

Strings plucked at same spot [you could prove here that changing the place where you hit the strings has more impact on tone than wood eg : pluck near the bridge and you can get 'snappy' with alder]

To eliminate string pressure you could use a capo and go trough all the notes to 13 fret on E and higher D G

Put the bass on a stand at your playing height.. so you touch the bass as little as possible ..the wood can't resonate properly if you press it against your body

After recording compile a project in a widely used recorder [eg audacity] so people can swap tracks more easily.

real life FTW:

And after all this hard work and PITA 'prudencies' for the sake of tone wood get your basses and jam a few tunes with your band..live setting style and record the whole mix [mic and from the mixer if you have a usb/firewire out] but also the solo bass track.

And finally get some other bassists to do some testing and post the results for each..i bet the alder bass of one will sound like the maple from another.

I wrote all that passed trough my head and over it..so some might be stupid..

 

[GlennW]

That's how I do all my videos, lol. They seem to lack this "clack" you speak of .

Maybe you misunderstood me, maybe I misunderstood Bongolation, maybe both.

I'm talking about passive basses with no tone and no volume controls; that's the pickup wired directly to the output jack with nothing between them. That will give maximum treble. Everytime I've done that the result was a trebly and useless tone (for me).

I listened to your youtube clip and it sounded nice, and I'd never have guessed that your pickups are wired to the jack with no controls. If they are, I'd guess the pickups are very bassy sounding to begin with and/or you rolled off some treble from the amp or somewhere.

 

[mongo2]

BINGO!
Dan, this is THE most important step in structuring any kind of meaningful test. Clearly articulate your hypothesis first. Then set out to prove or disprove it with test protocols that isolate as many of the variables as possible or in this case as your limited resources (being the two test basses) allow.

While you are correct about clearly articulating the hypothesis being the first step, the use of an extremely limited sample pooches the test process from the start.

All it can hope to do is demonstrate the effects of the test sample which can't logically be extrapolated to validate a hypothesis. Many more datapoints (from random samples) are required to begin to show patterns which may then indicate whether a hypothesis is worthy. However there are other things that get in the way of the validity of these types of tests.

In my day gig as a test engineer, the possibility of "cherry picking" the samples to falsely validate or invalidate a hypothesis must be guarded against. Remote testing provides no possibility of verifying the integrity of the sample selection process. No accusation intended, simply stating a reality of testing in the real world.

No one can deny that people hear things differently so another issue is the varying ability of those listening to be able to differentiate between the samples. One thing these threads have shown is that people's sensitivity to the differences varies. In short, some can hear it (to varying degrees, so maybe it's not really a binary condition) and some can't hear it at all.

Also the listening environment is not standardized which could easily mask the ability of anyone to hear the difference and therefore skew the results.

One way to alleviate the listening aspects would be to use a spectrum analyzer, but then it's not a listening test or a test of listening, whichever you prefer.

There are a host of other variables to consider such as the density of the samples, plucking accuracy, setup parameters, etc. that also need to be considered.

In short, there's simply too many variables some of which are uncontrollable.

 

[GBassNorth]

While you are correct about clearly articulating the hypothesis being the first step, the use of an extremely limited sample pooches the test process from the start.

All it can hope to do is demonstrate the effects of the test sample which can't logically be extrapolated to validate a hypothesis. Many more datapoints (from random samples) are required to begin to show patterns which may then indicate whether a hypothesis is worthy. However there are other things that get in the way of the validity of these types of tests.

In my day gig as a test engineer, the possibility of "cherry picking" the samples to falsely validate or invalidate a hypothesis must be guarded against. Remote testing provides no possibility of verifying the integrity of the sample selection process. No accusation intended, simply stating a reality of testing in the real world.

No one can deny that people hear things differently so another issue is the varying ability of those listening to be able to differentiate between the samples. One thing these threads have shown is that people's sensitivity to the differences varies. In short, some can hear it (to varying degrees, so maybe it's not really a binary condition) and some can't hear it at all.

Also the listening environment is not standardized which could easily mask the ability of anyone to hear the difference and therefore skew the results.

One way to alleviate the listening aspects would be to use a spectrum analyzer, but then it's not a listening test or a test of listening, whichever you prefer.

There are a host of other variables to consider such as the density of the samples, plucking accuracy, setup parameters, etc. that also need to be considered.

In short, there's simply too many variables some of which are uncontrollable.

Yep, all good points that would have to be addressed depending on the hypothesis. Waiting for Dan at this point to reply.