If you are a physicist as you claim then you should certainly know that the vast majority of modern physics was not based on experiential evidence. The biggest breakthroughs in modern physics were based on logistical extensions and new equations, much of which was eventually verified as modern technology progressed to be able to accurately measure what was already known.
Also, it cannot be refuted that wood takes decades to dry (this can be measured by weight over the years). As the moisture level is reduced, the wood itself vibrates more readily and the tone gains harmonic complexity as the wood is not dampened by the moisture (and I mean sonically dampened, not wet).
As far as the settling of the fibers in the wood (as well as the breakdown of the interconnecting fibers WITHIN the wood), that is, as I said above, a logistical conclusion that is based on the observations of hundreds of years of luthiery. As it is improbable that someone is going to cut apart an instrument over the course of 200 years and molecularly analyze the wood as it ages, I will cede the issue to you and agree that it is not feasible to take on such a task.
But if you want to argue with what has been learned over the millenia that people have been making carved instruments (and particularly the last 600 years or so) then go right ahead.