The first stage is very important. I think that the phase inverter is also an important component of the tone. As Charlie said, everything matters. This includes the tube stages, the tone stage, the phase inverter and power stage, the output transformer, as well as the power supply and how much ripple is present. They all come together to define the performance and tone of the amp.
In the B-15, the first stage is designed to amplify in a very conservative, pure way, minimizing compression and other forms of distortion. It is basically pure 6SL7 tone applied to the instruments signal. They achieve this by the way the tube is biased, with a relatively high plate voltage, and not using a cathode capacitor. Very different than what Fender was doing at the time. This sets the B-15 apart from other amps and is what imparts some of the magic it offers.
The phase inverter in the B-15 is what they call a floating paraphase design. It works very well in the B-15. It was used in some of the early radio circuits. The purpose of the phase inverter is to split the input signal into two out of phase signals that are used to drive the power tubes. The problem with phase inverter designs is that they are not perfect. These imperfections affect the performance of the power amp, and can impart some of the character than an amp has. This is why some people, especially in hi-fi where accuracy is desired, install a phase inverter tube where the two halves of the tube are matched. This helps optimize how consistent the two output signals of the phase inverter are. This isn't as important in musical instrument amps. As I mentioned, an imbalance can add character to the sound of the amp. A big plus of the 6SL7 tube is that the two halves age fairly consistently. This offers stability over time in terms of how the phase inverter performs. It keeps the amp sounding the same.
Here is an explanation of the contribution that the phase inverter makes in an amp. Take what they say with a grain of salt. The contribution of the phase inverter, as far as tone is concerned, will vary from amp to amp. For an amp that offers a lot of distortion, the phase inverter can make a big contribution to the sound. Not as much in the B-15, although don't underestimate its contribution.
In the early 60's, not that long after WWII, designs looked to the future. Everything was radically different, new and improved. Fender moved away from octal tubes in their designs. Ampeg needed to be different to compete with Fender. The B-15 was intended to be a hi-fi sounding bass amp. They took their queues more from the hi-fi world than they did from the musical instrument industry at the time. The octal tubes, the potted transformers, the James tone stage all harked back to the 1950's hi-fi designs which had great tone. It was old school and vintage which was in the opposite direction that contemporary designs were going in. The amp had an artistic, conservative, and vintage look that matched the tone. The design has stood up to the test of time in a way that very few other amps have. It has also proven that new and improved isn't always best. There's definitely something in the sound of those old octal tubes that people love.