Price absolutely makes a difference ... the question is, does it make a difference to YOU ....
I gig a lot and I don't want to have any issues with my basses. They must be able to hold up to my sloppy handling of them and they must be professional instruments in all ways. So, I have to pay for that luxury. Could I get away with using a production bass instead of a custom bass? Sure. But my basses are built to be exactly what I expect from them.
It's really very simple, IMO .... buy what you can afford, buy what sounds good to you, buy based on your realistic needs. If you can afford very high end stuff, then go for it. If el-cheapos is the best you can do, then find the best el-cheapo in your price range
I have never felt the need to justify a bass expenditure in my life and I surely don't care what other people think of my basses. As someone said above, play it first and fall in love with it, then worry about the price .....