True, but on the other hand, the definition of believe includes faith in something and proof is not a requirement.
But to say "I believe in ancient astronauts" is problematic; what do you believe about them? That they came here? Etc. Saying that you believe in God is not the same problem; there is a document that holds tenets of the belief, just as evolution has. This usage is correct.
Ermm... I'd suggest that there are very similar epistemic problems in asserting belief in ancient astronauts and supernatural entities. But that similarity is not something I can explain here....