Yes, I decided my major eight years ago, and the deeper I get into it the more I love it. I always wanted to be that relative who couldn't come to holiday dinners because she was gallivanting around the world, and with a doctorate I could.
Anthropology major (psych minor), I was thinking of doing Psych in grad school so I could become a adjunct professor at the community college level, but my adviser suggested that I look into doctorate programs and I'm giddy at the thought of being an anthropologist.
Well, both of those fields require graduate school if you want to work in them. You're going to need at least a master's in either field to teach at a community college and most likely a doctorate to teach at the university level. If you go into any applied field of psychology (e.g., clinical, counseling, I/O, community, school, etc.) you're going to need at least a master's and if you go into the research side (e.g., social, personality, cognitive, legal, bio-comparative, etc.), you're going to need a doctorate. If you want to be a practitioner, a PhD or PsyD is probably where you're going to end up eventually.
It is kind of ironic that you are an anthro major with a psych minor, since the former discipline largely comes from an anti-positivist/constructionist ontology and the latter is pretty firmly rooted in post-positivism. If your interest is in cultural anthropology more so than say physical anthropology or archaeology, don't discount sociology programs either. In a good soc program, you should get a good helping of both constructionist and post-positivist perspectives and walk out of it as both a good statistician and a good ethnographer (that's a lot of "goods" in one sentence lol). Related - and you probably know this - if you are serious about going to grad school for any subfield of psychology, make sure you take at least one methods course and one stats course in your minor.
With that said, only you can answer the question of whether or not you should go to grad school. It's all in what you want to accomplish and where you want to go with your career. I know I said that you need grad school to work in anthropology or psychology, but nothing says you have to pursue those careers. I know attorneys with biology undergraduate degrees.
I will say this. Going for a doctorate, whether it's a PhD, PsyD, DEd, ThD, or whatever, requires a true love of the discipline and you need to be an autodidact. As Mohawk said, it requires a great sacrifice. I'm currently ABD with two or three more chapters to write on my dissertation. Many of my friends from college own their own homes and are pretty rooted in their careers at this point. I gave up a lot of stuff to pursue a doctorate, and I wouldn't change it for the world. I made the decision to pursue a PhD before the first day of my freshman year of college. I don't need it for my career, but I'd rather have it than not have it; although, in a world where you need a bachelor's degree to serve coffee anymore, I found it really annoying that I had to justify my doctorate to a lot of hiring managers and search committees to counter the BS "overqualified" argument. But it's been a very personally and intellectually fulfilling experience. I'm fortunate (or smart?) in that I picked a dissertation topic that I love, so working on it is not as painful as it could be. With that said, I should be doing data analysis right now, but I'm hanging out on Talkbass and I'm gonna go watch Marc Maron's show on Netflix.
My last piece of wisdom for what it's worth: It's not uncommon for people to question if they've made the right decision or not or their worthiness when it comes to graduate, especially doctorate level, study.
P.s. I also came from a family of blue-collar workers and understand feeling overwhelmed. I didn't go to college for four years after high school. The reason I didn't go is that it was communicated to me by people like guidance counselors and such that since I wasn't an academic track student, then I wasn't groomed for college. I'm not making excuses, but I simply didn't have the cultural capital to understand the whole academic process.