Fells Point has a couple of good records stores: Soundgarden is the largest and maybe the best (depending on what you're looking for). El Suprimo is near the corner of Aliceanna and Broadway and is great for vinyl. Record Guru is also on Broadway, tucked away down a tiny alley. I haven't been there, apparently its a lot of older records.
There's also True Vine up in Hampden, but that specializes in weirder, obscuro vinyl and music.
If its late and you're looking for a bite, your best bet might be Paper Moon, on the corner of 29th and Cresmont Ave. It used to be 24 hours, but the recession forced it to cut back. Now the last seating Sun-Thurs is midnight and 2AM on Fri/Sat.
If you're into comics and the like, Atomic Books in Hampden is a great store, lots of indies, majors and crazy books on Satanism and stuff. If you prefer a more conventional comic store, there's Mystic Sprial (or something Spiral, I always forget), a couple blocks north in this building called the Rotunda, on 41st st. between Keswick and Roland.
The Charles is a great indie movie theater, and the Harbor East theater at the end of Presidents St. (sort of between the Inner Harbor and Fells Point), usually has a pretty solid selection as well.
The Sidebar is a good bar, it's always got good specials and a great dive-bar feel.