I compiled this a few years ago when someone asked about NYC shops...
MIDTOWN
Guitar Center (West 44th street & 7th Avenue) - This is reportedly the new Guitar Center flagship store in NYC. I've yet to go to this store yet myself, despite the fact that I work about 5 blocks from here.
CHELSEA
Sam Ash on 34th street & 9th Ave. The Hartke Bass lounge is effectively Sam Ash NYC's bass department. You can expect to see some Sadowsky's and Fender Custom Shop instruments here, along with a few other high end offerings that you won't see in a run of the mill Sam Ash store.
D'Angelico Showroom (141 West 28th street) This is formerly the location of GTR Store - I haven't been here since it was GTR store, but it was pretty cool. (I think it's now appointment only)
30th Street Guitars (234 W 27th St, New York, NY 10001). Great vintage/used gear, albeit a bit pricey
Dan's Chelsea Guitars (West 23rd street). More of a guitar spot. Very tiny. but also good for some cool vintage gear
THE VILLAGE
Guitar Center (14th street btwn 5th & 6th Ave). The best bass inventory I've ever seen at any Guitar Center, and they've got a vintage room.
Matt Umanov Guitars (Bleecker Street). Cool vintage & new stock, and let me echo awesome pizza right across the street at John's. I saw word that they were closing, too. And it's a significant loss for the neighborhood. But with rents going through the roof I can understand why this happened.
Carmine Street Guitars - Rick Kelly is the guy who runs this place. He's made some amazing instruments for some of NYC's rock royalty. And he's got a series of instruments (including basses) which are made from reclaimed timbers. Some of that wood has come from historic New York buildings, and some of it dates back to the mid 19th century.
LOWER EAST SIDE
Rivington Street Guitars (East 4th st, NOT RIVINGTON ST). I've been to this store once. It's a nice little spot with some cool, funky vintage gear.
SOHO
Rudy's Soho (461 Broome Street) Rudy's was one of the legendary spots on West 48th Street, but they've closed the original shop and moved down here. I haven't been there yet, but the original Rudy's was one of the places where you could always find a great high end instrument.
QUEENS
Sadowsky shop (Long Island City). I haven't actually paid the Sadowsky shop a visit since they were still in Midtown Manhattan in the late 90's. But Roger & crew are very accomodating, even if you're just there to get one of their awesome preamps.
BROOKLYN
Main Drag Music (Williamsburg). It's pretty much all vintage stock here, with some very, very cool offerings.
Southside Guitars (Williamsburg). Great vintage offerings. They also deal in some modern gear (Nash). Very cool guys to deal with.
Ludlow Guitars (189 Front Street, Inside Christian Benner Custom, New York, NY 10038.) Effects pedal heaven. Some modern Fenders, but this store is geared a bit more towards guitar players. However, they are Aguilar dealers which is a nice plus if you're looking for amps. FWIW, this place used to be on the Lower East Side in Manhattan, right across the street from Katz's deli and around the corner from some venues I used to play on a regular basis. I miss having it on Ludlow street.
Fodera Shop - While they're not an instrument store, any time I've run into Joey Lauricella of Fodera he has been kind enough to invite me to come check out the shop.
If you can get out of NYC proper, there are some other great spots. The Music Zoo is just outside the city limits on Long Island. There's Lark Street Music in Teaneck, NJ, which is an amazing vintage spot. Alto Music in Middletown, NY is fantastic. And I'll add another vote in favor of New Jersey Guitar & Bass Center in Edison, NJ.