pre-amp - whatever you need to shape your tone, plus maybe some basic functionality for ringing out whatever room you are playing in.
to my ears, tubes overdrive into a nice fuzzy saturation. silicone tends to be a bit more metallic and edgy sounding.
power-amp/speaker area - balance of wattage and speaker area to achieve your volume requirement, plus a little headroom.
tubes require a bit more TLC, but can add a natural compression. solid-state has a few different types, some are beefy and powerful, some are super light. i've heard people complain that the digital amps sound sterile, while others bless them for instant power.
speaker-type - again whatever you need to shape your tone. bigger cones aren't as articulate in the high-end response, and they may also be slower to respond. smaller cones are faster and can articulate better. depending on cabinet, any driver, even the best, can have a boomy or nasal character to it. look for a cabinet with thick sidewalls, strong bracing and internal padding. A solid box will not add any overtones of its own. Only the speakers/ports/baffles will be creating the sound.
look for a grille that doesn't vibrate.
neodymium vs ceramic/ferrous - weight is the biggest issue. I cant tell the difference aurally, though some people report huge differences. i'll leave elaboration to them.
Depending on where you practice/play, you might also want to find a set of gear (rig) that is capable of creating the tone you prefer at low and high volumes. e.g. you may achieve a great sound with the gain cranked, but that can be a hindrance when practicing at 2am. in that case you might want to look for something with a master volume as well.
The more components, connections and cables you add, the more you introduce potential sources of noise.
I would suggest if you're starting out, to get a combo. Pre-amp, power amp, speaker all in the same cabinet. If you need some more serious fair, a head (pre-amp/power amp) and a large cabinet arrangement should fit that need. Unless you're going on tour or have a fair amount of disposable income or really like tweaking things, you probably won't need a rack with separate effects, pre-amps, power amps and crossovers. Keep it simple is a decent rule.
Summarily- head to a music shop, play through everything, pick what you like that's also practical (i.e. not the amazing 810 that wont fit in your bedroom).