Been doin' this for a long time. A lot depends on what all you want to do. Do you want to lay down multiple tracks for arrangements? Use software synths/samplers/virtual keyboards? Record vocals and acoustic instruments?
If you haven't gotten the computer yet, there's the question of what specs to look for and what kind of machine. I use Macs and Linux for almost everything, but Windows for all my DAWs, because of the body of synths and plug-ins I've amassed over the years. They're more expensive, but I tend to use high-end laptops for DAWs, for the portability and for the (acoustic) noise factors. If all you're doing is recording audio, you can use a really wimpy computer BUT chances are you're going to want to have some EQ, effects, compression, etc. and those consume compute cycles. I recommend almost-the-fastest CPUs you can buy (the very fastest always cost a ridiculous premium) and 4 or more cores. USB3/USB-C is plenty fast for multi-channel audio. I personally would never buy an audio interface that didn't have MIDI in/out because I've got a lot of MIDI controllers, but you could save a few bucks on audio only if you're sure that's all you're going to do for a while. I've used RME, PreSonus, and Focusrite interfaces over the years. RME stuff is fabulous but really expensive, especially in the US. Focusrite seems to be the current market leader in volume and number of products, and they have an interface for almost any budget. I have not been wildly impressed by the stability of their drivers (RMEs were legendarily solid), but on the other hand their pre-amps are pretty good for the money. I could plug a bass or guitar directly in and it would be OK, but as I happen to have pre-amp pedals for bass and EUB, I keep those and run the XLR outs into the interface, instead of the high-z 1/4" instrument-in.
As for which DAW software to use, if you're going for best value-for-money and aren't fussy about user interfaces, Reaper is an obvious option. You can use it free in uncrippled demo mode forever, but buying a license is very cheap for what you get. I just find the user interface the least intuitive of any DAW I've tried (which are basically Creator, Logic, Sonar, Studio One, Ableton Live, and Reaper). Sonar was great, but they folded and the folks who took over the source base aren't the same people. I went to Studio One after Sonar, and it's very well designed in many regards, but its MIDI management is nowhere near as good as Sonar's was.
IK Multimedia has a virtual SVT amp plugin that's actually quite good, but something of a one-trick pony. I also play guitar/stick/etc. so I've been using Native Instruments' Guitar Rig for years, which has a big portfolio of guitar amp/cab models as well as a few bass rigs, and adds a HUGE toybox of cool effects.