If you want a lot of power and flexibility very cheaply, a computer DAW has a lot going for it.
But don't be fooled into thinking it's a panacea, because that's stupid. That something designed as a 1980s wordprocessor can be used for audio recording is the result of 30 years worth of bodges, and this frequently shows.
A new machine
First, you will really want to buy a separate machine and use that exclusively for audio, ideally not attaching it to your home network either. Modern operating systems tend to run all kinds of background tasks instead of capturing the audio. Windows could never do it reliably IME, Linux has recently started getting dropouts and underruns as well. OSX I can't speak for.
Whichever the case, using your day-to-day machine as the DAW will mean that it's fighting for resources with whatever other software has been installed, since everything loves to have its own update daemons or other gubbins running in the background, and that's assuming nothing abnormal happens like some kind of malware. So you want to set aside a machine that has the bare minimum needed for the DAW software to run and hope windows doesn't want to phone home in the middle of a take.
Most of the above problems can be alleviated to some degree by getting someone to build and set up the machine for you. There are companies like Carillon audio who specialise in this. But it will not be cheap, and there is footage out there of pro artists like Spock's Beard and Blancmange having the DAW die on them in the middle of recording session, or in Blancmange's case, during their Liverpool 2011 gig.
Nothing lasts forever
Then you have the dilemma of whether you keep the machine up to date as they put the stake through XP's heart to try and stop it getting back out of its coffin and new hardware and software stops running on Windows 7 so that people are forced to migrate to 8 or 9.
It might also interest you to know that Microsoft recently described the applications running the entire audio and video production industries as 'legacy' because their goal is to kill off the workstation market so they can make tablets instead. That way, everything has to be bought and sold through their app store where they can get a 33% cut from each purchase (and the new security model for 'non-legacy' apps makes interprocess communication impossible, so no plugins. Ever.)
I have a fine collection of hardware and software that Windows no longer supports, so you'll have to choose whether you want to stay in a timewarp and hope that nothing dies that can't be replaced. For example, disk interfaces have evolved from MFM -> RLL -> IDE -> SATA, and SATA too shall pass away. Each upgrade is incompatible with the last, so if you do stick with a Known Good configuration and never upgrade, you will have to hit ebay for spare parts, just like a tape deck.
The alternative is to try and keep up with the bleeding edge, and constantly having to fork out for upgrades as the hardware and software evolves.
The Mac is even worse for this because while Microsoft usually tend to have some form of compatibility layer - though they have been changing their minds about that lately - Apple make no bones whatsoever about breaking things. At best, this means you have to buy a new version of the DAW package/plugin/whatever. Unless of course, the vendor has gone out of business, in which case, it sucks to be you.
While I've never used OSX for audio, I have seen this happen over and over again with other things, e.g. when OSX Tiger dropped support for Photoshop 7, upgrading to a version that worked was nearly as much as the Mac itself, something I could not afford to do at the time. I write software myself, and have essentially given up on the mac as a dev platform because it's such a constantly moving target.
Now, with OSX, you do get the GarageBand DAW software with it. That has the strong advantage that Apple are maintaining it themselves, so it will be upgraded as a freebie with the OS update bundle so you won't need to buy it again every 12 months.
However, Apple have a habit of discarding or replace bundled software with something new and shiny, which maybe worse, and is most likely incompatible with your existing projects. In other words, you probably won't be able to go back and remix the album in 5-10 years time because it will be gone or replaced with something incompatible. Apple left a lot of big names high and dry with the Final Cut Pro fiasco, and I dropped iMovie like a brick for much the same reason.
Oh, yes. And the industry is moving towards a ransomware^W rental model, like Photoshop where you have to keep paying every month to keep using it. Kind of like tape costs, except you don't get anything to show for it.
The audio interface
Then we have another piece of the puzzle, the audio interface. You can get these cheaply, but of course they will be made very cheaply. The converter chip will be the cheapest they could find, the drivers will have been cobbled together from the reference implementation by interns and the capacitors will not be audio-grade, increasing the amount of noise. What I have seen happen over and over again is that the coupling caps die after a couple of months so you end up with a digital ground loop and the RF noise from the computer or harmonics from the USB bus get sprayed all over the audio signal. (Hello, Edirol!)
Again, this is a problem that can be fixed by throwing money at it - better converters, ideally with capacitors from an established brand.
Backups
Next up: Backups. Your hard disk will die at some point unless you're really lucky. SSDs crash in a similar manner, large chunks of data turn to slush and the drive fails to be recognised in the morning.
You can't prevent that, but you can mitigate the damage, by getting external USB disks - at least two - and regularly copying all the important stuff onto the external disks, say, every month. If possible, keep one of them off-site and swap them over every month or week or whatever your backup schedule is.
If possible, try to dump stuff from finished albums onto a collection of other media, DVD, Blu-ray, tape or whatever (the more the better!) - since the data can rot on the original disk, or one of the bits can flip in the disk controller or memory. Unless you're using something really special like ZFS, BTRFS or ReFS, the computer won't know that the data has been damaged and will happily copy the broken data over the good backups.
Standalone DAW
All this presupposes that you're recording on a computer DAW (except for the backups thing, which you should do anyway, regardless of what system you're using).
A standalone DAW will alleviate the problem of dropout and Windows going 'I saw a squirrel!' when it should be recording your divinely inspired guitar solo, since they will run an OS like QNX or VXworks that is actually intended for realtime use (think life-support machines, engine controllers etc) rather than the bloaty one-size-fits-all system software we use on general purpose computers.
However, the editing side of a standalone DAW is usually shit unless it cost about as much as a 24-track tape deck. One approach is to do all the tracking onto a standalone DAW, and then, when it's finished, pull the tracks into a computer DAW for editing and mixing. With that you can get away with using your normal desktop/laptop for the editing since it won't have the additional overhead of having to be able to capture audio reliably.
And analogue?
Unless you're using something small and simple like a cassette multitrack, I won't pretend that analogue is less complicated than digital. But I have had more than my fair share of problems with digital as well. In my experience they require similar amounts of work to keep going, you're just replacing one set of problems with different ones.
And yes, cost-wise, analogue has the additional expenses of the media if you wish to maintain a library of it. You can do the hybrid approach, where you track everything to tape, digitize it and then edit it in the DAW. After that you could then wipe the tape and reuse it for the next session.
The key thing is that with most digital systems, you're getting something that's been hacked into working with a certain amount of latency and jitter. With analogue systems, you're getting something that was designed from day one to do audio and nothing else.
Personally, I started out doing everything digitally (because I had a computer already), but switched to analogue as my needs increased. I think a lot of it was a romantic desire to be able to experience to some degree the process which went into making great albums of the past.
I could probably get better results with digital, because each time I flub a note on the bass or the vocals I could go back in and edit it into its proper place or chop in the same note from elsewhere in the song or another take or something. But I kind of like being forced to do it right in the first place (or at least with overdubs) - I find these limitations help me creatively.
That said, I do cheat rather a lot in that I'm composing the songs on a sequencer first and tracking that to tape instrument by instrument - if I was playing every single instrument by hand it would probably be a lot more painful.
The take home message
The more that things change, the more they stay the same.
Analogue: Cheap or Expensive, often temperamental, requires regular maintenance, may require a lot of ongoing expense on media
Computer: Cheap or Expensive, often temperamental, requires regular maintenance, may require a lot of ongoing expense on software
...you pays your money and you takes your choice. Either way you go, you will have problems, because at the end of the day, technology is hard.
