Well, I can speak on this with experience, because I've seen both sides of the fence. I have a digital setup (8 input interface) that I use for my work stuff, but I prefer to do all my "fun" stuff on tape.
Depending on how much you want to do on analog, it can get very expensive compared to digital. If you want to avoid the CPU altogether except for maybe using it as a final mixdown "deck," then you'll not only need the tape recorder, mixer, and cables, but you'll also need:
external signal processors (such as verb, but also compression and maybe EQ depending on your mixer)
the cables to hook all those up
probably a patch bay or two depending on the amount of processors you have and the options on your mixer
rackspace to mount the external processors
If you want to mix down to tape as well, then you'll need a half track deck (such as the Tascam 22, 32, or Fostex E2 or 20 or something similar).
I would love to mix down to tape, but I haven't yet been able to afford the half track deck (anywhere between $200 to $500 probably). So I keep everything analog and don't touch the computer until I mix down to it.
Obviously, digital rigs can get very expensive too, but at the home recording "prosumer" level, I can't lie. It's
significantly more expensive to go analog.
Just to compare, here's the breakdown of my two rigs, including similar elements (in other words, not including things like mics, external preamps, etc.)
Digital:
Dell computer
$80 (already had a monitor)
External HD
$40
Delta 1010LT interface
$150
Memory upgrade to 4G of RAM
$30
Reaper
$40
Toontrack EZ Drummer
$70
Total: $410
Analog:
Fostex R8 reel to reel (8 track)
$260
Yamaha M216 mixer
$100
Peavey Addverb III
$65
Roland SDE-2000 (delay)
$65
DOD 31 band graphic EQ
$55
2nd DOD 31 band graphic EQ
$50
Yamaha SPX90
$115 (and this died, so it's not usable right now)
Fostex 3050 (delay)
$60
DOD R825 compressor
$50
2nd DOD R825 compressor
$40
DBX 166 compressor
$35 (my best deal yet)
Rack accessories (I built my own mixing desk with built-in rack from the wood of my father's childhood bed)
$20
49 point patchbay
$30
32 point patchbay
$30
2nd 32 point patchbay
$30
1/4" tape for Fostex R8
$110 (so far---this is obviously an ongoing thing)
Connecting cables - I really don't know for sure, but it's at least
$200, and that's with making many of them
Total:
$1315
So on the analog rig, I have 4 channels of compression, one multi-effects processor, two delays, 8 tracks, and 16 mixer channels (4 buss).
Now, obviously these figures can lean a little bit depending on your view of necessity. I included the EZ Drummer software in my digital rig because it's something I use all the time with that rig, because it's usually just me recording. And there are a few things I didn't include, such as a MIDI keyboard controller, because it's the same one I've had for 20 years, long before I even thought about putting a digital rig together.
I consider myself to have a modest digital rig and a modest analog rig, and so my point is, when you compare those two (IMO, obviously), the analog rig is about 3 times as much.
I love analog, and digital is a necessary evil for me with my work, so I definitely don't want to dissuade anyone from going the analog route. But I just wanted to give you a fair comparison so you know what you're getting into.
People don't get into analog recording because it's cheap!