If you're not going to be doing any overdubbing or punching--i.e., you're just going to be recording one stereo take and then dumping that to a CPU--then yes, all you'd need would be a preamp and the recorder (and tape, and cables, obviously).
A half-track (2-track) would do the job, plus you'd be able to use it for mixdown if you wanted.
You could probably snag something like a Tascam 22 or 32 for around $400 in pretty good condition. But that's never a sure thing with gear of that age.
Those machines use 1/4" tape, which goes for about $25 a reel new. I think you get around 20 minutes per reel if memory serves me.
You may need a take-up reel as well, which will probably be $30 or so.
However, that's the bare minimum, and other things are dependent on what kind of machine you wind up with. If you really want to do things "by the book," you'd want to get a calibration tape to make sure that your machine is up to spec. Those usually run a couple hundred dollars.
Then you'd need a HandyMag demagnetizer, which is around $70 I think.
99% alcohol for cleaning the tape path is pretty cheap (under $10 for a bottle that'll last a long time).
There are other things as well that you may or may not need, but that's hard to say until you wind up with the machine.
All in all, I'd say you should expect to pay anywhere from $500 (on the lower end) to $700 or $800 on the higher end. That's not including the preamp.
Preferring analog is kind of like what you hear about being gay: It's a hard life that no one would choose if they saw any other way.
It's expensive, and it's rare that you won't have to perform some type of repair or, at the very least, a good deal of maintenance. But it sure is fun when everything's working, and it sounds great, of course!
Hope this helps