+1 on the zoom r16 ($400)
or a zoom h2 ($150)
I have an Emu 0404 USB ($200) interface that I really like. Also really like my Fostex MR16HD recorder, which you might find these days for $350, but for that price range, I'd get the newer zoom r16.
You are going to need a mic, mic stand, cable to go into either your interface+computer or your zoom R16/h2 recorder (although on these recorders you could use the on-board mics for awhile).
You can get the excellent Reaper audio recording software for $40 if you are using a computer. USB your zoom r16 tracks to the computer, or record directly into your computer with the interface+Reaper.
So one of your first decisions: Do I want to record onto a computer?