If you are an analog purist go ahead and start out with a 4 track cassette tape recorder and gradually move into the more expensive tape equipment (2" reel to reel, etc) as your experience and bankroll grow.
If you're just an amatuer recorder wanting to get a few ideas down and expect to gradually expand your skills and involvement in recording, go the digital route. You can eventually port your work from the 4 track recorder to a computer or all-in-one digital DAW. Working on a computer means you have the option of using hardware or software processors, and editing in the digital domain is much easier.