I agree with the lieutenant. You CAN get started out on your limited budget, just understand that you won't be putting out studio-quality material. You can start off small and build up your equipment over time as your finances allow and your expertise grows.
But stay away from the $100 four-trackers, they are too limited in what they can do. For one thing, they can only record one track at a time.
If you have to be able to record four tracks at a time,
the Tascam Portastudio 414 is probably your best bet at $250. To go digital, on your budget
the Fostex MR-8 is proabably the best choice at $300.
The Zoom MRS-4 is not bad either at $230. You will be limited to two track simultaneous recording. The Zoom only records at 32 KHz, versus CD quality 44.1 KHz for the Fostex. Also, you will need minimum 128 MB Smartmedia card for Zoom, as the 32 MB card included with the unit is not enough.
You can track using headphones or you can hook up to a home stereo system. That's not ideal, of course, but it will work.
Most importantly, have fun.