I agree a small four track (to cassette) costs about $99 (not counting mics, cords, headphones, etc). Both Fostex & Tascam have low cost 4 tracks.
Don't be in too much of a hurry to record. Learn to be a musician first. Regarding teaching yourself (I assume you mean on guitar)- try to get some lessons sooner rather than later.
While many people are self taught, most eventually try to get some formal education and often wish they would have recieved lessons sooner. It is very easy to learn bad habits and bad technique - and it is very hard to unlearn it.
I am basically self taught on several instruments (guitar, bass, drums, keys, violin, banjo0, harmonica, voice) but when I have had lessons, I always found there were things I was doing poorly. It was often difficult to correct (after teaching myself the wrong way) but once corrected, my playing always improved.
Once you start to learn a little on guitar - find people to play with. Interacting with other musicians (in particular better musicians) is a great way to improve as a player!!!!