Do you already have a sampler?
Over the years I've collected about a dozen CD's, a dozen cassettes and a dozen records of sounds effects - everything from crowd noises to animal sounds, to appliances (phones, razors, etc. etc.) and even belches, farts and hiccups. All of them royalty free!!! I doubt that I have spent more than $200 total, with most of the cassettes costing $2-$4, the CD's maybe $5-$7 and the records maybe $1 each.
I found the majority of these at used book stores (that also sold recordings), CD exchange stores and "cut-out" bins of various "record stores". It took some time and effort - but it saved plenty of cash (plus many were collected before "sample libraries" were as plentiful as they now are). Plus, I rather enjoy (when I have time) searching for old books, and recordings.
Although I've got a decent sampler, most of the time I use a small "phrase sampler" to pop in a sound effect or two when needed. I've work on several jingles and post production for a few independant films, and stage plays - and I've always found the sound (or combination of sounds) that were needed.
There have also been times where I created sounds and various "foley effects" useing stuff around the house - but that was when I had the time and creative energy to spend a day trying to create (vs. sampling a sound from a CD).
