Doesn't need to be expensive.
I have a Samsung Ace 3, was once € 150 or something like that, got it for € 29 with a contract...
Does record 'ok' for what it needs to do... If I have a nice melody in my head, I can just record right away and humm or play my acoustic guitar to get the idea on the device. To re-record later, on the computer.
However, I once recorded the sound of a train, when commuting back home, the engine went: ggggggggggggrrrrr, and brrrrr, vvvvvvvvvv , zzzzzjjjjj
You can listen that sample here:
I added some guitars on top, to create some sorth of soundscape.
Hope you like it ;
More on: http://users.telenet.be/bulevardi/