OK, listen and shiver!
The sound of your midi (and audio) will be better, but the soundfont handling will be worse....expect to miss some notes now and then and stuff like that.
On the other hand you can use LiveSynthPro for soundfonts (easier when mixing) and everything is alrighty!
Tough choice right?
I still wouldn't recommend anyone to buy an Audigy. But if you already have one, don't give up on it too quickly. Depending on your needs, you can still get a semi pro sound out of it.