Although my experience is more with Roland VS1824CD and Korg PXR4 Pandora, on those units, it makes a huge difference. Acorec is right and wrong. He's right that the signal will be routed through the preamps, and they do have an effect on the signal. However, using the cruddy pres for gain makes it much worse.
Here's a trick that worked for me. Jack the 2 balanced line outs from your outboard pre into a TC Electronics M300, a $200 FX box. Push the bypass button, which bypasses the FX and routes the signal straight to the coaxial S/PDIF out. Then to the S/PDIF in on your SIAB. The A-D conversion isn't really any worse (or any better) than the conversion on my Roland, but it bypasses the pres in the Roland completely. And- it's not really a bad little multi-FX box. Then you can use any pre you want. Even a DMP-3 trashes the pres in my Roland. And you get a De-esser, chorus, delay, flange, and a rudimentary compressor. For $200, what's not to love?-Richie