I used to work exclusively with Logic, but I've progressively been switching to Reaper, with which I'm seriously impressed. I also use Vegas for video work, though not for recording.
I've not experienced latency problems, and I'm struggling to find further possible causes for your problem. I usually leave the interface latency for my Firepod at its default setting of 10ms, though I have recorded successfully at 3ms. However, I am using the Presonus Firepod drivers, not ASIO4all.
Perhaps its something to do with the Peavey USB interface?
While I don't discount the possibility that Reaper may be the source, I can figure out how it could be.