I've found that latency really became an issue even with my 4/4 M-audio Quattro running just two in and two out simultaneously. I returned it, doubled the money out of pocket and invested in a M-Audio Delta 1010 (pci-based) and I couldn't be happier with the results. If you're only looking for several tracks at one time, check out the Delta 44 (or 66 if you need S/PDIF in/out). For the same price of a lot of USB 4/4 cards you get the PCI interface, which I think improves speed and causes less hassles.
That's my personal opinion.
Christiaan is right, strictly speaking about sound quality, but I think ease of opperation and latency issues should be taken into account as well when purchasing a sound card like this.