I also have Outputs 1,2,3 and 4 on my soundcard, but I think my situation is a little different than yours. Here's what I do with mine:
1 and 2 = port 1 from the soundcard.
3 and 4 = port 2 from the soundcard.
1,2,3 and 4 all go directly into an external 12 channel mixer.
If I wanted to, I can...