The separate CueMix busses are really only good for .... well, separate mix busses as you would have with any other multi-bus mixer.
As far as being a computer audio interface, you will have to specify within your audio software as to which output/outputs to use for whatever track/tracks or audio software mixer output busses.
CueMix is essentially just software control of the MOTU's ability to be a 4 stereo bus mixer. Level attenuation even needs to be set prior to the inputs of the MOTU (with the exception of the two front preamps .... level attenuation is made via their gain knobs). CueMix gives no control over attenuation of the inputs to the recording software. You have to think of the 828mkII as a mixer that all of the inputs are feeding their signals to the computer via imaginary inserts that are pre fader of CueMix.
The only time you can get attenuation via CueMix is if you are using the output of MixBus 1 as an input to your recording software (selected within your recording software).
As far as you being able to get output to SPDIF by selecting that as the master output of CueMix bus1 ... I'd have to play around with mine, but it sounds as though you have somehow selected CueMix bus1 as the master output of your recording software thus allowing MixBus 1 to change the physical output that the signal is routed to. Or maybe that's just how the MOTU works. I've never tried changing the output of MixBus 1 to see if that effected the physical output that I selected as the output for my master output bus in my recording software.
I always just create another output bus in my recording software and assign it to use whichever outputs I want.