The real issue isn't obvious: clock skew and drift. It's be relatively easy to do with 4 S/PDIF sources that all used the *same master clock*, because you could then put the output stream together without needing extensive buffering or sample rate conversion. Unfortunately, some schmoe is going to plug in 2 from one unit, and two from another unit (with a slightly different clock frequency), and then sue you into the ground when they get pops and clicks...
For this product to be really viable, you *have* to handle the general case of asynchronous clocks, and that'd be somewhat more expensive and challenging. You have to use phase-locked loops and multiple sections of some pretty sophisticated FIFO.
But having said that, this is already a solved problem, and there's a product out there that does it. The Z-Sys z8.8a digital detangler can do this:
http://www.z-sys.com/pp_detm.html#z8 . It's a little on the pricey side (around a grand), but it has the SRC hardware that you need built in, and was designed around this problem. And though it may not seem like it, synchronizing clocks that are .000001% off (like with two units with different clock crystals) is really *exactly* the same problem as converting 44.1kHz to 48kHz, anyway...
I have the Z-Sys Optipatch (just like Bruce), because I just needed to route lightpipe at 24bits. It doesn't include the SRC hardware, so it can't do the combine/split trick. But it works like a champ- and was significantly cheaper than
the z8.8a.
In general, Glenn Zelniker (the Z in Z-Sys) seems to have a pretty good handle on shipping digital data around. I've talked to him on the phone quite a bit, and his background is in industrial DSP design, similar to mine. Multirate signal processing is just one small portion of that discipline. It was nice being able to talk to him nerd-to-nerd!
Anyhoo, standard disclaimer applies: I have no connection with them other than being a happy owner of an Optipatch, which I think is the greatest thing since sliced bread. But if you really need to do this, you really need to do it _right_- so this would be worth checking out...