It is usually safe to SPLIT a signal via a Y-cable or other straight-wire means like a patchbay mult.
It is almost never safe to MIX two signals via those straight-wire connections. It might not hurt anybody, but it usually won't sound very good.
I'm not sure I'd want to be jacking in and out of the inputs connected directly to my speakers without any way of protecting against pops, but if you're using patchbay, you should make sure that only one of the interfaces is connected to the speakers at any one time.
I use a headphone amplifier to select between my two computers and route them to a couple different sets of speakers as well as headphones. It works really well (well, the switches are flakey, but...) even though it's not balanced between the HA and the PA.