Signal multiplexing (blending) with a standard Y adapter?

[mindwave_21]

Is it possible to use a standard Y adapter that is a signal splitter to do the opposite? I have a 1/8 to 2x1/8 signal splitter (radio shack kind that you use to get 2 headphone outs from a single CD player output), a headphone amp, and my bassist's headphone out from his amp.

Here's what I'm doing...sorry I have no pics:
1. I connect the single splitter end to headphones (via a female-female 1/8 coupler)
2. I connect one of the 2 1/8 to the bass amp's headphone out (just the bass player).
3. I connect the other end of the 2-way to our firepod (has everybody, including the bass player).

With the headphones in and one of the 2 "inputs" connected, I get a signal, but when I connect both, I only get the bass amp's output. When I turn the headphone amp up, he gets very little of the signal, but it's super distorted.

Is this a gain mismatch, or something else?

Thanks!