I think it's all about how good the composer is/how good the arrangement is. Symphonic bass drum right next to a flute playing at the low end of its register might not be a good idea for instance, unless you don't really feel like hearing the flute. That said, it can all boil down to the performers. For instance, if you're working on something that has a passage, say for French Horn, and the horn player never practiced, then uh oh....
I recently wrote a piece for flute, clarinet, bass clarinet, French horn, violin, and three sopranos. We rehearsed the piece as I was composing it, so it was written for that particular ensemble (and their level of difficulty), but I think they had a bit too much on their plate and weren't as serious about practicing as they needed to be. We only performed two movements and, well...it's almost unlistenable. So in that context, almost NOTHING goes together well!