the fact your even asking is I'll gamble and say it will bother you and you'll always want a matching set. I doubt I could stand having two different speakers.
sound wise who knows? technically?
but from a human standpoint...itd be like having two different shoes on. imo.