There is no easy answer for your question. It's not so much about how many instruments play at a time as it is how many notes at a time,in which octaves and by what instruments. As an arranger you have to know what you want to achieve musically and sonically. 3 saxes and a clarinet will sound like Glen Miller's band (if you dont know who he was, maybe you should check him out.
Guitar ,Bass ,Kbd and drums can sound like a lot of different groups, depending on what instrument has which note(s) If there is a lot of movement by some of the instruments you may want to simplify what's going on among the other instruments. The average untrained listener can track 3 parts at a time. A trained and experienced listener can track 8 at a time . By track I mean that they can follow the part and identify the interactions. Even symphonic conductors need scores to look at to keep up but it takes an experienced ear to be able to tell if the players are doing it right.
Sorry 'bout the rant, hope this gives you some new ideas to think about.
chazba