When a woman tells you size is not important, she's lying. When an audio engineer tells you that size is not important, he is telling the truth (unless it's a female audio engineer, then it starts getting confusing...)
Your apartment is going to mess with your bass response (both up and down) regardless of whether you have 5" or 15" woofers. And a 6" woofer with a 18oz magnet, carbon fiber cone, and long throw in a well-designed cabinet hooked to a well-matched amplifier is going to give you better bass than a 10" woofer with a 12oz magnet, resin impregnated cone, and short throw in an average cabinet design hooked up to an underpowered amplifier.
I have heard 6-inchers that were tight and crisp down to 40Hz, and I have heard 6 inchers that spewed nothing but mud below 70Hz. I have heard 18 inchers that would rock your world, and 18 inchers that don't rock anything but back and forth.
Don't worry about size. Pick what works for you ears and lets you make a mix that works for everybody else's. That's all that matters.
G.
