Amplifiers also have a characteristic called "Damping Factor". If you Thevenise the amplifier output equivalent circuit, you have a voltage source with a series impedance. The ratio of speaker impedance to amplifier output impedance is the damping factor. Damping stops the speaker cone from waffling. Typically, you want the damping factor as high as possible. Numbers in the 50 to 80 range are typical. So, if your headphone is 8 ohms, a damping factor of 50 would imply an amplifier output impedance of 8/50, or 0.16 ohms. Your 100 ohm series resistor just killed your damping factor. Forget about getting a nice tight bass sound out of it.