You really need to qualify what it is you want to do with the drill press. How much precision do you need, what size holes will you be drilling, what materials will you be working with. If you plan to drill holes larger than 3/8" in steel then you need a fairly ridged machine. The throat depth will determine how big an object you can get under the press. If you're doing a lot of small holes, especially in plastic or fiberglass you need high RPM (smaller the drill the faster it needs to spin). The percision of the bearings and chuck are also more important for small tight tolerance work. If you're changing sizes and materials a lot in the same project look at how difficult it is to change speeds (typically a belt) and how much fine control you have over it. A bench model, if it works for you will be cheaper and more stable.
In general though I agree that you don't need to spend a lot on a drill press. Put the savings into better tooling. But on the other hand don't buy less than you need.