Bit depth refers to the smallest possible increments of volume change in a digital system. The bits refer to powers of 2.
A simple one bit system , for instance would have "2 to the 1st power" (sorry, don't know how to do exponents) possible levels. Since "2 to the 1st power" = 2, that means a 1 bit system would consist of audio that was either completely silent, or as loud as the system could handle - nothing in between.(Not very musical!)
So, a 2 bit system would be "2 to the 2nd power" or four levels of volume: Zero, medium-low, medium- high, and full volume. (Still not very musical!)
The higher the bit depth the more gradations of volume you get. Thus, a 16 bit system has "2 to the 16th power" amount of levels between total silence and the very loudest sounds. That comes to over 65,000 little slices of the volume spectrum. Sounds like a lot, and it is. That's why CDs sound pretty damn good.
But the human ear is an incredibly sensitive mechanism. If we upgrade to a 24 bit system (2 to the 24th power) it means that we are going to subdivide the smallest available volume increment in our 16 bit system another 256 times! And you know what? Most of us can flat out hear that difference - we hear better stereo imaging, smoother fades, less grainy reverb tails, etc. With 24 bit we also get an overall increase in dynamic range, which means you can record at lower levels without worrying about approaching the noise floor.
Now, this discussion is assuming equivalent hardware quality, which is not always the case. It is possible to buy "cheap" 24 bit systems that sound crappier than good 16 bit systems, because of clocking, converters, filters, etc. So "24 bit" by itself is not necessarily a guarantee of a superior system.
The final question usually asked is: why bother recording at 24 bit if it's all going to end up on a sixteen bit CD anyway?
The details may have to be in another post (I'm wiped!) but the short answer is it is always better to record and mix at a higher bit depth, and then reduce to 16 bits at the mastering stage, than to do the whole project at 16 bits. The long discussion will involve concepts like dithering and rounding errors... but, sorry, it will have to be another day... zzzzzzz