If you are looking for a cheaper option than a full phone amp, Radio Shack sells a really neat little 1 to 3 headphone amplifier that is really cheap. I can't remember what it cost but it wasn't much. I originally got so my kids could both hear the mobile DVD player thing on long car trips. We started with a wye adapter, but that didn't work so well. (I've since figured out that if both are using exactly the same headphones (or better yet earbuds) a wye does work, but if they're different, one will tend to suck up more of the signal leaving one too loud and one too quiet.). Anyway, it works great, powered by a couple of double A batteries, so I don't know whether it will power big driver headphones or not, but for earbuds or the kind of middle of the road headphones I normally use, it works great. My kids have been out of the Disney movies in the backseat stage for years nown but I've used this little gizmo for exactly what you're asking about many many times. It doesn't have individual volume controls, so if you want that, you'd either need to go with the proper headphone amp or (my preferref option) earbuds with the little L-pad type volume controls build right into the cord.
Good luck.
J