Thank you guys so much for your responses.
It's good to now understand that an exhaust fan is needed to get the hot air out of the closet, and that the fan should be as far away as possible.
@Slouching Raymond, thanks for the idea of an exhaust fan at the top of the chimney and/or a chimney...