While it is wise to change any connections while in an un-powered state, I will add that I've done it many times and have suffered no ill effect.
When dealing with an input, be certain to have all your gain staging set to INF (turned all the way down).
Likewise with outputs, turn the output of your devices all the way down prior to making any connection.
Grated, constantly plugging and unplugging can and will wear the contacts of the inputs/outputs though.
In regard to your Delta 44's outputs .... you are aware that the 4 outputs are Mono right?
You won't get a stereo signal out of just one of the outputs.
Also, you wont get a very good signal to your headphone directly off of the outputs.
You would do better by using two of the outputs to send the stereo signal to a HiFi stereo with a headphone output (like had been mentioned) or use a cheap dedicated headphone amp such as
this one using an
insert cable to get two of the mono outputs to a single stereo input.
HTH