This matters because the "Return-path" (bounce address) usually needs to be on the sending server for email to go out.
If you have bounce enabled, you must make sure the "bounce to" address is set according to this rule and the Admin Email doesn't matter.