U don't need to remove the door panel. there should be a plastic cover on the opposite of the mirror. Pry it out with a flat head screw driver and u should now see the bolts fastening the mirror body. Be careful not to break the wires when take the mirror out.
Good luck