As far as I can remember to completely remove the mirror you have to remove the inner door panel and disconnect the wires to it and release the grommet.
Then on the outside you turn the mirror to the inner position, this exposes the two Torx screws which undone to release the mirror. This can be seen in the PDF on how to instal the indicating mirrors. This can be found in this sticky post
I