Had same loose interior mirror. Swivel ball was not the prob. Spring clip inside the mount was loose. Bent clip to solve prob. Hole in mount is NOT access to a set screw. It is used to rotate mount 90 deg to remove mount. Spring clip is held w/ a T20 torx screw. For ease of "adjustment" i.e. bending, remove clip. Used locking needle nose pliers to bend wings of clip toward center. Test fit until clip is snug. Then reattach clip to mount. Careful, mount looks like pot metal. May be easy to crossthread T20 screw. When clip is snug U will need a rod (I used shaft of a hex wrench) to rotate the mount/mirror into position.