Dave, I know what you mean, but I think it depends on the type of tape.
I placed a rectangle of adhesive-backed Velcro on the glove box door 3 years ago for my cell phone.
I removed it ~ 1 month ago. No damage. Not even a mark where it was.
The "adhesive" on the back was more of a tacky plastic than a glue. Do you know what I mean?
This is what I used. The "fuzzy" side. Which would work for the OP.
https://www.amazon.com/VELCRO-Brand-...5%3A2470955011