99 times out of 100 there's a castle nut under there that holds the hub to the spindle. It also covers the bearing to stop dirt from getting into the grease. You could just hammer a dent into it so your new wheel caps will fit properly. Definitely wouldn't suggest leaving it off.