I'd just give it a good cleaning with M39 Heavy Duty Vinyl Cleaner and then dress it with M40 or 303 Products stuff. Taking an abrasive to the surface, even a lufa type thingy, may open you up to more problems. If you don't get satisfaction out of a good cleaning and dressing, I'd replace the parts.