just the rubber, the rubber sits on metal...you really do have to pull but once it budges a little it comes right off. You could try sticking a small flat stong piece of metal between the rubber and the metal to pry it away a little, but be careful.