Yes you could just replace the bearing itself but as the rubber mounting needs to be removed to get the bearing of you might as well replace it at the same time.

Mine had a fault on the bearing and the rubber was giving way. At £18 for an aftermarket part its not going to break the bank.

Matt.