I had the same problem when I fitted an aftermarket idler pulley, I found a metric spring washer (M16 or M20 I think) was just about the correct I/D to use as a spacer, ground a little from the I/D, and it fit perfectly to pull the pulley out from the block.