I have found that shims on shafts can cause problems when the hole in the shim does not allow for the radius in the corner of the shaft. The shim is forced into the radius but never fits against the shaft face like it should. It looks like this could be that situation, but I am not familiar with the set up you have.

If shims are not required on the factory pulleys then they should not be required for any after market pulleys. Shims are to compensate for manufacturing errors, that means the maker says 'we do not throw out poorly made parts we ship a shim with it instead.'