Alright, just a final update - you can learn from my fail. I broke the hub down and noticed I had some wear on the back plate (looked like it had been polished). I was being too gentle when I replaced the seal and did not press or drive it to the correct depth, the very edge of the seal was contacting the edge of the dust shield once every full rotation.

I think the bigger issue was the inner bearing could potentionally have had a little slop due to not being snug against the seal. I think if left alone this would have caused bearing failure and spindle damage.