I've only had my black XF for 6 months, so I can't comment on how the color "holds up" but I will say that I wasn't keeping my XF in the garage, I would be much more frustrated with having bought a black car. I kept my XF outside for three days while working in the garage on a project and it accumulated some major dust just from sitting outside.

I keep my XF in the garage during the week and it always comes out looking exactly the way it went in...clean!

Someone else here at work drives a black limited XF roadster as a daily driver and he keeps it fairly clean. Sometimes there's some dust on the finish. But from twenty or thirty feet away, it always looks amazing. I believe it's an 05.