Hey all, finally got the time and money to take care of my 04. I've been neglecting her for awhile, but after the RCM/CPS failed, I didn't have a choice.
After browsing around and seeing this thread, went out to pull the rubbers... and sure enough, the rust was bad. Only 50k miles.
Just posting to say, after I bit of research, I decided to:
1) Wire brush off what rust that I could
2) Clean the areas with 2 coats of Naval Jelly (you can find this at lowe's/home depot)
3) Apply 2 coats of Rust Bullet (had to order from amazon, the tiny can was more than enough)
4) Let it dry all day with fans before replacing the seals

Some tips:
Use proper protection (gloves, trash clothes, drop cloth) while painting the Rust Bullet. It does NOT come off skin for a long time, and it's permanent on everything else.
Have a wet rag ready in case you get any on the rest of the car.
I used the butt end of an old metal spoon to push the seals back in. It's not easy to do it right, especially on the corners.