I just removed my "rear fascia" (aka rear bumper cover), as part of my new push to install a backup camera. I didn't fully plan on removing the fascia, but when my plastic wedge fell down into it (Doh!

), I couldn't resist.
The above pictures, and the FSM were helpful. But the FSM is not very complete or accurate. You (too) must remove the rear taillights and side markers (easy). Remove the two black plastic pans first from underneath -- don't leave them installed on the fascia like the FSM shows. And there were seven screws/nuts each, not four, including one from the wheel well.
There's also two 10 mm hex head screws on each side that aren't described in the 2004 or 2006 FSM -- they go down vertically through the top of the sides of the fascia -- you access them through the taillight holes. There's also a couple of rubber strips in the wheelwells that just pull off.
My 2005 didn't have the third (just off-center, below the license plate) hidden 10 mm nut as shown in the FSM, and implied in the above pictures. Removing the center interior trim (after the right side lower and sail panels), made the job easier, and would be necessary if you have the third hidden nut.
Here's some pics. If they don't show up well (or at all), go to my gallery and enlarge them:
Have it mostly reinstalled now -- is being easy. Just make sure both lower-sides slide forward onto their brackets, and realign the fascia/body seams as you tighten the fasteners.