Update:

I jusr rewired my head unit to use the RCA pre-outs rather than the speaker level outs on the wiring harness. This was all pretty easy. I modified the after-market harness so I can still use the original if I ever decide to put the factory head unit back in.

Here's what I've noticed so far. It now takes more volume on the head unit get get loud. For example, what used to be the volume listening level on 4 now needs to be set at about 15. This is fine by me because the normal listening level used to be 3 (out of 40) and 7-8 was very very loud. I never even attempted to go above 15. Now the normal listening level will probably be around 15 and it will get very loud at around 30+. CyberGreg, this is probably what you noticed as well.

Second, when listening to music at lower volume levels (cd and mp3 discs) there is MUCH less hiss/noise than there used to be. It used to be like listening to a cassette tape without dolby noise reduction. Now it's barely noticable.

Third, I can now turn the internal amp off in my head unit. (this may not be a feature of all head units). This makes the head unit use less power and according to the manual, produces better sound quality as a side affect. I didn't really notice this right away, but on some sub-sonic level it's probably true.

Anyway, I'm going to leave it the way it is now. I have much more control over the volume and the lack of noise/hiss is a reason within itself. Overall I'm very happy that I did this. Thanks for the advice everybody.