You can clean it but that is dry debris. Sounds like you might have some mold going on there. The filter is easy to do yourself either way and probably way cheaper than what that dealer is quoting you. I'd buy one from parts and do it myself. I've had mine out and cleaned it. its a little tough to get to but with a little patience you can save a good del of money. Somebody here I'm sure will know the part number.
After you have it done or do it yourself try running your cabin fan on high for a few minutes before you shut your car off without the AC on. I've used this trick on my PT's and my Crossfire and never had an odor problem.

Pat