Well, actually, the wing is designed to provide approximately 356 Newtons (~80lb) of downward force at 80mph.

(Actually, I did a conversion error, 356 Newtons is more like 40lbs.)

Adding 80 lbs of "weight" will definitely have an impact on your fuel mileage to some degree.

The wing is there for a reason, albeit we may not be driving at speeds to make it matter but remember what happened to those early Audi TTs without a rear spoiler?