I've experienced the same thing, exept I thought the cold air was coming from the gear shift area (I have an automatic). It's almost like the car flexes and cold air comes in from the outside.