Danimal, the noise that he's describing, sounds to me like the "wind buffeting" sound that you mentioned, I too have noticed it when I drive with just my drivers window down. Usually all I do to make it go away is "crack" open the passenger window just a little bit.
But 13k9 mentioned it happens when his windows are up.
As Damian pointed out, it could be the "interior air exhaust flaps" opening and closing, or maybe they are blocked shut. But if that's true, where is all the massive amounts of air coming from that's building up in the interior.
13k9, try driving at your normal speed with the windows up, blower fan off, and set your ventilation system on recirculate. If you don't hear anything, then lower just your drivers window and see if you notice the sound that you mentioned. (You should be able to hear it even a lower speeds too). If that's the same noise that you described, then crack open the passenger window to make it stop.
I don't know what to tell you to do to stop it from happening when the windows are up, unless try Damians suggestion and open up the interior exhaust flaps. But I wouldn't remove them, maybe just block one open to see if that fixes the problem with the windows up. The only problem with blocking them open, is you may get carbon monoxide from your exhaust inside the interior.
If the fluttering sound goes away with the windows up, and recirculating on, and the fan off, but with the flaps blocked open, then your getting alot of air coming in from another location, and you shouldn't be. Good Luck.