I, and many others here have experienced the same issue. It is most often the breather covers(the small covers on top of the valve covers)leaking, or the valve covers themselves.
The small occasional drip can hit the exhaust manifold and cause the fumes. Whether this is your problem, or a leak somewhere else; I would locate and repair first, before contaminating a new charcoal filter(that's just me). Others have needed to clean out the cabin air intake area below the cowling, as a buildup of old rotting leaves and debris can also cause a foul odor. But yours sounds like engine fumes as you describe. Use the Forum search engine here for a tutorial on this issue. Good Luck w/ it.