my
guess is that the bike has been jetted richer in the idle circuit to compensate for the Kleen system in operation. when you removed the air source with the block off plate, the bike runs richer, which would account for the lower idling.
like I said, pull the plugs to get an idea if your bike is running rich (plugs will be black).
otherwise, I would put the Kleen system back on and live with it. It seems your bike ran well with it on.
I would not simply put a filter on the hose leading directly from the "chimney" as that would supply extra air
all the time to the exhaust flow, which
might lead to a too lean condition at other throttle positions other than at idle, which that switch controls.