most helmets have two types of frontal airflow vents: crown vents, and visor vents. The upper vents are generally the crown vents that push air over your head. These are not likely the ones causing you the issues. The vent on the chin in normally a visor vent that pushes air up the visor to the top of the head (exhaust vents are normally on the top in the back) for fog and visor clearing. That does put air right in your face.
These vents can be (maybe even normally these days) combo vents that push to the visor and to he face and whatnot but they normally do not have separate shut offs for the visor air.