I used to go everywhere with my Panniers on, 42L keyless ones, don't think I took them off for about a year, filtering was never an issue!! you just get used to them being there.
Did become a slight pain in the arse when in very heavy city centre traffic, but still managed fairly well :D