I'd say that balls thrown by lefty QBs are caught by players of the same team, who practice daily with that same QB.
Punt returners mostly catch balls from righty punters who produce a clockwise spin, so when they face a week where they have to catch from a left footed punter who imparts a counterclockwise spin, it's different from their normal. Also the greater height and distance of punts vs. throws magnifies the effect.
For further explanation, google "Coriolis effect" (joking)