This is the problem with football when compared to so many other sports. There are so many ways for a player to draw a penalty that one can be thrown on literally any play. A ref could throw a flag if he wanted to without seeing anything, then say "holding on player XX" and a replay might show that there was a borderline call. Same with pass interference, offensive pass interference, the list goes on. I think it's the fact that they just aren't consistent that people get so aggrieved. When it isn't consistent it makes them stand out as being thrown on purpose to help the team gaining the yardage.