This is the reason I worry about the eye in the sky and/or AI.
As the old adage says "there's holding on every play". Refs can be brutal but I get the feeling unless there was a way of policing it with some sort of tolerance, there will be a flag on every other, if not every, play.
If the eye in the sky is watching real time every individual player to see things and that's all, I think it could work. Giving them replay and slow-mo before the next play happens and then a flag comes in would absolutely kill the game and also take 5 hours to play.