Ross hasn't been found guilty of anything yet, so people calling for their "removal" can't be listened to.
And you're correct, no owner has been forced to sell his team (Richardson volunteered) under Goodell's tenure.
Ross, by the way, has come out swinging, denying everything and pledging complete cooperation with the NFL investigation. He is a lawyer and an actual self-made real estate tycoon. Related Companies is a massive RE player in NYC and worldwide.
This could get exciting soon.
Fines, suspension, draft picks.