Yeah as a boxing fan I can tell you the reason they have weight classes is to make more money, and not having 'little dudes fight big dudes' is just a byproduct.
There are so many exceptions to your example it's a waste of time keeping it going. If you honestly think Steve Smith, a notoriously tough, undersized, and ridiculously strong wideout famous for taking a ton of abuse going over the middle back when safeties could actually make contact, is losing a fight to a college QB...nope.