Because to most people, including the folks who wrote most dictionaries, a sport is simply a physical competition. You're the one with the absurdly specific definition.
I could go around claiming that a poem is only a poem if the author uses simile, metaphor, personification and onomatopoeia...doesn't mean I'm right.