I can agree with this. I do think Mayweather and Pacman are recognizable to far more than 10% of sports fans though. Their HBO specials alone have generated a great deal of notoriety.
IMO, its a combination of quite a few different things.
1. Boxing doesn't have a commish or governing entity. Therefore a league or association cannot lose money or become shamed. Which actually makes for less of a story, given the lack of punishment or suspension.
2. Boxers routinely say stupid things, like you pointed out. Very few listen I suppose.
3. Hateful remarks are less of a story, if it isn't against whites and blacks. This is where the inconsistency lies. For whatever reason all the folks who get their panties in a bunch over idiotic remarks, seem to fall silent when it comes to certain groups. If Floyd goes on camera and starts throwing out cracker this and whitey that, rest assure its all over the news.
It's not exactly a thrilling topic but i find it interesting.