It's not ludicrous. Mahomes is probably more consistent of a player, but they are very close in talent level. Mahomes has for sure done more in the league, including winning a Superbowl and an MVP. So he has that going for him.
And then there is stuff like leadership, likeability, temperament, competitiveness, etc. I like Josh Allen more than Mahomes in all those areas. And that is part of why Allen is so endearing to Bills fans. They like him as a person, as a leader, etc. That's why we wouldn't trade him for Mahomes.
If it was reversed and KC had Allen, I'm sure they'd feel the same and Bills fans would similarly not want to trade Mahomes. Each fanbase likes their guy.