I totally agree. And not only that, but a hefty percentage of Middle America was not really interested in seeing Best Picture go to a couple of gay cowboys, Best Actor go to someone playing "an effete New York intellectual," or Best Actress go to someone playing a transsexual. As it turned out, only Hoffman won among the three, but Brokeback and Felicity Huffman got quite a bit of pre-Oscar press (not to mention most of the pre-Oscar awards).
I'm not saying that makes it right. I'm just saying that the pre-Oscar buildup, especially for Brokeback, didn't do the ratings any favors. Jon Stewart had absolutely nothing to do with whether people watched on Sunday night.