Only one game: Bills at Philly. I personally did not have a problem, but a friend of mine did. After the game, a woman and her extremely drunk boyfriend saw my buddy and his Bills shirt (the Bills won 10-7, IIRC). She kept egging on her boyfriend to beat the hell out of my buddy for being a Bills fan. Fortunately, this guy was too drunk to do anything and was substantially smaller than my buddy and nothing happened. The woman was not happy that nothing happened.
Otherwise, no big deal. Like previous posters have said: if you behave yourself, you will be fine. When you get obnoxious about the Bills in their stadium, you are in trouble.