I think it is because Dan Brown had a lots of factual material written into it, from the Dead Sea Scrolls, burning of documents and heretics, the transition of western religion from paganism to Christianity, the Priory of Scion, to the method of selection of which Gospels would comprise the modern New Testament.
Because he did such a damned good job on historical background, some mistake his drama for fact.
This is almost tame compared to what goes on in the prequel, Angels and Demons, as far as the Vatican!