In fairness, Martin added/expanded on 30 some tertiary characters with fluff chapters in books 4-5, which doesn't really translate well to screen. Jaime does very little in books 4-5 but he's a lead character. That means they gotta shoehorn his character somewhere. Likewise with Sansa/Olenna. In fact, I don't think Olenna is even in any book past ASOS.
Martin's writing/story development suffered as well in these books, IMO. The Oberyn Martell plotline tried too hard to be an echo of the Ned Stark story. Characters like Sansa, Bran got pushed to the wayside when their story arcs ended in 4 chapters, leaving lots of chapters for snooze Mereen politics, and Tyrion walking. AFFC and ADWD were far worse than ASOS, even though I enjoyed them.