As perhaps Marvin's most devoted fan on this board, I feel I must add my two cents here:
The easiest thing would be to get "The Marvin Gaye Collection", a 4 CD package that breaks down his work into a CD full of duets (w/ Tammi Terrell, Kim Weston, Mary Wells, Diana Ross), another CD featuring him doing ballads (includes a few standards like Mack the Knife, Too Young, and more-one of my favorites of the set), another that has live and unreleased material (such as his classic rendition of the Star Spangled Banner before the NBA All Star game in 1983), and one CD featuring 20 of his best hits called "20 Top 20s". The set has just about everything you'd want if you're a Marvin Gaye fan (like I am; he's my all-time favorite singer).
As for studio albums, What's Going On and Let's Get It On are classics, as are:
Midnight Love ("Turn On Some Music" is one of my all-time favorite Marvin Gaye songs)
Here, My Dear (made as part of a divorce settlement, it has some really nice gems on it that aren't very popular, but are good nonetheless)
Trouble Man (sorry movie, but a terrific album)
That's The Way Love Is (released a year before the landmark What's Going On, this one has a ton of great material, w/ the title track being one of the best songs of his career)
Happy hunting. You won't find any bad Marvin Gaye, so whatever you choose, I'm sure you won't be disappointed.