To me, the right way to look at a HOF candidate is how they measured up to their peers in the time that they played. With Diggs having a season where he lead the league in receptions and receiving yards and other seasons where he’s ranked very high, I think he’s on the way.
Since the Pro Football HOF, in its finite wisdom, guarantees that six eligible candidates will get elected each year, Hall of Very Good level players are often enshrined. That being said, I believe Diggs is a legit candidate.