You remember, it's a pretty simple identity from the national income accounts: S-I = G-T + CAB (current account)
Assuming CAB =0, the public sector's deficit (G>T) creates a private sector surplus (S>I). The Household and Corporate sector's savings (yes, profits tend to rise) are increased by the size of the deficit.
You are referring to the use of deficits to scare people into believing "social programs need to be cut because we can't afford them."
Deficits can matter, but not in the way most people believe.