Smith was last year handed a one-year suspension by Cricket for his involvement in the Sandpaper Gate fiasco, which also saw Cameron Bancroft and David Warner getting banned for nine months and one year, respectively.
Though the 30-year-old has been in top form in the ongoing Ashes and hit three centuries so far in Ashes 2019 helping Australia take an unassailable 2-1 lead in the five-match series, he has received harsh treatment from the English crowd throughout the tournament.
Smith has collected a total of 671 runs in five outings so far, nearly the double of what the second-best has in eight innings (Ben Stokes, 354 runs).
"I don't think you can forgive him," Harmison told talkSPORT.
"When you're known as a cheat and he is, I'm not going to sugar-coat it, the three guys cheated that's on your CV.
"You're marked and you take it to your grave.
"Whatever Steve Smith does, he'll always be remembered for what happened in South Africa."
Harmison added that is "something he's got to live with".
"I can't see anybody's opinion changing on Smith, Bancroft, or Warner. Because of what they've done, they've tarnished the game," the former England pacer said.
Smith recently took over the No. 1 ranking from Virat Kohli and has been applauded across the cricket fraternity for his heroics in the ongoing Ashes series.
( With inputs from IANS )
