Fr I think he was upset about Suguru's parents, because just the fact that he mentioned them is significant. But I think more than anything he felt rejected and abandoned. Suguru did something that meant he could never be a part of the Jujutsu world. And Gojo couldn't leave it.