It actually worries me what's going to happen to tools like GPT when it starts being influenced by commercial interests and manipulating people. What's going to happen when a sports drink manufacturer pays GPT to never ever ever mention water when people are asking about dehydration, but instead to pitch the sports drink? Are we gonna block all kinds of knowledge just because it interferes with some corporation making a profit? What happens when GPT starts promoting a political candidate or demoting another? Who controls which candidates GPT prefers?
What are we doing to protect GPT from this kind of outcome?
While I don't think that you're wrong about being concerned, I would suggest that we currently exist in a society where teens are held in rapture by TikTok, Google serves custom results full of ads and Amazon is only barely trying to purge fake reviews. Kelly Anne Conway can describe lies as "alternative facts" on television and be lauded for being "good" at her "job".