Are these LLM models going the way of search and social media? Optimize them for selling product, optimize them for engagement (to sell products). They become ad machines - and it's gonna be really hard to tell. There is already evidence [1] that you can optimize LLMs for engagement. Hopefully open source can keep this in check... but I'm not optimistic.

[1] https://arxiv.org/abs/2303.06135 - Rewarding Chatbots for Real-World Engagement with Millions of Users