The data you need is mostly not remote. A friend works at a software development company, they can use LLMs, but only local ones (local as in their datacenter) and it can only be trained on their code base). Customer service LLMs need to be trained on in-house material, not generic Internet sources.

The general advantage is that you know that you're not leaking information, because there's nowhere to leak it to. You know the exact input, because you provided it. You also get the benefit of being able to have on device encryption, the data is no good in the datacenter if it's encrypted.