The worst remote experience I've had was when leadership was not bought-in.

A butts-in-seats manager or VPE or whatever can really ruin it. The lack of trust is really toxic.