Another aspect to consider is that given the sensitive nature of our work and access to what is often crown-jewel intellectual property, we have several clients who require we work from their offices. They are sometimes unwilling to cover travel costs. With a more geographically distributed base of employees we'd either have to eat the costs of bringing someone remote in for these projects, or rely more heavily on the now smaller subset of our staff that is present in these geographies. Neither is a desirable scenario.
Most of us like working in an office close to home, and don't like traveling a lot. It works for us.
Working remotely is a skill unto itself like any other and it must be developed and honed across an organization.
I guess what frustrates me is so few people see a need to develop and hone that skill. Some even actively argue against doing so.