Not every post office has a "back," albeit many due.

Many of the ones with backs still have keyed fronts; apartment complexes are notorious for front-access break-ins.