If a bunch of miners flee, the expected profit for the remaining miners actually increases (marginally). Assume instant network propagation, the probability that a given hash will result in a block is constant. Since miners fleeing does not decrease the hash power of the miners that choice to stay, they have the same hashrate, so the rate at which they mine blocks is constant, and their expected profit is unaffected.
If we acknowledge the propagation delay, their expected profit increases slightly because of the reduction in the probability that their blocks would be orphaned.
Additionally, the increased congestion would likely increase the transaction fees, further increasing the per-block profits.
Once the network readjusts and the difficulty decreases, the remaining miners would see a windfall profit.
The potential death-spiral would only come if the temporary increase in delay causes the value of bitcoin to fall, in which case more miners will likely flee. If this happens fast enough, the blockchain may never reach its next adjustment.
If we acknowledge the propagation delay, their expected profit increases slightly because of the reduction in the probability that their blocks would be orphaned.
Additionally, the increased congestion would likely increase the transaction fees, further increasing the per-block profits.
Once the network readjusts and the difficulty decreases, the remaining miners would see a windfall profit.
The potential death-spiral would only come if the temporary increase in delay causes the value of bitcoin to fall, in which case more miners will likely flee. If this happens fast enough, the blockchain may never reach its next adjustment.