Unless you are super careful, the result of taking services out of rotation as they become overloaded, very often results in cascading failures as each server in turn falls over because more increasing amount of traffic is directed at healthy ones.

You need to be super careful that a service never allows more concurrency than it can handle and fast fails requests when it is overloaded. Otherwise, tcp queues back up and the server ends up only working on requests that are too old to be useful.