Yes obviously we aren't sitting here measuring the earth's rotation to set our clocks. The point is the meaning of the number remains the same. Seconds, minutes and hours have meaning, and the meaning of those measured amounts of times is directly related to some fraction of the rotation of the planet we are currently sitting on. The point was moving this system of time-keeping to some other planet would discard much of its relevancy to our every-day life, which matters a lot.