There's a lot of waste heat put out by RTG. The one powering Perseverance has an output of about 110W but thermocouples have about 5% efficiency so there's ~1900W of heat you have to deal with. For a fridge the running power can be almost double that it's just completely silly on Earth except for odd places like the lighthouses along Russia's northern coast where there was no way to get a grid connection, not enough sun for solar, and suppling diesel constantly was impractical.