Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

"AWS' availability zone isolation is better than the other cloud providers."

Not better than all of them.

A geo-redundant rsync.net account exists in two different states (or countries) - for instance, primary in Fremont[1] and secondary in Denver.

"S3 even operates at a scale where we could detect "bitrot""

That is not a function of scale. My personal server running ZFS detects bitrot just fine - and the scale involved is tiny.

[1] he.net headquarters



Backing up across two different regions is possible for any provider with two "regions" but requires either doubling your storage footprint or accepting a latency hit because you have to make a roundtrip from Fremont to Denver.

The neat thing about AWS' AZ architecture is that it's a sweet spot in the middle. They're far enough apart for good isolation, which provides durability and availability, but close enough that the network round trip time is negligible compared to the disk seek.

Re: bit rot, I mean the frequency of events. If you've got a few disks, you may see one flip every couple years. They happen frequently enough in S3 that you can have expectations about the arrival rate and alarm when that deviates from expectations.


> The neat thing about AWS' AZ architecture is that it's a sweet spot in the middle

What may be less of a sweet spot is AWS' pricing.


Sending the data to /dev/null is the cheapest option if that’s all you care about.


Seems the snark detector just went off :)

Back on topic, I'd hope all of us would expect value for money for any and all services we recommend or purchase. Search for "site:news.ycombinator.com Away From AWS" to find dozens of discussions on how to save money by leaving AWS.

EDIT: just one article of the many I've read recently:

"What I’ve always found surprising about egress is just how expensive it is. On AWS, downloading a file from S3 to your computer once costs 4 times more than storing it for an entire month"

https://robaboukhalil.medium.com/youre-paying-too-much-for-e...


And that is egress which works as expected, unlike the AWS S3 denial of wallet attack...: https://news.ycombinator.com/item?id=39625029


> They're far enough apart for good isolation, which provides durability and availability

It can't possibly be enough for critical data though, right? I'm guessing a fire in 1 is unlikely to spread to another, but could it affect the availability of another? What about a deliberate attack on the DCs or the utilities supplying the DCs?


> but could it affect the availability of another

Availability is a different beast than durability. I think people are paranoid here about durability instead of availability.

S3 advertises four nines availability and 12 nines durability.


Yes, if a terrorist blows up all of the several Amazon DCs holding your data, your data will be lost. This is true no matter how many DCs are holding your data, who owns them, or where they are. You can improve your chances, of course.

There have been region-wide availability outages before. They're pretty rare and make worldwide news media due to how much of the internet they take out. I don't think there's been S3 data loss since they got serious about preventing S3 data loss.


> the network round trip time is negligible compared to the disk seek

Only for spinning rust, right?


Yes, which is what all the hyperscalers use for object storage. HDD seek time is ~10ms. Inter-az network latency is a few hundred micros.



Yes, but S3 has single region redundancy that is better than GCP. Your data in two AZs in one region is in two physically separate buildings. So multi-region is less important to durability.


Agree.

> S3 even operates at a scale where we could detect "bitrot" - random bit flips caused by gamma rays hitting a hard drive platter (roughly one per second across trillions of objects iirc).

I would expect any cloud provider to be able to detect bitrot these days.


I think the point the OP was trying to make is that they regularly detected bitrot due to their scale, not that they were merely capable of doing so.


Ah, thank you. This makes more sense. And I think I remember reading about it once. Apologies for the misinterpretation!


Everyone with significant scale and decent software regularly detects bitrot.


How does the latest ZFS bug impact your bitrot statement?

I mean, technically it’s not bitrot if zeros were accidentally written out instead of data.


Probably none because they didn't update to the exact version that had the bug




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: