AWS deleted my 10-year account and all data without warning
AWS deleted my 10-year account and all data without warning

AWS deleted my 10-year account and all data without warning

AWS deleted my 10-year account and all data without warning
AWS deleted my 10-year account and all data without warning
The cloud is, and always has been, merely other people’s computers.
Their only legitimate use case is as disposable, transient, dumb nodes and synapses of a system you retain control and agency of.
It's truly amazing that Amazon is able to convince anyone that it's a good idea to store valuable business data on their computers
People are so eager to relinquish control of their business, even their life, for the tiniest bit of convenience.
Before anyone says “you put all your eggs in one basket,” let me be clear: I didn’t. I put them in one provider, with what should have been bulletproof redundancy
This shouldn't happen and the OOP clearly knows what he was doing but putting everything in a single provider with multiple services clearly is not redundancy.
Multi-cloud is a significant amount of effort to pull off.
Being on one cloud provider across multiple regions is often plenty of redundancy.
Being available across multiple cloud providers is really REALLY difficult
Multi-cloud is difficult, that's true. But having backups outside the single cloud is easy.
That way if your cloud provider pulls the plug, you will have to reconfigure everything but at least your data stays intact.
To be able to recover from something like that you don't need multiple working cloud setups. You just need backups, so that in an event like OOP's, you spend a few weeks rebuilding the configurations instead of years rebuilding your projects.
In terms of service availability, yes.
In terms of data backup, no.
The author put it well:
What if you have petabytes of data? How do you backup a backup? What happens when that backup contains HIPAA-protected information or client data? The whole promise of cloud computing collapses into complexity.
Multi-region cloud computing is already difficult and expensive enough, multi-cloud is not only technically complex but financially and legally fraught with uncertainties. At that point you're giving up so much of the promise of cloud computing that you might as well rent rack space somewhere, install bare-metal infra, and pay someone to drive there to manually backup to tape every 3 months.
This level of technical purity is economically unfeasible for virtually everyone, that's the whole point of paying a vendor to deal with it for us. And you know who doesn't need to put up with the insane overhead of multi-cloud setups? That's right, Amazon, Microsoft, and Google, who will be getting paid for hosting everyone else's multi-cloud setups while they get to run their huge infra on their own cloud without fear. The last thing GAFAM competitors - especially OSS projects - need is even fewer economies of scale.
Stop with the victim-blaming, this blunder is squarely on AWS.
Yeah this is the right take, IMO.
Good on you to those of you who actually do multi-cloud backups.
Even if this was just a loss of their infrastructure, it would be catastrophic to any company without good infrastructure as code practices.
Not to mention the downtime.
This sure as shit doesn’t look “customer obsessed”.
Yeah, but he was paying the provider for providing the redundancy on this level.
We shouldn't blame him on the technical level for what is a problem with the provider on the organizational level. If this had happened to a Fortune 500 company, Amazon would have had an army of lawyers descend on them.
Before anyone says “you put all your eggs in one basket,” let me be clear: I didn’t. I put them in one provider, with what should have been bulletproof redundancy
Tldr; "I didn't put all my eggs in one basket. I put them into multiple smaller baskets and put those into one basket. That's still more than one basket right? ....right!?"
Platform decay strikes again.
Bezos and the shareholders weren't happy with AWS already being one of the single most profitable products in human history, because capitalism demands unending infinite year-on-year growth.
Whenever you hear them cry that capitalism breeds innovation, the kinds of innovation they're talking about are this shitty AI profit-maximising algorithm that created OOP's problem. This isn't a crazy conspiracy theory either; I've consulted in the software dev teams of dozens of major multinationals and the projects were always, without exception, some variant on "how can we replace people" or "how can we reduce costs by doing something slightly worse".
I've consulted in the software dev teams of dozens of major multinationals and the projects were always, without exception, some variant on "how can we replace people" or "how can we reduce costs by doing something slightly worse".
Always might be an overstatement, but this has been true over the past couple years for myself and the people I know at these companies. Especially right now - upper management seems to be deluded into thinking that LLMs can do anything, or more likely, they're just trying to sell hype like everyone else just to raise the stock price.
Trust me, they never delete your data, just your account.
If they never deleted any data of close accounts, wouldn't that be a lot of wasted storage space? Wouldn't it be way too much data?
Sure, but they'll never make the effort to let you access your data.
Basically "try to sue us"
As an ex AWS employee, you are so full of shit, it's leaking out of my screen. It also shows a low level of technical literacy imo.
This is a really interesting read, but this dude missed a major component of best practices, which is that your architecture needs to be able to respond to a proper disaster, which includes Amazon just dropping out of the sky and nuking your entire account.
Frankly, it's shocking that he didn't have local copies or a home server that he kept backups too. I've seen some people mention that a multi-cloud architecture is hard to set up, which is true, it's also expensive, but I don't think it would be super hard to set up, like, a blob storage in Azure, or a Google Cloud Storage, to just keep backups of whatever you're working on. We should always keep in mind that our accounts getting locked is always a possibility.
It's kinda weird, normally when you hear about things like this, it's the other way around where somebody was running a major production component on personal infrastructure and couldn't handle the bare metal.
Yeah the 3 2 1 backup strategy involves local backups
and why you should never trust cloud providers with your only copy of anything
well, yeah. Even the old 3-2-1 backup policy is against that.
More reason that other countries developing smaller, regional cloud providers is a good thing. Part of the reason AWS thinks they can get away with this is that there are 2-3 other providers they compete with, and moving is onerous. If there were 200+ cloud providers, there would almost certainly be a standard set of tools and much better customer service.
Well, at least for the OP, they made his moving really easy
A fair point. Kind of like an arsonist could make my next move really easy.
This stuff terrifies me. I'm de-googling as fast as I can and reviewing all my local backups plus add encryption to what stays in the cloud.
i have:
cryptsetup luksOpen
method.the password for encryption is a three-word sequence, example "apple broccoli joanna", and i store a paper on it with the three questions "my favorite food, first vegetables i planted, first girl i liked" or sth
Fuck, this is bad.
Mr Bezos doesn't give a shit about you.
Never has. Never will.
Ruining your entire life doesn't bother him in the slightest.
He doesn't even give a shit about your money,
he has incomprehensibly vast amounts of money
and you don't make the tiniest molecule of difference to his life.
All he gives a shit about is
extracting
the largest possible amount of money
on average
from the largest possible pool of people
and businesses.
So you don't matter to him. Not even slightly.
He cares much, much more about a worm he's never seen that lives under his lawn than he cares about you.
And he doesn't give a fuck about the worm.
Quit giving Bezos your money, everyone. He doesn't need it. You do.
Not that I don't think Bezos is a complete piece of shit, but he is no longer the CEO of Amazon.
Anyway, fuck Jeffrey Bezos.
I think he's now chairman and certainly still profits from Amazon.
Ngl this guy comes off as a massive schizo and I dont believe the story is how he makes it out to be.
I laughed at the part where he went and cried to Grok and Chatgpt.
JFC this poor dev.
Luckily many of my jobs or clients don't like AWS and have gone elsewhere. One of my projects is self-hosted (with an off-site backup).
I wish this person the best moving forward, I didn't recommend AWS before, but I'll definitely make sure to push against it now.
It's interesting that LLMs emotionally saved them, allowing them to bounce back from a destructive to a constructive mindset.
Reading another post of theirs, they seem to really love AI. Albeit in that post, it feels to me like they took AI responses too literally, with too much meaning (as if sentient, or ignoring potential training bias, etc).
Yeah thats what happens when you host your shit on some corporate "cloud", but will they learn from this and move to self hosting, or will they just find some smaller provider who can do the same rug pull again.
Do you mean self-host on your own hardware and infrastructure? At home? Otherwise you'd still be dependent on the server or infrastructure providers.
Do you mean self-host on your own hardware and infrastructure? At home?
Ideally yes, or alternatively if you do need to depend on other people's infrastructure, choose companies that aren't massive multi-billion dollar operations that have zero concerns about you or keeping your business.
If my years have taught me anything, it's the latter.
Probably some of both
Tha's what happens when you dance with the devil.
So, 3-2-1 backup, with other cloud service 🫤
I really need to start backups of my S3 bucket and my Hetzner Server to a local hard drive.
And Google locks entire accounts, including Google Voice, Fi the phone service, email, drive, YouTube, everything cause your kid's pediatrician needed a photo for a telehealth visit.
Just stop thinking your shit is safe in the "cloud" people. It isn't. And trusting in a single company to have your back, or even care about you is naive at best.