Cloudflare went down today and took half of the internet with it. How does one company have that much impact? Do you think that's concerning?

It just means the internet is built on a very flimsy stack of technologies and any of them failing causes huge downstream issues. We saw that with AWS, and now with Cloudflare.

It's only concerning if there are no alternatives, but as it stands there are other companies that all of these websites could have done a failover to when both AWS or Cloudflare went down. But they decided that their websites having a single point of failure was worth the risk over paying for having a proper backup system ready to go.

Relevant XKCD, as always:
XKCD//2347
- (Joke stolen from another post that's since been deleted, so reproduced here.)
- I like to think there was a specific person in Nebraska the author had in mind. The University there had a tap into the ARPANET back in the day and always had interesting projects going in that one wouldn't typically expect in Nebraska.
I now imagine all the websites to fail over to the same backup services, effectively ddosing them and creating a chain reaction :D
- Yeah! We call those "Cascading Failures"
  They're a nightmare! 😄
DNS doesn't fail over, unfortunately.
And azure also went down too
- So many people seem to have just forgotten the crowd strike outage, which halted air traffic for a day and stopped a not-insignificant amount of public infrastructure

I remember experts saying 5 or 10 years ago that the increased standardization and centralization of the internet would lead to more frequent and widespread internet blackouts.

First AWS, and now this. It looks like they're right.

Two things happen when we centralize. Doesn’t matter if it’s big business or infrastructure.
Profits go up for the controlling few
consumers get fucked.
We get fucked when things go wrong, the system fails, our data gets hacked, our power goes out, our rents go up, insurance rates go up… etc etc. MegaCorps all say sorry, give us 50¢ off our next purchase and a free credit check, and carry on while we eat the losses and increasing costs.
Don't forget the Azure/Intune outage not one week after AWS, too.
The outages are almost beginning to feel deliberate at this point.
- Are we just forgetting the gigantic Crowdstrike outage a year ago by Microsoft that halted air traffic for a full day?

I mean, the entire internet is owned by a few corporations. everything from the infrastructure to the entire DNS system is owned and controlled by corporations. in the case of DNS, it's even an american corporation, that so far has kept its hands off of things and supposedly has not been interfered with by the US government.

What’s the fear there, that they would figure out what domain names you are resolving?
- well, if the people in charge of DNS decided to do something to DNS, they just could and the whole world would be at their mercy for potentially months. with how everything is digital these days, by taking down DNS for a single country you could cripple their economy and many of their public services. that means power, water, infrastructure like bridges, their internet, banking, etc. basically, you name it, it probably uses the internet in some way, and if it uses the internet then chances are it uses DNS. now, eventually, people would work around it if given the chance, but if you do something like that it's probably happening right before a general invasion of their country.
  it's really bad that we have just one authority in charge of all that, especially one based in the US. with how authoritarian the US is getting, I fully expect DNS to be weaponized in some way at some point.
- I'm guessing the concern would be resolving them to the wrong address, either to censor or to serve disinformation.
- maybe you could hijack sessions by redirecting and capturing authentication i don't know im not a wizard my grandson is
- Try to do secure communication without that sweet domain mame... You can't!
  My thoughts is that they feel the need to control everything. And we all know how that goes usually...

.

Edit - cloudflare now says it was a misconfigured config, not a DOS attack as they initially reported

except you're getting it wrong, because it wasn't an attack, according to Cloudflare. they fucked up a config file on one of their systems and that caused a cascading effect of failures in one system after another.
it was quite literally not working as they intended.
- Most of the reporting I have seen suggests a massive traffic spike. Do you have some more information about the config file?
"30 minutes"- me when I lie on the internet. Where did you get that number? You realize we can check the news and see that big sites like x and chatgpt were down like 4 hours? Not only that, they said themselves it was not an attack but a misconfiguration. News were reporting it fixed around evening utc while the issue popped up around noon. That's not a 30 min outage and is a huge failure.

There was the Crowdstrike failure that tangled the airports last year, and the AWS outage that took out half the Internet just a few weeks ago. It seems like some one might be probing for vulnerabilities. One day, EVERYTHING might go down, for a while.

We'll get a chance to find out what it was like to read a book instead of a screen.

Yes, the increasing centralization of the internet is concerning, and the fact that companies have been vibecoding stuff increases the chances of stuff going wrong. And quality control and testing aren't a priority anymore, it's as if they're just chasing short term profits. Oh wait, they are.

Imma switch my services to Bunny CDN to decrease my reliance on a huge service. And its Slovenian, so that's pretty nice.

quality control and testing are not a priority anymore
We can see it with crowdstrike some times ago, they fucking rolled out a system-breaking update, this mean that they just builded it without testing!

Honestly about the same as I did with crowdstrike, the AWS outages. It's not a good idea, could lead to ruin, people won't diversify, Goto step one. It's easier to just have a sensible chuckle and move on at this point.

The service providers get 100% of their money all the same.

This causes endless amounts of laziness on their side, and quality goes to hell.

We are causing this laziness.

Unless we, their clients, hold them accountable, and make them feel the impact of their faults in their pockets, things will continue to get worse and worse.

The fact that Cloudflare controls half the web is concerning both for unintentional crashes like this, and for something even more insidious; what if they're coerced to cause an intentional outage should cyber war ever break out? An intentional outage for half the web in a cyber war would be devastating to put it nicely.

It even took down the very instance we're posting this on.

The snark of the following comment is not directed towards you, OP, but at the tech industry at large.

What I don't understand is why people are still surprised when this shit happens. Today, cloudflare takes down half the internet, last month it was AWS. Crowdstrike did it last year even more severely. Akamai has also caused major issues like this before, as has Google. M365/azure outages barely get reported on because they are so frequent. Yet, they are all still being used to hold up most of our infrastructure. Every single company I've done IT for has used at least one of these companies for critical infrastructure. There just aren't any other realistic options due to the refusal of non IT people to learn about IT.

If you try to use something other than one of the big companies, you're hit with one or more roadblocks.

You "don't have the budget" to selfhost. Bean counters would rather pay $100 a month indefinitely than $5k to buy new hardware that will save $1000 a month for years.
No approval for non giant corpo option, because using AWS is cheaper and has brand recognition. This is due to the same economics and myopia that caused Walmart to be one of the only places you can get groceries.
There is no other option. Every year that goes by, more small companies get gobbled up by big tech M&A. Unless your company opts to create its own implementation of a service/software, you're stuck with one of only a few options, even if you could get the approval to use something not run on big tech.
Even if you manage to jump all of the previous hurdles, the Internet connected software you're using probably relies on big tech infrastructure too. Every company has to navigate all of these hurdles for every saas/infrastructure implementation, and the only ones that successfully do it have to have leadership that not only understands why the decisions have to be made, but also need to be willing to accept the extra cost. Anyone that has dealt with upper management knows that this is exceptionally rare.

So what we are left with is a system that every professional knows is deeply broken and monopolized. The people that actually make the final decisions are largely ignorant and unwilling to invest money in fixing it, instead choosing short term savings and lack of commitment over long term security and continuity.

For a lot of people who would self host, $100 at a time is easier to get together than a few thousand at once.
- Not talking about individuals, homie. Talking about companies.
I hear where you are coming from, but I think your criticisms are misdirected. For the majority of businesses, using an infrastructure provider is a sensible decision that leads to greater security and stability in the long run for less money than trying to build the same thing on their own. This isn't a decision made out of stubbornness, laziness, or ignorance about IT. It's simply that it's the better option for each individual business.
But when most companies make the decision to use an infrastructure provider, outages and risks are centralized. As you pointed out, the services you rely on are likely to use a provider even if you don't use one, so this isn't a problem that a business can solve by buying a server and hiring an IT team. These massive failures aren't a sign that businesses need to make different decisions. It's a sign that the infrastructure providers must work harder and spend more money to improve their internal isolation.
When a bridge collapses because the pedestrians happen to walk in step with the resonant frequency of the bridge, we don't blame the pedestrians for walking incorrectly or for deciding to take the bridge instead of a boat. We blame the designer of the bridge for failing to account for the mundane stresses that the bridge is expected to sustain.

So, and I'm gonna pull my shameless plug ofc, but what about a decentralised internet?

Check out tenfingers or the sub (I put the weblink, is it !lemmy.world/c/tenfingers on lemmy browser apps?).

What about we take the internet back?

This seems interesting but I struggle to see how it helps, seems more like its for file sharing. Or is the suggestion that it could form the host side of a web ecosystem, with files for websites hosted in this decentralised way?
- Exactly.
  File sharing, a chat, backup systems, are just the low hanging fruit IMO. You can make a takedown safe "web" presence, with whatever you want on it. Decentralised, accessible with your PC off. And FOSS.
  You can try it out, the new "web" WIP, if you have a couple of minutes and access yo a PC (Linux works, would love to get feedback from other platforms).

I am, but what can we do about it? We don't run things.

Not only does it make things brittle, but it makes a smaller number of things for governments to intercept to scan all internet traffic.

You must not have been paying attention when that one Java programmer quit and took his code with him.

I mean... not only is it not very concerning, I barely noticed. If not for news about it here on fediverse, I might not have known. I guess I dont visit the corpo internet all that much.

I have been concerned recently that despite my best efforts I am still too attached to the corporately owned internet.
The fact that I felt no impact from this was a nice treat to start my week.

Being a good CDN is an expensive exercise that requires the ability to run POPs in many countries around the world.

Cloudflare captured the market by basically being simultaneously much cheaper, better distributed and ultimately better performing than the incumbents at the time (Akamai and Limelight IIRC)

The rest of the story is capitalism doing capitalist things

I didn't even notice that it went down.

It took down a fifth of the Internet, not half.
I found two websites that didn't work, that's it.
- A third of the “top 100” were in that 1/5th total. Most websites I personally wanted were down, including lemmy for me.

Most corporate IT hires fucking morons as admins :(

I've been a contractor most of my career and I can assure you, fair share of ID10Ts in my circles as well.
- Those are the ones that get occasionally hired as admins for someone's corporate IT ;)

I find it at least concerning for CloudFlare's change control process. Apparently some new traffic analysis config took half the web? Maybe test things a little more?

How does one company have that much impact?

Because they are a very good CDN and provide excellent DDos protection. They then expanded to do a whole host of other things, to the point where they do pretty much everything. Basically, they have become the first name most folks think of when they want to put something on the internet. A one stop shop for your web hosting needs. Wouldn't surprise me to learn they rent servers and VPS's as well.

Been seeing it in the selfhosting communities and subreddits for a while now. "Oh I want to put this selfhosted service on the internet. I should put it behind Cloudflare!" Most of the time it's not needed in that context.

Do you think that's concerning?

Well, they did just take out "half" of the internet today so.... In general, if it seems like "everyone" is using a single service, it's probably a good idea to see if an alternative exists and will suit your needs. Which reminds me, I should probably start looking for a replacement for Tailscale. They're starting to look a bit like Cloudflare to me, in the sense that "everyone", including myself tends to recommend them as a VPN.

It's good that it goes down once in a while, so that people notice.

I barely noticed

Amazon the other week, then this. Really does show how vulnerable much of the net is

People have other things to worry about. It's concerning but there is a barrage of shit going on that this barely registers. And companies will always choose what's cheap in the short term. They believe the risk of something going wrong is small enough to warrant the possibly large impact. It's like that everywhere: in the car industry, chocolate industry, clothing industry, and so on. There's always one seemingly small decision that could fuck up the entire company but isn't worth investing in in the short term.

I wish cloudfare (whatever it is and whatever it does) had more reach and went down for longer. For so long that competitors would be considered. But alas...

I think it's very concerning, but not exactly surprising. It makes sense that there ends up being market leaders for digital services, because they can provide economies of scale much more than traditional services. As another commentor pointed out, they're are alternatives, even just for back-up service providers, but most sites don't pay for them.

What was more personally distressing was I realised how much I rely on lemmy when my instance, and backup instance, both went down. I'm not sure where I'd go for immediate news, especially about something niche like "why is lemmy down?". I don't use other social media and I found myself checking r/redditalternatives just to see if there was some info about the shutdown. Obviously, it was useless because reddit...