this post was submitted on 19 Jul 2024
354 points (95.6% liked)

linuxmemes

20629 readers
776 users here now

I use Arch btw


Sister communities:

Community rules

  1. Follow the site-wide rules and code of conduct
  2. Be civil
  3. Post Linux-related content
  4. No recent reposts

Please report posts and comments that break these rules!

founded 1 year ago
MODERATORS
 

A global IT outage has caused chaos at airports, banks, railways andbusinesses around the world as a wide range of services were taken offline and millions of people were affected.

In one of the most widespread IT crashes ever to hit companies and institutions globally, air transport ground to a halt, hospitals were affected and large numbers of workers were unable to access their computers. In the UK Sky News was taken off air temporarily and the NHS GP booking system was down.

Microsoft’s Windows service was at the centre of the outage, with experts linking the problem to a software update from cybersecurity firm Crowdstrike that has affected computer systems around the world. Experts said the outage could take days from which to recover because every PC may have to be fixed manually.

Overnight, Microsoft confirmed it was investigating an issue with its services and apps, with the organisation’s service health website warning of “service degradation” that meant users may not be able to access many of the company’s most popular services, used by millions of business and people around the world.

Among the affected firms are Ryanair, Europe’s largest airline, which said on its website: “Potential disruptions across the network (Fri 19 July) due to a global third party system outage … We advise passengers to arrive at the airport three hours in advance of their flight to avoid any disruptions.”

https://www.theguardian.com/australia-news/article/2024/jul/19/microsoft-windows-pcs-outage-blue-screen-of-death

top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 24 points 1 month ago (2 children)

What amazes me is that so many big companies still use windows in critical core infrastructure.

Windows endpoints is one thing, but anyone using windows servers and MSSQL for mission critical application stacks need to be hit with the modernization hammer.

And then on top of that, they do not have a test rollout of any changes in a test environment, before rolling it out in the production stack.

Good luck to all the engineers in the trenches, having to fix the mistakes of their leadership.

[–] [email protected] 2 points 1 month ago

I've not used crowdstrike, but looks like a part of the pitch is "cloud managed", which often implies that the vendor takes care of everything, including updates. Particularly since they market it as a security solution, they weld likely emphasize that they can update rapidly enough to keep up with security attacks that move very quickly because they don't care about "risk".

[–] [email protected] 11 points 1 month ago (2 children)

There are many, many, many specialized enterprise applications out there that are windows only.

[–] [email protected] 3 points 1 month ago

Way too many. It's not the 90's or early 2000s anymore.

[–] [email protected] 12 points 1 month ago

not when it comes to server software. In that regard, linux is infront.

[–] [email protected] 7 points 1 month ago (2 children)

What does the issue do?

My first company I worked for used crowdstrike. Does it think the computer is infected and locking them down?

[–] [email protected] 8 points 1 month ago
[–] [email protected] 15 points 1 month ago

They pushed a driver update. That update is broken and causes the bootup sequence to fail.

[–] [email protected] 18 points 1 month ago* (last edited 1 month ago)

We did all that work in 1999 just for... this ??

goes back to writing cobol

[–] [email protected] 32 points 1 month ago (3 children)

.... my work uses Crowdstrike

I didn't see any issues rise up yesterday. Is today gonna be a bad day?

[–] [email protected] 8 points 1 month ago

This occurred overnight around 5am UTC/1am EDT. CS checks in once an hour, so some machines escaped the bad update. If your machines were totally off overnight, consider yourself lucky

[–] [email protected] 33 points 1 month ago

The second I read this post my phone started blowing up. Good luck brother.

[–] [email protected] 22 points 1 month ago (1 children)

I made an announcement on our Teams channel, and its blowing the fuck up.... today is going to be a bad day :(

[–] [email protected] 15 points 1 month ago
[–] [email protected] 37 points 1 month ago* (last edited 1 month ago) (4 children)

Everyone shitting on windows, yet this thing exists on Linux as well… I also started to dislike windows, yet this is not the time to be against windows users, this is to go against Cloudstrike together for even letting this happen.

[–] [email protected] 4 points 1 month ago

Citation needed, my NUC running Fedora made it through this without a hitch

[–] [email protected] 10 points 1 month ago* (last edited 1 month ago)

Exactly, the blame here is entirely on Crowdstrike. they could just as easily have made similar mistake in an update for the Linux agent that would crash the system and bring down half the planet.

I will say, the problem MIGHT have been easier to fix or work around on the Linux systems.

[–] [email protected] 11 points 1 month ago

It's the time to go against proprietary monopolizing software

[–] [email protected] 19 points 1 month ago (1 children)

I agree. I also think part of the blame can be placed on the system administrators who failed to make a recovery plan for circumstances like these -- it's not good to blindly place your trust in software that can be remotely updated.

In Linux, this type of scenario could be prevented by configuring servers to make copy-on-write snapshots before every software upgrade (e.g. with BTRFS or LVM), and automatically switching back to the last good snapshot if a kernel panic or other error is detected. Do you know if something similar can be achieved under Windows?

[–] [email protected] 4 points 1 month ago

Sadly, I don’t know. I’m way worse with computers than I want to be, just careful about where I get my information.

load more comments
view more: next ›