Linux

48185 readers

1266 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
No misinformation
No NSFW content
No hate speech, bigotry, etc

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago

MODERATORS

[email protected]

783

Today I'm grateful I'm using Linux - Global IT issues caused by Crowdstrike update causes BSOD on Windows (www.timesnownews.com)

submitted 4 months ago by [email protected] to c/[email protected]

193 comments fedilink hide all child comments

This isn't a gloat post. In fact, I was completely oblivious to this massive outage until I tried to check my bank balance and it wouldn't log in.

Apparently Visa Paywave, banks, some TV networks, EFTPOS, etc. have gone down. Flights have had to be cancelled as some airlines systems have also gone down. Gas stations and public transport systems inoperable. As well as numerous Windows systems and Microsoft services affected. (At least according to one of my local MSMs.)

Seems insane to me that one company's messed up update could cause so much global disruption and so many systems gone down :/ This is exactly why centralisation of services and large corporations gobbling up smaller companies and becoming behemoth services is so dangerous.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 64 points 4 months ago (2 children)

I've just spent the past 6 hours booting into safe mode and deleting crowd strike files on servers.

[–] [email protected] 1 points 4 months ago (2 children)

Can't you automate it?

[–] [email protected] 5 points 4 months ago (1 children)

Sadly not. Windows doesn't boot. You can boot it into safe mode with networking, at which point maybe with anaible we could login to delete the file but since it's still manual work to get windows into safe mode there's not much point

[–] [email protected] 7 points 4 months ago (2 children)

It is theoretically automatable, but on bare metal it requires having hardware that's not normally just sitting in every data centre, so it would still require someone to go and plug something into each machine.

On VMs it's more feasible, but on those VMs most people are probably just mounting the disk images and deleting the bad file to begin with.

[–] [email protected] -1 points 4 months ago

The real problem with VM setups is that the host system might have crashed too

[–] [email protected] 5 points 4 months ago

I guess it depends on numbers too. We had 200 to work on. If you're talking hundreds more than looking at automation would be a better solution. In our scenario it was just easier to throw engineers at it. I honestly thought at first this was my weekend gone but we got through them easily in the end.

[–] [email protected] 10 points 4 months ago

Since it has to happen in windows safe mode it seems to be very hard to automate the process. I haven’t seen a solution yet.

[–] [email protected] 18 points 4 months ago (1 children)

Feel you there. 4 hours here. All of them cloud instances whereby getting acces to the actual console isn't as easy as it should be, and trying to hit F8 to get the menu to get into safe mode can take a very long time.

[–] [email protected] 6 points 4 months ago (2 children)

Ha! Yes. Same issue. Clicking Reset in vSphere and then quickly switching tabs to hold down F8 has been a ball ache to say the least!

[–] [email protected] 2 points 3 months ago

Just go into settings and add a boot delay, then set it back when you're done.

[–] [email protected] 2 points 3 months ago* (last edited 3 months ago)

What I usually do is set next boot to BIOS so I have time to get into the console and do whatever.

Also instead of using a browser, I prefer to connect vmware Workstation to vCenter so all the consoles insta open in their own tabs in the workspace.