this post was submitted on 20 Aug 2024
48 points (96.2% liked)

Sysadmin

7640 readers
21 users here now

A community dedicated to the profession of IT Systems Administration

No generic Lemmy issue posts please! Posts about Lemmy belong in one of these communities:
[email protected]
[email protected]
[email protected]
[email protected]

founded 1 year ago
MODERATORS
 

What's the best way to monitor and log which processes are responsible for high system load throughout the day? Tools like top and htop only provide immediate values, but I'm looking for a solution that offers historical data to identify the main culprits over time.

@sysadmin

#sysadmin #linux #server

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 16 points 2 months ago* (last edited 2 months ago) (2 children)

Netdata is excellent, simple and I believe FOSS. Just install locally and it should start logging pretty much everything.

[–] [email protected] 3 points 2 months ago

I run this in a Docker container on my home network without connecting it to their cloud platform (despite their - increasingly strident, it feels - "encouragements" to do so). It's very powerful, and the majority of low level configuration is done via text files. But 99% of it is automatic.

The UI is unique. It's a single, long and scrollable page, which may be an issue for some.

There are other tools out there, too. I previously used one that integrates Grafana, Prometheus and Node Exporter, which is more complex to set up and configure.

[–] [email protected] 6 points 2 months ago (2 children)

Clicked the link, started reading .. closed the window when I read "Netdata also incorporates A.I. insights for all monitored data".

[–] [email protected] 3 points 2 months ago* (last edited 2 months ago) (1 children)

this limited scope ML trained analysis is actually where "AI" excels, e.g. "computer vision" in specific medical scenarios

[–] [email protected] 1 points 2 months ago (1 children)

If the training data is available, yes, in this case, no chance.

[–] [email protected] 1 points 2 months ago

you don't think they could get training data from friendly customers using their service?

[–] [email protected] 9 points 2 months ago* (last edited 2 months ago) (1 children)

~~Eesh. Yeah, that’s a nope from me, dawg.~~

Actually, it’s all self-hosted. Granted, I haven’t looked at the code in detail, but building NNs to help efficiently detect and capture stuff is actually a very appropriate use of ML. This project looks kinda cool.

[–] [email protected] 1 points 2 months ago (1 children)

Machine Learning might be marketed as "all fine and dandy", but I'm not planning on running a monitor training system loose on my production server under any circumstances.

Not to mention that for it to be useful I'd have to give it at least a year of logs, which is both impossible and pointless, since the system running a year ago is not remotely the same as the one running today, even if not a single piece of our own code changed, which of course it did, the OS, applications and processes have been continually updated by system updates and security patches.

So, no.

[–] [email protected] 2 points 2 months ago

That’s why I put in the caveat about looking at the code. If you can’t grok what’s going on, that’s fine, but someone who does get it and can comfortably assert that no sketchy “phone home” shit is going on can and should use stuff like this, if they’re so inclined.