this post was submitted on 23 Sep 2024
768 points (95.6% liked)
linuxmemes
21009 readers
1045 users here now
Hint: :q!
Sister communities:
- LemmyMemes: Memes
- LemmyShitpost: Anything and everything goes.
- RISA: Star Trek memes and shitposts
Community rules (click to expand)
1. Follow the site-wide rules
- Instance-wide TOS: https://legal.lemmy.world/tos/
- Lemmy code of conduct: https://join-lemmy.org/docs/code_of_conduct.html
2. Be civil
- Understand the difference between a joke and an insult.
- Do not harrass or attack members of the community for any reason.
- Leave remarks of "peasantry" to the PCMR community. If you dislike an OS/service/application, attack the thing you dislike, not the individuals who use it. Some people may not have a choice.
- Bigotry will not be tolerated.
- These rules are somewhat loosened when the subject is a public figure. Still, do not attack their person or incite harrassment.
3. Post Linux-related content
- Including Unix and BSD.
- Non-Linux content is acceptable as long as it makes a reference to Linux. For example, the poorly made mockery of
sudoin Windows. - No porn. Even if you watch it on a Linux machine.
4. No recent reposts
- Everybody uses Arch btw, can't quit Vim, and wants to interject for a moment. You can stop now.
Please report posts and comments that break these rules!
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The thing with journalctl is that it is a database. Thus means that searching and finding things can be fast and easy in high complexity cases but it can also stall in cases with very high resource usage.
But why?
I just can't grasp why such elementary things need to be so fancied up.
It's not like we don't have databases and use them for relevant data. But this isn't it.
And databases with hundreds of milions of rows are faster than journalctl (in my experience on the same hardware).
Thing is that they could have preserved the textual nature and had some sort of external metadata to facilitate the 'fanciness'. I have worked in other logging systems that did that, with the ability to consume the plaintext logs in an 'old fashioned' way but a utility being able to do all the nice filtering, search, and special event marking that journalctl provides without compromising the existence of the plain text.
Plain text is slow and cumbersome for large amounts of logs. It would of had a decent performance penalty for little value add.
If you like text you can pipe journalctl
As I said, I've dealt with logging where the variable length text was kept as plain text, with external metadata/index as binary. You have best of both worlds here. Plus it's easier to have very predictable entry alignment, as the messy variable data is kept outside the binary file, and the binary file can have more fixed record sizes. You may have some duplicate data (e.g. the text file has a text version of a timestamp duplicated with the metadata binary timestamp), but overall not too bad.
But if journalctl is slow, piping is not helping.
We have only one week of very sparse logs in it, yet it takes several seconds... greping tens of gigabytes of logs can be sometimes faster. That is insane.
Strange
Probably worth asking on a technical