I found that moderate compression (like zstd-2 or -3) not only increases your effective storage capacity, but increases R/W speeds for HDDs.
this post was submitted on 07 Jul 2024
840 points (99.0% liked)
Memes
45529 readers
1110 users here now
Rules:
- Be civil and nice.
- Try not to excessively repost, as a rule of thumb, wait at least 2 months to do it if you have to.
founded 5 years ago
MODERATORS
My usual go to drive layout, when it's impractical to put everything on a single drive, is to have a fast, but small, OS drive with core applications, if it's large enough then also use that for user data. Add in drives for anything/everything else size intensive. Like for games, I'll get a lower quality SSD that's larger than my OS drive, like grabbing a SATA SSD that's 3-4 TiB for games, with a 500GiB NVMe OS drive for programs and user data.
If money is tight, then having your fastest storage for OS and using a HDD for everything else, is a decent option..
For a while there I was running a 240GiB OS drive, and relocated all my user data, and games to a 1TiB HDD. The system ran fine like that, with few exceptions.
One big issue was that major windows updates basically failed every time, it would seem that having your user account/profile anywhere other than C:\ is problematic for that kind of thing. It's odd, but ultimately not that big of a deal. Regular security updates and whatnot worked without any issues.
Dont skimp on ssd for games. Large sata is fine. Hdd for games is not fine. Get worse anything else instead . Loadings are gonna be the death of you.
It's not as bad if the HDD is dedicated to games. With your os and games on the same drive, you're going to get wrecked.
I'd still recommend all flash everything in a decent home computer/gaming rig, but on a budget, at least separating your os from your games on different physical drives can help quite a bit.
I feel like attaching a second drive to my deck defeats the purpose
Not me got a 4tb gen 3 nvne for my main and a 2tb gen4 nvme + 2tb sata ssd.... And a 18tb, 12tb, 4tb, 3tb, 1tb HDD for other things....
How do you even manage that much storage? Btrfs or snapraid or something?
Windows.....
I'm so sorry.
Good golly gee whiz that's a lotta porn! You saving up for a rainy day or somethin?
Sorry, that first line reminded me of this
The zombie apocalypse, also known as DJT winning the presidency, somehow... Again...
well i dont need much space for my minimal linux install.
games, however, are getting bigger and bigger.
Also the system files aren't really the most important files. While it's a pain in the ass, you can reinstall your OS and get that all back again.
Reinstalling all of your games is going to take more time, and if you lost a save file, well you're never getting that back. Personal photos, videos, etc. are even a bigger priority.
So I tend to to think of the drive /home is mounted on to be the "primary drive" as it's the most important. The root is just the system files, needed for the OS, but not nearly as important as /home.
i like keeping it backed up.
It blows me away when I play a game like Valheim or Vampire Survivor and find out the game that took 1000 hours of my life is smaller than a two hour movie.
Meanwhile Quake: I took you life and soul in 50 megabytes
When I built a new PC last year, I was wondering how I managed to filled up a 4TB NVME in only 6 months... until I downloaded one of those programs that breaks down your hard drive usage.
Games, it's all games. I don't even consider myself a gamer. I can't even begin to imagine the struggle of an actual gamer who is still stuck with a 256GB SATA SSD as their only high speed drive. What do you do when nearly every game that comes out these days is 100GB+ and requires an SSD?
until I downloaded one of those programs that breaks down your hard drive usage.
Is it baobab?
Could be Filelight
256 GB root NVMe, 1 TB games hdd, 3* 256 GB SSD as raid 0 for local backups, 256 GB HDD for data, 256 GB SSD for VM images.
Why would you put local backups on RAID 0?
Because that's what Raid 0 for, basically adding together storage space with faster reads and writes. The local backups are basically just to have earlier versions of (system) files, incrementally every hour, for reference or restoring. In case something goes wrong with the main root NVMe and a backup SSD at the same time (eg. trojan wiping everything), I still have exactly the same backups on my "workstation" (beefier server), on also a RAID 0 of 3 1 TB HDDs. And in case the house burns down or something, there are still daily full backups on Google Cloud and Hetzner.
Well its for faster speeds. So I dont get why you would do a backup on a more fragile but faster storage. You described in another comment that you have many other backups, which is awesome. So good on you for taking care of everything. But yhea, using the opposite of what would be better for backups seems a bit counterintuitive to me. And to presume that it doesn't matter to use the more secure option because you have many other backups anyway, is also slightly weird since why bother in the first place then.
I don't mean any hate, you're doing way better than me. Can I ask how fast the RAID 0 gets? And how much it would be on individual drives. And how much data you have to backup daily.
Much respect for your setup, you've taken redundancy seriously and I doubt you'll ever lose anything.
The local backups are done hourly, and incrementally. They hold 2+ weeks of backups, which means I can roll back versions of packages easily, as the normal package cache is cleaned regularly. They also prevent losing individual files accidentally through weird behaviour of apps, or me.
The backups to my workstation are also done hourly, 15 minutes shifted for every device, and also incrementally. They protect against the device itself breaking, ransomware or some rouge program rm -rf'inf /, which would affect local backups too (as they're mounted in /backups, but those are mainly for providing a file history as I said.)
As most drives are slower than the 1 Gbps ethernet, the local backups are just more convenient to access and use than the one on my workstation, but otherwise exactly the same.
The .tar.xz'd backups are actual backups, considering they are not easily accessible, and need to be unpacked and externally stored.
I didn't measure the speeds of a normal SSD vs the raid - but it feels faster. Not a valid argument, of course. But in any way, I want to use it as Raid 0/Unraided for more storage space, so I can have 2 weeks of backups instead of 5 days (considering it always keeps space for 2 backups, I would have 200- GB of space instead of 700+).
The latest hourly backup is 1.3 GB in size, but if an application is used which has a single, big DB that can quickly shoot up to dozens of GB - relatively big for a homeserver hosting primarily my own stuff + a few things for my father. Like synapses' DB has 20 GB alone. On an uneventful day, that would be 31 GB. With several updates done, which means dozens of new packages in cache, that could grow to 70+GB.
Raid 0 offers no redundancy though. If any of those three disks fail, you lose the entire volume.
For the sake of backups, switching to Raid 5 would be more robust
If it fails, I will just throw in a new SSD and redo the backup. I sometimes delete everything and redo it anyway, for various reasons. In any case, I usually have all copies of all files on the original drive, as local backup on the device and backup on the workstation. And even if those three should fail - which I will immediately know, due to monitoring the systemd job - I still have daily backups on two different, global hosters as well as the seperate NAS. The only case in which all full backups would be affected would be a global destruction of all electronics due to solar storms or a general destruction of earth, in which case that's the least of my problems. And in case the house burns down, and I only have the daily backups, potentially losing 24 hours of data, that's also the least of my problems. Yes, generally using Raid 5 for backups is better, but in my case I have multiple copies of the same data at all times, surpassing the 321 rule (by far - 622, and soon 623). As all of my devices are connected via Gigabit, getting backups from eg. the workstation after the PC (with backups) died is just as fast as getting backups from the local PC backup Raid itself. And using Raid 0 is better (in speeds) than just slapping them together in series.
view more: next ›