this post was submitted on 05 Feb 2024
46 points (97.9% liked)

Selfhosted

40113 readers
851 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
46
submitted 9 months ago* (last edited 9 months ago) by [email protected] to c/[email protected]
 

A year ago I set up Ubuntu server with 3 ZFS pools on my server, normally I don't make copies of very large files but today I was making a copy of a ~30GB directory and I saw in rsync that the transfer doesn't exceed 3mb/s (cp is also very slow).

What is the best file system that "just works"? I'm thinking of migrating everything to ext4

EDIT: I really like the automatic pool recovery feature in ZFS, has saved me from 1 hard drive failure so far

(page 2) 17 comments
sorted by: hot top controversial new old
[–] [email protected] 36 points 9 months ago* (last edited 9 months ago) (6 children)

ZFS is a very robust choice for a NAS. Many people, myself included, as well as hundreds of businesses across the globe, have used ZFS at scale for over a decade.

Attack the problem. Check your system logs, htop, zpool status.

When was the last time you ran a zpool scrub? Is there a scrub, or other zfs operation in progress? How many snapshots do you have? How much RAM vs disk space? Are you using ZFS deduplication? Compression?

load more comments (6 replies)
[–] [email protected] 5 points 9 months ago* (last edited 9 months ago)

MergerFS + Snapraid is a really nice way to turn ext4 mounts into a single entry point NAS. OpenMediaVault has some plugins for setting this up. Performance wise it will max out the drive of whichever one you are using and you can use cheap mismatched drives.

[–] [email protected] 9 points 9 months ago (1 children)

How full is your ZFS? ZFS doesn't handle disk filling and fragmentation well.

[–] [email protected] 4 points 9 months ago (1 children)

Around 70% full with 10% fragmentation

load more comments (1 replies)
[–] [email protected] 16 points 9 months ago (1 children)

3mb/s sounds more like there is something else going on.

[–] [email protected] 3 points 9 months ago (1 children)

Yeah, but I don't know how to diagnose it...

[–] [email protected] 3 points 9 months ago (1 children)

You could try to redo the copy and monitor the system in htop, for example. Maybe there's a memory or CPU bottleneck. Maybe one of your drives is failing, maybe you've got a directory with tons of very small files, which causes a lot of overhead.

[–] [email protected] 2 points 9 months ago

Yes, file size, drive types, the amount of RAM in the server, in the source and destination of the operation, can all have an effect on Performance. But generally if he’s moving within the same pool, it should be pretty quick.

[–] [email protected] 9 points 9 months ago (8 children)

XFS has "just worked" for me for a very long time now on a variety of servers and desktop systems.

load more comments (8 replies)
[–] [email protected] 27 points 9 months ago (7 children)

Ext4 does not have snapshots, COW or similar features. I am very happy with BTRFS. It just "works" out of the box.

[–] [email protected] 14 points 9 months ago (1 children)

I use BTRFS on everything too nowadays. The thing that made me switch everything to BTRFS was filesystem compression.

[–] [email protected] 2 points 9 months ago (1 children)

Yes compression is cool. Zstd level 3 to 6 is very quick too 😋

load more comments (1 replies)
[–] [email protected] 3 points 9 months ago (4 children)

Yes and BTRFS, unlike Ext4, will not go corrupt on the first power outage of slight hardware failure.

[–] [email protected] 5 points 9 months ago

Wut? Ext4 is quite reliable.

[–] [email protected] 1 points 9 months ago (1 children)

This will also happen to Ext4. You just wouldn't know it.

[–] [email protected] 2 points 9 months ago (7 children)

I'm confused with your answer. BTRFS is good and reliable. Ext4 gets fucked at the slightest issue.

load more comments (7 replies)
load more comments (2 replies)
load more comments (5 replies)
load more comments
view more: ‹ prev next ›