A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
Don't use filesystem snapshots as backup. They're a safety measure against accidental deletion or casual modifications but they're not backups.
If you want backups then use a proper, dedicated solution like Borg Backup. It connects remotely, takes care of deduplication, compression, encryption etc. and you can fully verify the backups and manage them individually.
This is the right answer. A better backup strategy is an actual backup strategy. Snapshots, drive mirroring, rsync copies, etc aren't really backups.
Have you tried a restore? A non-differential smap snapshot should be fine, but differential snapshots would make a restore difficult to impossible.
A zfssend and zfsrestore with a differential snapshot would be more traditional. If one put mbuffer in the middle, it would even be fast.
That is rsync.net's entire business model.
I still rclone my Borg repos there instead of relying on snapshots though.
I also use rsync.net but as direct host for my borg repos, why rclone after?
It works the same either way. Borg does a lot of different backups on my home network. I also have more than just Borg backups that I want off-site, so an rclone of everything from that nas share once after everything else is done makes more sense than duplicating Borg everywhere. The rclone'd stuff can be used directly just like if it was put there by Borg itself.
It'd be worth checking out Borg as an alternative to rsync. Borg will handle snapshotting, and automatically de-dupe on a block-by-block basis.
I use it for all of my remote backups, and it provides a lot of quality of life stuff that rsync isn't going to handle.
So for this, would i make another zfs pool on my remote backup server that is not snapshotted? Like, the problem i have is that i have snapshotting via rsync, but then the whole remote server zfs pool is further snapshotted so there’s a lot of redundancy.
Using plain rsync sounds sane.
Sending local ZFS snapshots to the remote ZFS might be problematic. Consider accidentally deleting important data locally and nuking all of your local snapshots, then sending that to the remote ZFS. You lost all of your snapshots and there's no way to recover the deleted data. Instead do what I do - keep the two ZFS systems separate and use a non-ZFS mechanism to transfer data - rsync, Syncthing, etc. That way even if you delete everything locally, nuke all local snapshots and send the deletions via rsync remotely, you could still recover your data by restoring the remote ZFS to a snapshot prior to the deletions. For reference I have two ZFS machines doing frequent snapshots and Syncthing replicating data between them on immediate basis.
!selfhosted, please do critique if you find some fundamental issues with this.
You don't sync the deletion of snapshots, you use expiry on the remote
Zfs send / receive might be what you want
Wouldn't send/receive also sync snapshots across ZFS instances?
Docs say this , so yeah. "send streams can either be “full”, containing all data in a given snapshot, or “incremental”, containing only the differences between two snapshots. ZFS receive reads these send streams and uses them to re-create identical snapshots on a receiving system. "
Hm, so send doesn't "create the same state, bits and snapshots" on the other side. Instead it "adds net new snapshots" on the other side. 🤔
Perhaps I could use send instead of Syncthing after all. But then again I'm typically syncing net new data so the optimization would be minimal.
I believe there is a method to do a 1-1 build copy, but my expertise ends at this point
This is fantastically helpful, thank you. I will do this.
I don’t know why I thought sending zfs snapshots was the better option
Cause it makes sense at a glance and it's efficient. Not for backup purposes though.