this post was submitted on 18 Mar 2025
29 points (100.0% liked)

Selfhosted

44530 readers
1289 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

Hello selfhosted! Sometimes I have to transfer big files or a large amounts of small files in my homelab. I used rsync but specifying the IP address and the folders and everything is bit fiddly. I thought about writing a bash script but before I do that I wanted to ask you about your favourite way to achieve this. Maybe I am missing out on an awesome tool I wasn't even thinking about.

top 39 comments
sorted by: hot top controversial new old
[–] [email protected] 2 points 5 hours ago

smb share if its desktop to desktop. If its from phone to PC, I throw it on nextcloud on the phone, then grab it from the web ui on pc.

Smb is the way to go if you have identity set up, since your PC auth will carry over for the connection to the smb share. Nextcloud will be less typing if not since you can just have persistent auth on the app / web.

[–] [email protected] 1 points 10 hours ago

I work from home, however my two systems (home and work) are on the same LAN, they don't see each other for file sharing. I get paid via direct deposit like everyone else which means my pay stubs are all electronic. I print those out and then use WinSCP to copy those over to my desktop. No other files are ever sent.

At home, depending on the amount of files, I either use SFTP via Filezilla, or if the mood strikes me and for a single file, I will just use SCP if I'm already on the cli which is most of the time it seems anymore doing work on my personal servers. I've found that SFTP is faster at transferring than doing a copy/paste to the NFS share to the same drive.

[–] [email protected] 2 points 12 hours ago (1 children)

Ye old samba share.

But I do like using Nextcloud. I use it for syncing my video projects so I can pick up where I left off on another computer.

[–] [email protected] 2 points 11 hours ago

Samba Bamba!!

[–] [email protected] 3 points 13 hours ago

WinSCP for editing server config

Rsync for manual transfers over slow connections

ZFS send/receive for what it was meant for

Samba for everything else that involves mounting on clients or other servers.

[–] [email protected] 2 points 13 hours ago

Syncthing and/or ftp.

[–] [email protected] 4 points 14 hours ago* (last edited 14 hours ago) (1 children)

rsync is indeed fiddly. Consider SFTP in your GUI of choice. I mount the folder I need in my file browser and grab the files I need. No terminal needed and I can put the folders as favorites in the side bar.

[–] [email protected] 2 points 14 hours ago (1 children)

If you want to use the terminal though, there is scp which is supported on both windows and Linux.

Its just scp [file to copy] [username]@[server IP]:[remote location]

[–] [email protected] 1 points 10 hours ago

That's essentially the same as rsync

[–] [email protected] 3 points 15 hours ago (1 children)

Magic wormhole is pretty dead simple https://magic-wormhole.readthedocs.io/en/latest/welcome.html#installation

I use this a lot at work for moving stuff between different test vms, as you don't need to check IPs/hostnames

[–] [email protected] 1 points 14 hours ago* (last edited 14 hours ago)

As I understand it, the establishing of the connection is reliant on a relay server. So this would not work on a local network without a relay server and would, by default, try to reach a server on the internet to make connections.

[–] [email protected] 9 points 16 hours ago

People have already covered most of the tools I typically use, but one I haven't seen listed yet that is sometimes convenient is python3 -m http.server which runs a small web server that shares whatever is in the directory you launched it from. I've used that to download files onto my phone before when I didn't have the right USB cables/adapters handy as well as for getting data out of VMs when I didn't want to bother setting up something more complex.

[–] [email protected] 4 points 16 hours ago
[–] [email protected] 2 points 18 hours ago

As a lazy person, I just prefer sftp on thunar.

[–] [email protected] 2 points 18 hours ago* (last edited 18 hours ago)

I'd say use something like zeroconf(?) for local computer names. Or give them names in either your dns forwarder (router), hosts file or ssh config. Along with shell autocompletion, that might do the job. I use scp, rsync and I have a NFS share on the NAS and some bookmarks in Gnome's file manager, so i just click on that or type in scp or rsync with the target computer's name.

[–] [email protected] 2 points 18 hours ago

By "homelab", do you mean your local network? I tend to use shared folders, kdeconnect, or WebDAV.

I like WebDAV, which i can activate on Android with DavX5 and Material Files, and i use it for Joplin.

Nice thing about this setup is that i also have a certificate secured OpenVPN, so in a pinch i can access it all remotely when necessary by activating that vpn, then disconnecting.

[–] [email protected] 8 points 18 hours ago (1 children)
[–] [email protected] 2 points 16 hours ago
[–] [email protected] 1 points 18 hours ago* (last edited 18 hours ago)

rsync if it's a from/to I don't need very often

More common transfer locations are done via NFS

[–] [email protected] 3 points 18 hours ago* (last edited 18 hours ago)
  • sftp for quick shit like config files off a random server because its easy and is on by default with sshd in most distros
  • rsync for big one-time moves
  • smb for client-facing network shares
  • NFS for SAN usage (mostly storage for virtual machines)
[–] [email protected] 8 points 18 hours ago (2 children)
[–] [email protected] 2 points 15 hours ago (1 children)
[–] [email protected] 6 points 14 hours ago (1 children)

SCP, the protocol, is deprecated. scp, the command, just uses the SFTP protocol these days. I find its syntax convenient.

[–] [email protected] 4 points 14 hours ago

Oh does it? I didn't realize that. I've just switched over to rsync completely.

[–] [email protected] 1 points 16 hours ago

Checks username… yeah that tracks

[–] [email protected] 1 points 18 hours ago

Resilion Sync

[–] [email protected] 1 points 19 hours ago

I have a shared syncthing folder on all my devices

[–] [email protected] 0 points 19 hours ago
[–] [email protected] 3 points 19 hours ago

rclone. I have a few helper functions;

fn mount { rclone mount http: X: --network-mode }
fn kdrama {|x| rclone --multi-thread-streams=8 --checkers=2 --transfers=2 --ignore-existing --progress copy http:$x nas:Media/KDrama/$x --filter-from
~/.config/filter.txt }
fn tv {|x| rclone --multi-thread-streams=8 --checkers=2 --transfers=2 --ignore-existing --progress copy http:$x nas:Media/TV/$x --filter-from ~/.config/filter.txt }
fn downloads {|x| rclone --multi-thread-streams=8 --checkers=2 --transfers=2 --ignore-existing --progress copy http:$x nas:Media/Downloads/$x --filter-from ~/.config/filter.txt }

So I download something to my seedbox, then use rclone lsd http: to get the exact name of the folder/files, and run tv "filename" and it runs my function. Pulls all the files (based on filter.txt) using multiple threads to the correct folder on my NAS. Works great, and maxes out my connection.

[–] [email protected] 1 points 19 hours ago* (last edited 19 hours ago)

Depends on what I'm transferring and to/from where:

  • scp is my go-to since I'm a Linux household and have SSH keys setup and LDAP SSO as a fallback
  • sshfs if I'm too lazy to connect via SMB/NFS (or I don't feel like installing the tools for them) or I'm traversing a WAN
  • rsync for bulk transfer and backups
  • Snapdrop/Pairdrop for one-off file/text shares between devices with GUIs (mostly phone <--> PC)
  • SMB if I'm on a client PC and need to work with the files directly from the fileserver
  • NFS between servers
  • To get bulk data to my phone (e.g. updating my music library), I connect via USB in MTP mode and copy from the server via SMB or sshfs.
[–] [email protected] 27 points 19 hours ago (2 children)

Not gonna lie, I just map a network share and copy and paste through the gui.

[–] [email protected] 3 points 19 hours ago (3 children)

Sounds very straight forward. Do you have a samba docker container running on your server or how do you do that?

[–] [email protected] 0 points 14 hours ago

I dont have a docker container, I just have Samba running on the server itself.

I do have an owncloud container running, which is mapped to a directory. And I have that shared out through samba so I can access it through my file manager. But that's unnecessary because owncloud is kind of trash.

[–] [email protected] 4 points 16 hours ago

I have two servers, one Mac and one Windows. For the Mac I just map directly to the smb share, for the Windows it's a standard network share. My desktop runs Linux and connects to both with ease.

[–] [email protected] 7 points 18 hours ago* (last edited 18 hours ago) (2 children)

I just type sftp://[ip, domain or SSH alias] into my file manager and browse it as a regular folder

[–] [email protected] 5 points 2 hours ago

YOU CAN DO THAT???

[–] [email protected] 1 points 14 hours ago (1 children)
[–] [email protected] 5 points 14 hours ago

Any file manager on Linux supports this

[–] [email protected] 5 points 19 hours ago

Same lol, somebody please enlighten me on a faster way!