Selfhosted

OP, I have been facing the same situation as you in this community recently. This was not the case when I first joined Lemmy but the behaviour around these parts has started to resemble Reddit more and more. But we'll leave it at that.

I think I have a solution for you if you're willing to spend $2-$3 a month - set up a VPS and run a Wireguard server on it. Run clients on your devices and the raspberry pi and connect to it.

As for your LAN: from the discussion you linked, it seems that Jellyfin will use the CAs present in the OS trust store. That's not very hard to do on Linux but I guess if you have to do it on Android you'd have some more trouble. In either case, using a reverse-proxy (I like HAProxy but I use it at work and it might be more enterprise than you need, for beginners Caddy is usually easier) will fix the trouble you're having with your own CA and self-signed certs.

I am interested in the attack vector you mentioned; could you elaborate on the MITM attack?

Unfortunately, if you don't have control over your network, you cannot force a DNS server for your devices unless you can set it yourself for every individual client. If I assume that you can do that, then:

1. Set up DNS server on Pi
2. Set up CA on Pi
3. Create root CRT, CSR and server certs from it (bare-minimim setup)
4. Copy over this stuff to Jellyfin image/VM, and copy root cert to clients trust store.
5. Run reverse proxy in front of Jellyfin and configure the correct IP address of the reverse proxy with an A record in your DNS server.
6. Configure reverse-proxy with server/application cert.
7. Use RethinkDNS on Android to pass everything through the wireguard server hosted on the VPS, and set private DNS to the DNS server hosted on the Pi.

I think that should do it. This turned out more complicated than I imagined (it's more of a brain dump at this point), feel free to ask if it is overwhelming.

I remember you were worried about your ISP messing things up for you, hence the VPN. I would recommend creating a "Virtual Machine" that does all of your downloading to whatever hard drive you're using. That VM can have proton installed. Then, on your regular computer (not within the VM), you can host Jellyfin with no VPN involved, making it accessible at 192.168.0.xx.

I think this hits your goals without needing to expose Jellyfin to the Internet. Plus it has minimal technical complexity. Your downloading traffic will be VPN protected, but Jellyfin will still be accessible to your local network.
edit: You can set up a password for Jellyfin, protecting it from your internal threats.

edit2: You can use letsencrypt to create a certificate that picky clients will accept. Buy a domain, any domain, and configure the "A record" to point to 192.168.0.xx (your Jellyfin IP). Then tell your client to go to whatever domain you get, like "luigiliterallydidnothingwrongplzfree.com", then the client will have to use the internet to ask DNS what the IP address is, but after that, it will just use your local network.

edit3: Since you just have the raspberry PI, instead of using a Virtual Machine, you could have 2 separate SD cards. One only has the downloader and VPN installed, the other only has Jellyfin installed (no VPN). Then swap as needed.

Maybe self host your own VPN on a VPS and connect the jellyfin server as a client as well as any other devices you want to see that jellyfin server as other clients and configure the VPN server to not override your default routing and to allow clients to see each other? In my head I don't think that would conflict with your protonVPN connection.

Your traffic would be encrypted between devices so I wouldn't say https is nessesary and thus no certs needed.

The rubs that occur to me are that I'm not sure you can do this on a free tier VPS which is the only option I see given your financial limitations. And your devices all need to be able to connect to said VPN.

Edit: Slightly less worse English.

I read the old thread and now this one.

As I understand it, you want to create connection between clients on your lan, but you don't trust your lan, so it's like having a raspberry pi server and some client both on the coffee shop network and you want them to communicate securely?

Tailscale is what you want. Easy setup, free, and allows exactly this to happen.

I previously proffered some information in the first thread.

But there's something I wish to clarify about self-signed certificates, for the benefit of everyone. Irrespective of whichever certificate store that an app uses -- either its own or the one maintained by the OS -- the CA Browser Forum, which maintains the standards for public certificates, prohibits issuance of TLS certificates for reserved IPv4 or IPv6 addresses. See Section 4.2.2.

This is because those addresses will resolve to different machines on different networks. Whereas a certificate for a global-scope IP address is fine because it should resolve to the same destination. If certificate authorities won't issue certs for private IP addresses, there's a good chance that apps won't tolerate such certs either. Nor should they, for precisely the reason given above.

A proper self-signed cert -- either for a domain name or a global-scope IP address -- does not create any MITM issues as long as the certificate was manually confirmed the first time and added to the trust store, either in-app or in the OS. Thereafter, only a bona fide MITM attack would raise an alarm, the same as if a MITM attacker tries to impersonate any other domain name. SSH is the most similar, where trust-on-first-connection is the norm, not the outlier.

There are safe ways to use self-signed certificate. People should not discard that option so wontonly.

How about creating your own LAN within the untrusted network?

Something like an inexpensive OpenWRT router would do fine. Connect all your devices and the server to the router. They are now on a trusted network. Set up Wireguard on the OpenWRT router to connect to Proton so that your outbound traffic from all your devices is secured.

ProtonVPN in its free tier does not allow LAN connections

This is the limiting factor. In order to get around this, you'll have to put your Jellyfin server on the Internet. Hopefully you can enable port forwarding. If not, you have painted yourself into a corner.

If you cannot use self-signed or internal CA certs, you will also need a domain name, and something like Let's Encrypt to issue certs for that domain.

Again, Tailscale.