this post was submitted on 01 Aug 2024
122 points (96.9% liked)

Selfhosted

39840 readers
455 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

I'm proud to share a major development status update of XPipe, a new connection hub that allows you to access your entire server infrastructure from your local desktop. It works on top of your installed command-line programs and does not require any setup on your remote systems. So if you normally use CLI tools like ssh, docker, kubectl, etc. to connect to your servers, it will automatically integrate with them.

Here is how it looks like if you haven't seen it before:

Hub

Hub Alt

Browser

Local forwarding for services

Many systems run a variety of different services such as web services and others. There is now support to detect, forward, and open the services. For example, if you are running a web service on a remote container, you can automatically forward the service port via SSH tunnels, allowing you to access these services from your local machine, e.g., in a web browser. These service tunnels can be toggled at any time. The port forwarding supports specifying a custom local target port and also works for connections with multiple intermediate systems through chained tunnels. For containers, services are automatically detected via their exposed mapped ports. For other systems, you can manually add services via their port.

Markdown notes

Another feature commonly requested was the ability to create and share notes for connections. As Markdown is everywhere nowadays, it makes sense so to implement any kind of note-taking functionality with Markdown. So you can now add notes to any connection with Markdown. The full spec is supported. The editing is delegated to a local editor of your choice, so you can have access to advanced editing features and syntax highlighting there.

Markdown

Proxmox improvements

You can now automatically open the Proxmox dashboard website through the new service integration. This will also work with the service tunneling feature for remote servers.

You can now open VNC sessions to Proxmox VMs.

The Proxmox support has been reworked to support one non-enterprise PVE node in the community edition.

Scripting improvements

The scripting system has been reworked. There have been several issues with it being clunky and not fun to use. The new system allows you to assign each script one of multiple execution types. Based on these execution types, you can make scripts active or inactive with a toggle. If they are active, the scripts will apply in the selected use cases. There currently are these types:

  • Init scripts: When enabled, they will automatically run on init in all compatible shells. This is useful for setting things like aliases consistently
  • Shell scripts: When enabled, they will be copied over to the target system and put into the PATH. You can then call them in a normal shell session by their name, e.g. myscript.sh, also with arguments.
  • File scripts: When enabled, you can call them in the file browser with the selected files as arguments. Useful to perform common actions with files

Scripts

Native window styles

The application styling has been improved to fit in better with native window decorations:

Windows style

macOS style

A new HTTP API

For a programmatic approach to manage connections, XPipe 10 comes with a built-in HTTP server that can handle all kinds of local API requests. There is an openapi.yml spec file that contains all API definitions and code samples to send the requests.

To start off, you can query connections based on various filters. With the matched connections, you can start remote shell sessions and for each one and run arbitrary commands in them. You get the command exit code and output as a response, allowing you to adapt your control flow based on command outputs. Any kind of passwords and other secrets are automatically provided by XPipe when establishing a shell connection. You can also access the file systems via these shell connections to read and write remote files.

A note on the open-source model

Since it has come up a few times, in addition to the note in the git repository, I would like to clarify that XPipe is not fully FOSS software. The core that you can find on GitHub is Apache 2.0 licensed, but the distribution you download ships with closed-source extensions. There's also a licensing system in place as I am trying to make a living out of this. I understand that this is a deal-breaker for some, so I wanted to give a heads-up.

The system is designed to allow for unlimited usage in non-commercial environments and only requires a license for more enterprise-level environments. This system is never going to be perfect as there is not a very clear separation in what kind of systems are used in, for example, homelabs and enterprises. But I try my best to give users as many free features as possible for their personal environments.

Outlook

If this project sounds interesting to you, you can check it out on GitHub! There are more features to come in the near future.

Enjoy!

all 30 comments
sorted by: hot top controversial new old
[–] [email protected] 1 points 2 months ago (1 children)

Looks really cool. I've been working on implementing SSO through Authentik for every home lab service that supports it (Like Proxmox!) Do you think you'll add SSO/SAML? If you do, I recommend not locking it behind the enterprise plan to encourage adoption.

[–] [email protected] 2 points 2 months ago

In terms of any authentication, XPipe doesn't implement anything by itself. It will just delegates to your local SSH client. If you can set that up with your ssh client so that you can successfully connect from the command line, it should also work in XPipe.

[–] [email protected] 1 points 2 months ago (1 children)

I’ve seen this project just get better and better. Thes improvements are awesome. The tool gives you so many ways to do things, it’s amazing. I covered it on my channel a while back, and someone who watched reported a bug and it was fixed within a couple of days. Try that with any of the big tech giants. You all rock! Well done!

[–] [email protected] 2 points 2 months ago

Thanks for your video showcase back then, it really helped the project get the initial traction.

The project was definitely rough around the edges back then. It held together somewhat but I would say it was around a 50/50 chance that it would work as expected for a new user. I think that has been the biggest improvement since then, the reliability and handling of edge cases so that the vast majority of users can now use it as they expect without issues. That was made possible with the help of the community which reported and tested all kinds of things I could not have done on my own. Having a community running a diverse set of systems helps out development immensely.

[–] [email protected] 1 points 2 months ago (1 children)

This seems really cool and might be very helpful to me, but I don't want to install it on my computer. I don't see a docker image for it, though it seems like it would be easy to create one; but this is a GUI app, so how would I run it in a container somewhere and use it via the GUI on my local computer? Or if I install it in its own VM (I use Proxmox), I'd have to use a remote desktop app like vlc or something, right?

I'm a noob at this so there's tons I just don't know.

[–] [email protected] 1 points 2 months ago (1 children)

It's intended to be installed on your local desktop because it integrates with your installed programs like your system shell, text editor, terminal, etc. This would not be possible if it would be installed in a container or VM. I can understand some concerns about installing software on you local machine, but this is a case where creating an isolated container for an application would not make sense.

[–] [email protected] 1 points 2 months ago (1 children)

I host a bunch of containers on a few servers, but I don't do any of it from my local computer. I have a VM (Debian) that I ssh into and do everything from there. Shouldn't XPipe work the same on that VM as it would on my local computer? I wouldn't think XPipe would care (or know) if it was running on a VM, as long as that VM has a shell it can integrate with.

But I suppose even if that's true and XPipe works fine in the VM, there is still the issue of displaying the GUI on my local computer.

[–] [email protected] 2 points 2 months ago (1 children)

It should also work in a graphical VM, but I assume that you have your tools installed on your desktop. E.g. your preferred terminal or editor since you only have a console in your VM via ssh.

If you install XPipe on your desktop, it can connect to the VM from there and through the VM also connect to all your other servers as a gateway.

[–] [email protected] 1 points 2 months ago (1 children)

Thanks for the help and suggestions!

It turns out that my template Debian VM doesn't have a DE in it, and that's why I couldn't forward the GUI from the VM to my local machine: there was not GUI. I installed XFCE on the VM and now I can run XPipe on the VM from my local computer, without XPipe being installed on my local computer: ssh -X user@vm_ip_address xpipe open

I look forward to playing with XPipe - it looks cool and very helpful!

[–] [email protected] 1 points 2 months ago

Alright I guess that approach works.

But installing it locally should be much easier. It can also access connections through your VM via ssh from there.

[–] [email protected] 2 points 2 months ago

I've been using this for a few months now. Its really great.

[–] [email protected] 4 points 2 months ago (1 children)

Looks really nice definitely gonna take a look at it.

And, did I read that correctly: the pro licence is a onetime payment and you keep the all the current features even after the licence expires?

[–] [email protected] 3 points 2 months ago
[–] [email protected] 2 points 2 months ago* (last edited 2 months ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
HTTP Hypertext Transfer Protocol, the Web
SCP Secure Copy encrypted file transfer tool, authenticates and transfers over SSH
SSH Secure Shell for remote terminal access
SSO Single Sign-On

3 acronyms in this thread; the most compressed thread commented on today has 4 acronyms.

[Thread #901 for this sub, first seen 1st Aug 2024, 20:35] [FAQ] [Full list] [Contact] [Source code]

[–] [email protected] 3 points 2 months ago* (last edited 2 months ago) (1 children)

Thank you for posting this - I have had a play and it's excellent.

I am a newbie in the Linux world and have been looking for something to replace the excellent WinSCP. XPipe is the first application I've used on Linux that actually makes remote SSH/SCP browsing easy to do, while still being able to handle more complicated SSH auth than just user/pass.

[–] [email protected] 2 points 2 months ago

That is great to hear!

[–] [email protected] 2 points 2 months ago (1 children)

Xpipe has rapidly become an essential part of my day to day workflow. I wish I could get it to play nice with MobaXTerm as I prefer that to any of the supported terminals, but that's really my only gripe. In every other regard it's just an excellent toolkit for managing remote devices.

[–] [email protected] 1 points 2 months ago

That is great to hear!

For MobaXTerm you can always try to ask the devs to maybe expose the functionality to launch their terminal from the command-line. That would work.

[–] [email protected] 2 points 2 months ago (1 children)

I’ve seen multiple markdown standards; which one did you implement?

[–] [email protected] 2 points 2 months ago

It should be close to the CommonMark spec, so it should support the same features as you find e.g. in GitHub markdown.

[–] [email protected] 8 points 2 months ago* (last edited 2 months ago) (1 children)

I'm going to take a look. Thanks for sharing!!

Edit: oh, I use Yubikeys for my home lab. Might not be an option for me without getting a paid license :(((

[–] [email protected] 3 points 2 months ago (2 children)

I assumed that yubikeys would be found pretty much only in enterprise environments but perhaps I was wrong there.

Maybe I can find a solution to that. The free plan restrictions are not perfect yet and I was planning to experiment with different solutions to it. If you just want to try it out, I can also offer evaluation licenses if you're interested.

[–] [email protected] 3 points 2 months ago (1 children)

I assumed that yubikeys would be found pretty much only in enterprise environments but perhaps I was wrong there.

As a datapoint, I am a home user and use Yubikeys. For example, they are one of the 2FA options supported by Bitwarden for home users.

I have recently trialed both NitroKey and OnlyKey to see if I'd want to replace my Yubikey with either of those, but the Yubikey is sadly superior. (Sadly because it's not as open as those other two options.)

[–] [email protected] 4 points 2 months ago (1 children)

Do you use the normal one or the FIPS one? Maybe I can use that to differentiate between personal and commercial use

[–] [email protected] 2 points 2 months ago* (last edited 2 months ago) (1 children)

Not the FIPS one, I use the normal 5 series.

Yubikeys are great because you can also add your TOTP codes on there, but require a physical touch to generate the codes.

You can do that with other products like the NitroKey as well, but the implementation is not as good - example the secrets are not encrypted on the NitroKey.

[–] [email protected] 3 points 2 months ago

Alright, I will have to look into whether it is possible to differentiate between normal and FIPS here

[–] [email protected] 2 points 2 months ago (1 children)

I wouldn't mind trying an evaluation, would be nice to see how it works with RHEL and Windows Server as well. I also work in an enterprise and would love to compare it to our current tools, but I am worried it won't like our "PAM".

My homelab is a bit more advanced than most as I use it for education as well as having a badassed home network. So I use security keys in it to keep up with the enterprise.

I think you'll find security keys will be picking up steam with home users, it's nice to have that extra layer for public facing stuff and private VPSs.

[–] [email protected] 2 points 2 months ago (1 children)

Alright I see. With the more professional homelab setups it will be always difficult to properly differentiate all cases for the community and professional edition here.

But you can send me an email at [email protected], I can provide you with an evaluation license.

[–] [email protected] 1 points 2 months ago

Yep, I'm kinda pushing it, I know! :))

I'll send you an email later in the day when I have a chance. Thank you for offering the evaluation.