this post was submitted on 19 Mar 2024
96 points (94.4% liked)

Linux

48006 readers
1015 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
 

What do you advice for shell usage?

  • Do you use bash? If not, which one do you use? zsh, fish? Why do you do it?
  • Do you write #!/bin/bash or #!/bin/sh? Do you write fish exclusive scripts?
  • Do you have two folders, one for proven commands and one for experimental?
  • Do you publish/ share those commands?
  • Do you sync the folder between your server and your workstation?
  • What should've people told you what to do/ use?
  • good practice?
  • general advice?
  • is it bad practice to create a handful of commands like podup and poddown that replace podman compose up -d and podman compose down or podlog as podman logs -f --tail 20 $1 or podenter for podman exec -it "$1" /bin/sh?

Background

I started bookmarking every somewhat useful website. Whenever I search for something for a second time, it'll popup as the first search result. I often search for the same linux commands as well. When I moved to atomic Fedora, I had to search for rpm-ostree (POV: it was a horrible command for me, as a new user, to remember) or sudo ostree admin pin 0. Usually, I bookmark the website and can get back to it. One day, I started putting everything into a .bashrc file. Sooner rather than later I discovered that I could simply add ~/bin to my $PATH variable and put many useful scripts or commands into it.

For the most part I simply used bash. I knew that you could somehow extend it but I never did. Recently, I switched to fish because it has tab completion. It is awesome and I should've had completion years ago. This is a game changer for me.

I hated that bash would write the whole path and I was annoyed by it. I added PS1="$ " to my ~/.bashrc file. When I need to know the path, I simply type pwd. Recently, I found starship which has themes and adds another line just for the path. It colorizes the output and highlights whenever I'm in a toolbox/distrobox. It is awesome.

(page 2) 22 comments
sorted by: hot top controversial new old
[–] [email protected] 4 points 7 months ago* (last edited 7 months ago) (3 children)

I use bash as my interactive shell. When ~20 years ago or so I encountered "smart" tab completion for the first time, I immediately disabled that and went back to dumb completion, because it caused multi-second freezes when it needed to load stuff from disk. I also saw it refuse to complete filenames because they had the wrong suffix. Maybe I should try to enable that again, see if it works any better now. It probably does go faster now with the SSDs.

I tried OpenBSD at some point, and it came with some version of ksh. Seems about equivalent to bash, but I had to modify some of my .bashrc so it would work on ksh. I would just stick to the default shell, whatever it is, it's fine.

I try to stick to POSIX shell for scripts. I find that I don't need bashisms very often, and I've used systems without bash on them. Most bash-only syntax has an equivalent that will work on POSIX sh. I do use bash if I really need some bash feature (I recently wanted to set -o pipefail, which dash cannot do apparently, and the workaround is really annoying).

Do not use #!/bin/sh if you're writing bash-only scripts. This will break on Debian, Ubuntu, BSD, busybox etc. because /bin/sh is not bash on those systems.

load more comments (3 replies)
[–] [email protected] 7 points 7 months ago (1 children)

That's the way I do it:

#!/usr/bin/env nix
#! nix shell nixpkgs#nushell <optionally more dependencies>  --command nu

<script content>

But those scripts are only used by me

load more comments (1 replies)
[–] [email protected] 2 points 7 months ago
  • Do you use bash? If not, which one do you use? zsh, fish? Why do you do it?
  • Do you write #!/bin/bash or #!/bin/sh? Do you write fish exclusive scripts?

I use bash, and I use #!/bin/bash for my scripts. Some are POSIX compliant, some have bashisms. But I really don't care about bashisms, since I explicitly set the bash as interpreter. So no, no fish exclusive scripts, but some "bash exclusive" scripts. Since fish is aimed towards being used as interactive shell I don't see a real reason to use it as interpreter for scripts anyways.

  • Do you have two folders, one for proven commands and one for experimental?
  • Do you publish/ share those commands?
  • Do you sync the folder between your server and your workstation?

I have my scripts in $HOME/.scripts and softlink them from a directory in $PATH. Some of the scripts are versioned using Git, but the repository is private and I do not plan sharing them because the repoand the scripts scripts contain some not-tho-share information and mostly are simply not useful outside my carefully crafted and specific environment. If I want to share a script, I do it individually or make a proper public Git repository for it.

Since my server(s) and my workstations have different use cases I do not share any configuration between them. I share some configuration between different workstations, though. My dotfiles repository is mainly there for me to keep track of changes in my dotfiles.

is it bad practice to create a handful of commands

It becomes bad practice if it is against your personal or corporate guidelines regarding best practices. While it is not particularly bad or insecure, etc. to create bash scripts containing a single command, maybe use an alias instead. The $1 is automatically the first parameter after typing the alias in the shell.

alias podup="podman compose up -d"
alias poddown="podman compose down"
alias podlog="podman logs -f --tail 20"

Not quite sure about the podman syntax, if podman exec /bin/sh -it "$1" also works, you can use alias podenter="podman exec /bin/sh -it, Otherwise a simple function would do the trick.

[–] [email protected] 5 points 7 months ago (2 children)

Am I missing something - doesn't bash have tab completion or of the box?

[–] [email protected] 5 points 7 months ago
load more comments (1 replies)
[–] [email protected] 3 points 7 months ago

A good idea i have been spreading around relevant people lately is to use ShellCheck as you code in Bash, integrate it in your workflow, editor or IDE as relevant to you (there's a commandline tool as well as being available for editors in various forms), and pass your scripts through it, trying to get the warnings to go away. That should fix many obvious errors and clean up your code a bit.

[–] [email protected] 7 points 7 months ago* (last edited 7 months ago) (4 children)
  • I use bash, because I never had the time to learn anything else.
  • Like @[email protected] said, I use the #!/usr/bin/env bash shebang.
  • Nope
  • Also nope
  • Nope. Shell scripts reside in Git repos on Gitlab/Gitea/Forgejo and are checked out using Ansible playbooks onto the servers as necessary.
  • For scripts? Python. Read this blog post by the great @[email protected]. For interactive use? bash is just fine for me, though I've customized it using Starship and created some aliases to have colored/pretty output where possible.
  • Use shellcheck before running your scripts in production, err on the side of caution, set -o pipefail. There are best practices guides for Bash, use those and you'll probably be fine.
  • Be prepared to shave yaks. Take breaks, touch grass, pet a dog. Use set -x inside your Bash script or bash -x scriptname on the CLI for debugging. Remember that you can always fallback to interactive CLI to test/prepare commands before you put them into your script. Think before you type. Test. Optimize only what needs optimization. Use long options for readability. And remember: Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows your address.
  • Nope, it's absolutely not bad practice to create aliases to save you some typing in interactive shell. You shouldn't use them inside your scripts though, because they might/will not be available in other environments.

I switched to fish because it has tab completion Yeah, so does Bash, just install it.

Oh, I also "curate" a list of Linux tools that I like, that are more modern alternatives to "traditional" Linux tools or that provide information I would otherwise not easily get. I'll post i

ToolsDebian-Packages available

  • mtr
  • iputils-tracepath
  • iproute2
  • zsh
  • httpie
  • aria2
  • icdiff
  • progress
  • diffoscope
  • atop
  • powertop
  • ntopng
  • ethtool
  • nethogs
  • vnstat
  • ss
  • glances
  • discus
  • dstat
  • logwatch
  • swatch
  • multitail
  • lynis
  • ncdu (du-clone), alias du="ncdu --color dark -rr -x --exclude .git --exclude node_modules"
  • nnn (fully-featured terminal file manager. It’s tiny, nearly 0-config and incredibly fast. https://github.com/jarun/nnn)
  • slurm
  • calcurse
  • newsbeuter
  • tig ("ncurses TUI for git. It’s great for reviewing and staging changes, viewing history and diffs.")
  • qalc -ttyrec
  • taskwarrior
  • ttytter
  • ranger
  • ipcalc
  • pandoc
  • moreutils
  • googler
  • weechat
  • pdftk
  • abcde
  • dtrx
  • tload
  • ttyload
  • cockpit
  • sar
  • ht (hte Hex Editor)
  • dhex
  • ack (grep-clone)
  • silversearcher-ag (grep-clone)
  • ripgrep ("recursively searches file trees for content in files matching a regular expression. It’s extremely fast, and respects ignore files and binary files by default.", https://github.com/BurntSushi/ripgrep)
  • exa (statt ls) https://the.exa.website/ ("replacement for ls with sensible defaults and added features like a tree view, git integration, and optional icons.")
  • fzf (CLI fuzzy finder), alias preview="fzf --preview 'bat --color "always" {}'"
  • fd (simple, fast and user-friendly alternative to 'find', https://github.com/sharkdp/fd) -entr (watch-clone)
  • csvkit (awk-clone)
  • ccze (log coloring)
  • surfraw -hexyl ("hex viewer that uses Unicode characters and colour", https://github.com/sharkdp/hexyl) -jq ("awk for JSON. It lets you transform and extract information from JSON documents", https://stedolan.github.io/jq/) -pass ("password manager that uses GPG to store the passwords", https://github.com/lunaryorn/mdcat)
  • restic ("backup tool that performs client side encryption, de-duplication and supports a variety of local and remote storage backends.", https://restic.net/)
  • mdp (Markdown Presentation on CLI) -grepcidr
  • qrencode
  • caca-utils (show images on the CLI)
  • fbi ( & fbgs) (show images in Framebuffer device)
  • fbcat (take screnshot on framebuffer device)
  • nmap
  • micro (CLI Text Editor, ab Debian 11, https://micro-editor.github.io)
  • masscan (https://github.com/robertdavidgraham/masscan)
  • socat (Nachfolger von netcat, https://www.heise.de/select/ix/2017/11/1509815804306324)
  • dc3dd (patched version of GNU dd with added features for computer forensics)
  • smem (memory reporting tool)
  • free (Show Linux server memory usage)
  • mpstat (Monitor multiprocessor usage on Linux, part of sysstat package)
  • pmap (Montor process memory usage on Linux, part of the procps)
  • monit (Process supervision)
  • oping & noping
  • saidar (Curses-basiertes Programm für die Anzeige von Live-Systemstatistiken)
  • reptyr (Tool for moving running programs between ptys)
  • gron (https://github.com/tomnomnom/gron, makes JSON greppable, kann HTTP-Requests absetzen)
  • jc (https://github.com/kellyjonbrazil/jc, CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.)
  • bat (cat-clone), alias cat='bat' ("alternative to the common (mis)use of cat to print a file to the terminal. It supports syntax highlighting and - git integration.", https://github.com/sharkdp/bat)
  • ioping (https://github.com/koct9i/ioping, simple disk I/0 latency measuring tool, auch für disk seek rate/iops/avg)
  • vd (Visidata, multipurpose terminal utility for exploring, cleaning, restructuring and analysing tabular data. Current supported sources are TSV, CSV, fixed-width text, JSON, SQLite, HTTP, HTML, .xls, and .xlsx)
  • pdfgrep
  • duf https://github.com/muesli/duf (combined df and du, ncurses-based)
  • nala (apt-alternate, https://gitlab.com/volian/nala, https://christitus.com/stop-using-apt/)
  • iprange
  • tldr
  • rmlint
  • nvtop (https://github.com/Syllo/nvtop, GPUs process monitoring for AMD, Intel and NVIDIA)
  • lf (lf (as in “list files”) is a terminal file manager written in Go with a heavy inspiration from ranger file manager)

no Deb pkg avail

___
[–] [email protected] 4 points 7 months ago* (last edited 7 months ago)

Rest of the list:

Tools pt. 2

DNS tools:

Good stuff for pentesters and security researchers:

  • contained.af
  • cryptohack.org
  • 0x00sec.org
  • hack.me
  • chall.stypr.com
  • crackmes.one
  • hackxor.net
  • tryhackme.com
  • ctftime.org
  • ctflearn.com
  • picoctf.org
### .bashrc
### CUSTOM FUNCTIONS
# https://www.linuxjournal.com/content/boost-productivity-bash-tips-and-tricks
ftext () {
        grep -iIHrn --color=always "$1" . | less -R -r
}
duplicatefind (){
        find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | \
                xargs -I{} -n1 find -type f -size {}c -print0 | \
                xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
}
generateqr (){
        # printf "$@" | curl -F-=\<- qrenco.de
        printf "$@" | qrencode -t UTF8 -o -
}

load more comments (2 replies)
[–] [email protected] 6 points 7 months ago* (last edited 7 months ago) (1 children)

I use Bash for scripts, though my interactive shell is Fish.

Usually I use #!/usr/bin/env bash as shebang. This has the advantage of searching your PATH for Bash instead of hardcoding it.

My folders are only differentiated by those in my PATH and those not.

Most of my scripts can be found here. They are purely desktop use, no syncing to any servers. Most would be useless there.

For good practice, I'd recommend using set -euo pipefail to make Bash slightly less insane and use shellcheck to check for issues.
This is personal preference, but you could avoid Bashisms like [[ and stick to POSIX sh. (Use #!/usr/bin/env sh then.)

With shortened commands the risk is that you might forget how the full command works. How reliant you want to be on those commands being present is up to you. I wouldn't implement them as scripts though, just simple aliases instead.
Scripts only make sense if you want to do something slightly more complex over multiple lines for readability.

[–] [email protected] 3 points 7 months ago (1 children)

#/usr/bin/env bash typo? #!/usr/bin/env bash

thx for the tips!

I prefer single files over aliases since I can more easily manage each command.

[–] [email protected] 3 points 7 months ago

You're right, it's #!

[–] [email protected] 2 points 7 months ago (1 children)

I use bash and I usually put /bin/bash in my scrtipts, because that's where I know it works. /bin/sh is only if it works on many/all shells.

I don't have many such scripts, so I just have one. I don't really share them, as they are made for my usecase. If I do create something that I think will help others, then yes, I share them in git somewhere.

I do have a scripts folder in my Nextcloud that I sync around with useful scripts.

Some of your examples can probably just be made into aliases with alias alias_name="command_to_run".

[–] [email protected] 1 points 7 months ago (1 children)

thx! Why do you think that aliases are better for it?

I moved away from aliases because I have a neat command management where each command is one script.

[–] [email protected] 2 points 7 months ago

I can't speak for anyone else, but for me, it's just one file to backup to keep all your custom commands (.bashrc) while it would be many files if you have a script for each.

I can't see the benefit of having a script for just one command (with arguments) unless those arguments contain variables.

[–] [email protected] 9 points 7 months ago (1 children)
load more comments (1 replies)
[–] [email protected] 28 points 7 months ago* (last edited 7 months ago) (3 children)
#!/usr/bin/env bash

A folder dotfiles as git repository and a dotfiles/install that soft links all configurations into their places.

Two files, ~/.zshrc (without secrets, could be shared) and another for secrets (sourced by .zshrc if exist secrets).

[–] [email protected] 2 points 7 months ago

dotfiles

Thanks! I'll check them out. I knew the cooncept existed but so far I didn't dig deep into managing them. This is my start I guess https://wiki.archlinux.org/title/Dotfiles

[–] [email protected] 10 points 7 months ago (1 children)

#!/usr/bin/env bash

This is the way!

[–] [email protected] 5 points 7 months ago (2 children)
[–] [email protected] 8 points 7 months ago

#!/usr/bin/env will look in PATH for bash, and bash is not always in /bin, particularly on non-Linux systems. For example, on OpenBSD it's in /usr/local/bin, as it's an optional package.

If you are sure bash is in /bin and this won't change, there's no harm in putting it directly in your shebang.

[–] [email protected] 20 points 7 months ago* (last edited 7 months ago) (2 children)

because bash isn’t always in /usr/bin/bash.

On macOS the version on /usr/bin/bash is very old (bash 3 I think?), so many users install a newer version with homebrew which ends up in PATH, which /usr/bin/env looks at.

Protip: I start every bash script with the following two lines:

#!/usr/bin/env bash
set -euo pipefail

set -e makes the script exit if any command (that’s not part of things like if-statements) exits with a non-zero exit code

set -u makes the script exit when it tries to use undefined variables

set -o pipefail will make the exit code of the pipeline have the rightmost non-zero exit status of the pipeline, instead of always the rightmost command.

load more comments (2 replies)
load more comments (1 replies)
load more comments
view more: ‹ prev next ›