this post was submitted on 20 Feb 2024
-1 points (46.2% liked)

Linux

2359 readers
10 users here now

Shit, just linux.

Use this community for anything related to linux for now, if it gets too huge maybe there will be some sort of meme/gaming/shitpost spinoff. Currently though… go nuts

founded 1 year ago
MODERATORS
[email protected]
-1
mv *.jpg is complete bullshit. (sh.itjust.works)
submitted 9 months ago by [email protected] to c/[email protected]
28 comments fedilink hide all child comments
 

'Well it's only passing mv a list of--' yeah yeah yeah, I know, and that's why I'm calling bullshit. It should be massively harder to execute filenames. Even if 1970s decisions make that the eternal hideous default: the lack of any idiot-proof standard workaround is incomprehensible.

StackOverflow's full of competing one-liners and people pointing out how each one is considered harmful. The least-skeezy options use exec. That sentence should make anyone recoil in horror.

This is not a filename problem. This is a tool problem. If a single printable character is going to silently expand into a list of names, then for god's sake, having it put each name in quotes should be fucking trivial.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 4 points 9 months ago* (last edited 9 months ago) (2 children)

This is not standard behavior. Got links to StackOverflow?

This works just fine in bash and zsh:

encoded_funky_name=IUAjJCVeJiooKVtde30nIjo7IGxzOyAmYW1wOyA8Plx8YH5gJChscykke1BBVEh9ISF8fCYmLmpwZw==
# decodes to: !@#$%^&*()[]{}'":; ls; &amp; <>\|`~`$(ls)${PATH}!!||&&.jpg

mkdir temp
cd temp
mkdir temp2

touch "$(echo "${encoded_funky_name}" | base64 -d)"

mv ./*.jpg temp2
ls -lR

mv temp2/*.jpg .
ls -lR

Note that it is recommended to use "./" before a raw * in globs to avoid filenames beginning with "-" being interpreted by the command as special arguments (this is not necessary in the above example, but it is a good habit). See https://dwheeler.com/essays/filenames-in-shell.html#globbing for more info. e.g.:

touch ./-laR 

# Bad ls command:
ls *

# Good ls command:
ls ./*

having it put each name in quotes should be fucking trivial

This is effectively what happens, except that it's actually more robust than that, since it also accounts for names with quotes and other special characters. Glob expansion does NOT output file names into the shell; it passes them as arguments to the command with no further shell processing. Arguments can contain any special characters.

Not sure why you're seeing this behavior. Are you doing something with these file names before passing them to mv? Storing them in variables? Saving them to files?

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago)

Note that it is recommended to use “./” before a raw * in globs to avoid filenames beginning with “-” being interpreted by the command as special arguments

Jackpot! I think! Based on OP's reply to me, this is almost definitely the real problem:

mv *.jpg /mnt/Example/Pictures

Where it will then die with an error like mv: invalid option -- '1' depending on the contents of the current directory.

[–] [email protected] 2 points 9 months ago (1 children)

It's literally just mv *.jpg /mnt/Example/Pictures.

Multiple StackOverflow answers take this as read.

mv ./*.jpg /mnt/Example/Pictures works fine, though, so thank you for the practical solution. I'm still disgusted there's any filename that gets treated as... not a filename. That broken stair might be older than I am.

[–] [email protected] 4 points 9 months ago* (last edited 9 months ago) (1 children)

Ahhh. That's a confusing thread, with a couple confounding factors.

The OP in that thread is attempting to a batch rename, which is out of scope of the mv command.

If you are NOT trying to change the name of the file, I recommend always making the last argument of mv your destination directory, rather than the destination file. This is strictly mandatory if you are moving multiple files.

OP in that SO thread used the filename as the destination path — a filename which did not exist, and therefore could not be resolved by the wildcard. And if it had existed, the move would have failed. Because of that, mv /data/*/Sample_*/logs/*_Data_time.err /data/*/Sample_*/logs/*_Data_time_orig.err is not a valid command.

OP cannot accomplish what they want with a single mv command. They would need to use a loop or something like find.

I agree that the lack of universal consistency in how commands handle arguments is unfortunate. There's no way around that without reinventing the entire OS. Many commands let you pass -- as an argument to signify that any subsequent arguments should not be interpreted as options even if they begin with -. But that's entirely up to the implementation of each program; it's not a shell feature.

Future OS designers: consider typed arguments. Abandon all legacy cruft.

[–] [email protected] 1 points 9 months ago (1 children)

There’s no way around that without reinventing the entire OS.

Disagree. If wildcard expansion is not a feature of the tool, it shouldn't be up to the tool to shape how each expanded input is passed. Some hideous escape syntax ought to force *.jpg to appear as "-a.jpg" instead of -a.jpg, once it reaches mv.

Instead * is acting as a cheeky little ls alternative, minus any degree of formatting control, and piping actual ls input into mv is similarly fraught with complications.

[–] [email protected] 2 points 9 months ago (1 children)

Some hideous escape syntax ought to force *.jpg to appear as "-a.jpg" instead of -a.jpg, once it reaches mv

The issue here is that there's no difference here between -a.jpg and "-a.jpg".

Go ahead and try these commands. They are 100% equivalent:

ls -laR
ls "-laR"
ls '-laR'

The reason these are equivalent is because quoting (and expansion, and globbing) is processed in the shell. The shell then passes the evaluated string as an argument to the tool. All of those evaluate to the same string, so they look exactly the same to the ls command.

What is the solution if there's a file name that matches something the tool thinks is an argument? What would the correct behavior be?

The only solution is to include some way to communicate to the tool what each argument is. In practice, you could do ls -- * and it would work even if the first filename was "-laR". However, not all tools use the -- convention. This is really core to the design of the OS, in that command arguments have no type.

[–] [email protected] -1 points 9 months ago (1 children)

Jesus. That's beyond Javascript levels of "helpfully" reinterpreting strings. That's borderline Excel behavior.

What is the point of strings allowing every character besides \0 if they're gonna eat quotation marks?

[–] [email protected] 3 points 9 months ago* (last edited 9 months ago) (2 children)

All I can tell you is that, in my opinion, it's ridiculous and terrible that old-school terminals haven't been replaced yet with something more user-friendly and self-explanatory, at least in the same-machine user space. But given that they are what they are, some basic understanding of what shells do is required in order to use them, and you don't have that understanding (I don't fault you for this).

The key point here is that programs/commands always receive an array of string arguments, and it's the shell's job to translate your command line into that. Quoting (like in -m="my message"), variable expansion (like $HOME) and various other things are processed by the shell and not the program, and the expectation is that the user knows this. So quotes are never visible to programs, and the upside is that programs never need to process such quotes - making these features universal without having various random bugs in each program's implementation.

[–] [email protected] 1 points 9 months ago (1 children)

I've been using command-line programs for twenty-five years. "Basic" is not the issue, here. This is obscene edge-case behavior for what honestly should be a Hello World level example.

Thoroughly explicable causes do not make the outcome any less of a problem.

[–] [email protected] 1 points 9 months ago

I’ve been using command-line programs for twenty-five years.

A general statement (not about you):

I've been using my human body for over 25 years (exact number withheld) and still don't know shit about how it works and what it does. I don't even know the muscle groups, which most people who care about their health learn. But I still use them. Years of experience don't always translate to understanding and knowledge.

More specifically about you (and me): I'm sure you know a lot about how to use the shell, and I'll absolutely bet you know more tricks than me to get shit done. But you still didn't know about the quoting thing. I have a tendency to dig into understanding how shit works while not developing my actual skills in using them. You might have the opposite tendency. The best is somewhere in the middle, I think.

[–] [email protected] 2 points 9 months ago (1 children)

This is exactly correct. This is why I say it would require a whole new OS with no regard for compatibility with current systems.

As long as arguments are typeless and transmitted in a single array, there is no universal way to make programs distinguish between an option and file name that are identical. Full stop. No way. This has nothing to do with quoting schemes.

There are conventions to work around this (like --) but that is at the command level, NOT the shell level. It is not universal.

[–] [email protected] 1 points 9 months ago (1 children)

Okay, I pretty much agree with you in theory, but in practice you're basically describing PowerShell which I just cannot. :P

[–] [email protected] 2 points 9 months ago

LOL. Now I feel dirty.

To be fair, there's a lot that PowerShell does much better than Bash and similar shells. Piping tables as tables is rad.