okdana/zsh

[FORK] Mirror of the Z shell source code repository.

zsh's Issues

Offer a ZSH_EXECUTABLE_PATH or similar?

(workers/42202)

Sebastian had wanted some kind of parameter that provided the real, absolute path to the running zsh executable. I responded:

It's slightly irritating because there's no single cross-platform way to do it.
On Linux you can use /proc/self/exe, and some BSDs have a similar mechanism, but
then there's macOS and Solaris and AIX and so on.

I have wanted this feature in the past myself, though, so i would definitely be
in favour of a ZSH_EXECUTABLE_PATH or something. In fact i may look into it now
that you bring it up.

I did look into it, and i came up with a solution that works reliably on all major platforms except for OpenBSD — see my dana/execpath branch. I didn't post it, though; was worried it was too much effort for a feature that not many people will use. I did find that there was some demand for it, however — see below. Not sure if i want to proceed with it or not. If i do, maybe it belongs in zsh/system or something instead?

• https://unix.stackexchange.com/questions/87061/absolute-path-to-currently-running-zsh
• https://www.zsh.org/mla/users/2013/msg00674.html
• https://www.zsh.org/mla/users/2017/msg00530.html
• https://www.zsh.org/mla/workers/2018/msg00008.html
• https://stackoverflow.com/questions/3572030/bash-script-absolute-path-with-osx
• https://stackoverflow.com/questions/10412162/how-can-i-simply-retrieve-the-absolute-path-of-the-current-script-multi-os-solu

Array elements not restored correctly after pre-command re-assignment

(workers/42238 [mentioned])

On 3 Jan 2018, at 18:03, dana <dana@xxxxxxx> wrote:

The way i had handled this in the _busybox function is:

_cmd_variant[${words[1]}]=busybox _normal

That way you can temporarily override what _pick_variant thinks the actual
variant is. This seems to work quite well, but i did feel some guilt about it,
since as you mention it's circumventing the interface.

I was playing with this some more to see if i could come up with a better way
and actually discovered a problem with the method i described: it is not a
temporary override. Apparently pre-command re-assignment of array members
doesn't work the same way it does with scalars — zsh doesn't put the value back
the way it was when it's done executing the command. I assume the same code is
involved here as with the issue i brought up in workers/42097....

I am not actually sure if it is related (workers/42097 = my #4), but maybe. Didn't look into it much.

zsh's own completion function is sad

(workers/42965 [mentioned])

zsh's own completion function (that is, _zsh) is a good demonstration of _arguments --, but it just doesn't feel very good. I think it's kind of depressing that we now complete bash better than we complete zsh itself. I'd like to improve it, but i'm unsure what the best way to go about doing it would be — all i can think of is manually enumerating and describing all 180 shell options, and that feels weird. If we are going to keep a list of shell-option descriptions, we should make it available for other purposes (maybe allopt could use them, for example), and we need to make sure it stays updated whenever new options are added.

Date-format completion is broken

Mikachu mentioned this last year, but i never bothered to look into it. The date-format completion for date is broken — the descriptions don't show and the _describe output is generally messed up. Date-format completion for strftime seems to work fine though. It's at least partially related to the fact that there are two specifiers (%b and %h) that have the same description; remove one of those, and the problem goes away. But i can't figure out where specifically the problem lies, nor why it only affects some completion functions and not others.

Tied parameters not restored correctly after pre-command re-assignment

(workers/42097)

Daniel's recent message to zsh-users got me experimenting with the use of tied
parameters in 'pre-command' re-assignments (i.e., the foo=bar baz form of
command).

I found that, in 5.4.1 and 5.4.2-dev, tied parameters are not restored correctly
to their previous values if re-assigned in this way after calling a shell
function or built-in:

% typeset -T SCA arr
% arr=( a b )
% arr=( b c ) typeset -p SCA
typeset SCA=b:c
% typeset -p SCA
typeset: no such variable: SCA

(It works the other way around too — with SCA=b:c ... it will unset arr.)

...

This doesn't happen when calling external commands, since the restoration isn't
relevant there....

Bart replied:

This works correctly if POSIX_BUILTINS is set, so I would guess it has
some relation to the thread on zsh-workers about "local -r path".

I think he's referring to workers/42051, and specifically workers/42055.

:A modifier reacts badly to pattern meta-characters

(workers/42259)

Hey there. I ran into the following problem today:

% print -r ${${:-a-b}:a}
/Users/dana/a-b
% print -r ${${:-a-b}:A}
/Users/dana/a?b

The ? in the second one is 0x9b (Dash).

Both chabspath() (:a) and chrealpath() (:A) receive the input as $'a\x9bb'. :a
doesn't care, because all it's doing is manipulating . and / in the string.
It is a concern for :A, though, because it has to pass the string to realpath(),
and then it has to metafy() the result. There are two related issues, then:

1. chrealpath() can't resolve paths passed into it this way if they contain
   pattern meta characters (any of them, not just -). It calls unmetafy(), but
   that doesn't help here.

2. chrealpath() metafies the result at the end, converting (e.g.) $'.../a\x9bb'
   into $'.../a\x83\xbbb', which breaks the expansion even in cases where it'd
   be accidentally correct (like the one above).

(Quoting/escaping the expansion works around these. So does assigning 'a-b' to a
variable first, as long as you don't do something like ${${~str}:A}.)

Simply having chrealpath() call untokenize() after unmetafy() fixes the above
test case, but then there's a new problem: ${:-a-b} and ${:-$'a\x9bb'} are
indistinguishable to the function and thus give the same result. This would
break resolution of multi-byte file names, as in ${${:-ツ}:A} (ツ = $'\u30c4' =
$'\xe3\x83\x84').

Just to clarify, both :a and :P seem unaffected, though in :a's case it seems like it only gets it right by accident. I didn't check the actual code for :P.

`kill -l` with bogus numeric signal n outputs n & 127

(unposted)

% kill -l 9000 8000 7000
40
64
88

That's related to this code in jobs.c:

} else {
    if (WIFSIGNALED(sig))
        sig = WTERMSIG(sig);
    else if (WIFSTOPPED(sig))
        sig = WSTOPSIG(sig);
    if (1 <= sig && sig <= SIGCOUNT)
        printf("%s\n", sigs[sig]);
    else
        printf("%d\n", sig);
}

I know 127 is kind of a special number but i don't understand why it's doing this and i feel like it's undesirable. bash behaves more like i would expect:

$ kill -l 9000
bash: kill: 9000: invalid signal specification

Enhancements to getopts re: +o options

(unposted)

Hadn't got around to posting this to the ML yet, but it interests me:

zsh's getopts is peculiar in that if you give it an option spec abc, it defines not only the options -a, -b, and -c (as all other Bourne-like shells do), but also +a, +b, and +c.

There are two (sort of competing) limitations related to this functionality that i think could be addressed:

1. Issue: For people who only want the 'standard' behaviour, there is no 'built-in' way to tell zsh not to treat the + variants as options. You have to manually handle them in your loop. (I don't even think most people realise that they have to do this, so there are probably a tonne of zsh scripts out there that either have confusing behaviour or just break entirely if someone gives them an argument with a leading +.)

   Proposal: Have getopts treat +-prefixed arguments as regular operands — i.e., halt option parsing when it encounters them — when some shell option (maybe posix_builtins, if nothing else?) is set.

2. Issue: For people who actually like the + variants, there is no way to define them separately from the - ones. If you have a -o, you must have a +o, and if -o takes an argument, +o must take one as well. The former is easily addressed with an extra case clause in the loop, but the latter is unsolvable without giving up on getopts entirely.

   Proposal: Support separate - and + variant definitions in the option spec. For example, getopts -a:+a could define an option -a which takes an argument and an option +a which does not take an argument. The existing 'unadorned' syntax (as in getopts a) would continue to define both -a and +a together.

The main issue with this idea is that it severely complicates the ability to define the options -+, ++, and +-, to the point where you'd probably just want to disallow them entirely. (For obvious reasons, it's already not possible to define the option --.) But i don't think that's a major concern: POSIX says that the use of other option characters that are not alphanumeric produces unspecified results, and those particular options are too confusing to be useful anyway.

I am personally more interested in the second proposal, since i really like the flexibility offered by the two different option forms and would like to round it out. But i'm willing to work on either issue, if it seems reasonable.

Completion of options containing special characters is unreliable

You're supposed to be able to include special characters (+ and = are the only ones documented, but ideally you'd want also :, \, and [) in option names defined for completion by escaping them with a back-slash. This sort of works, but not really. Things get confused in the bowels of the completion system because the back-slashes aren't removed from the names before storing them. At the same time, many of the higher-level functions, like _arguments and _describe, handle characters like : and \ too naïvely.

I've submitted a patch, workers/43922, which fixes this well enough for the C stuff, but it either doesn't affect or subtly changes the higher-level behaviour. These are really weird edge cases that probably nobody will ever care about, but it would be nice to fix them if it's not too ridiculous.

De-duplication of unsorted completion possibilities is very slow

(workers/43124)

I would have expected that the line

zstyle ':completion:*' sort false

speeds things up if there is a huge number of completions.
However, quite the opposite is true.
If the above zstyle is in effect, the completion for the
artificial command 2 defined by the completion file

#compdef 2
local expl
_description a expl a
compadd "$expl[@]" - {1..40000}

takes ages. Inserting e.g. touch commands, it can be verified
that the delay happens after the return of the completion function,
i.e. it is not the compadd itself which is slow.

Moreover, without the _description line there is no delay.

This is caused by the poorly optimised de-duplication method used for compadd -V — see compcore.c @ 3239 (in particular, the call to matcheq()).

err_exit misbehaviour involving function call inside else

(workers/42126)

The following script prints 0 in ksh93, dash, and bash, but 1 in zsh:

fn() { return 1; }
(
  set -e
  if [ x = y ]; then
    fn || :
  else
    fn || :
  fi
  :
)
echo $?

It doesn't attempt to execute the || : part at all, just immediately returns
after fn.

It seems like it has something to do specifically with calling a function inside
the body of an else. If i change x = y to x = x, then the first fn || :
behaves as expected. Or if i change fn to false, that works too. Or if i
remove the conditional entirely.

Bart replied:

See the thread w/Subject: "set -e" handling is broken with zsh 5.3.1 and 5.4.1

In particular workers/41608 and workers/41615 ... I wonder if you have
found the regression that wasn't found at the time of that patch.

At the very least this sounds related.

Still an issue in 5.5.1.

Issues with signames.c generation (SIGRTMIN, &c.)

(workers/43134 [mentioned])

The way zsh generates its list of recognised signals (signames1.awk -> signames2.awk -> signames.c) isn't ideal:

1. When zsh can't find the definition for a signal in between the min and the max, it just puts the number in its place. This is useful in some ways — it ensures that subsequent signal numbers aren't affected by the missing ones, and in the event that the signal actually is valid it ensures that we at least recognise the signal by number, even if we don't know the name.

   However, it's not useful when the signal is actually undefined. In that case, we have a scenario where zsh thinks a signal exists when it doesn't at all. This is (was) especially noticeable on AIX, which leaves signals 26 and 40 through 59 undefined. In terms of platforms that people actually give a shit about, it isn't currently a problem, but it would be if we addressed the following points.

2. zsh does not support SIGRT* signals on any platform (except SIGRTMIN on Solaris). That's because...

3. zsh can not generate signals which are defined in terms of other macros, expressions, or function calls:

   // Example from glibc
   #define SIGRTMIN __SIGRTMIN
   // Example from OpenIndiana
   #define SIGRTMIN ((int)_sysconf(_SC_SIGRT_MIN))

   I don't see a way around this except to actually compile something from C to evaluate those macros. That'll require a few changes to the signames.c-generation pipe-line.

4. In order to properly support SIGRT* signals, we'll also need to generate the intervening ones, and we'll need to give them symbolic names. Some systems use names like SIGRTMIN+1 whilst others use SIGRT1. I guess we should probably support both.

References:

• https://github.com/okdana/zsh/blob/master/configure.ac#L1468
• https://github.com/okdana/zsh/blob/master/Src/signames1.awk
• https://github.com/okdana/zsh/blob/master/Src/signames2.awk
• https://www.rpi.edu/dept/acm/packages/egcs/1.1.2/rs_aix41/lib/gcc-lib/powerpc-ibm-aix4.3.1.0/egcs-2.91.66/include/sys/signal.h
• https://github.com/freebsd/freebsd/blob/82974662ad9f9ece5f8374d2c898e83bd03aece9/sys/sys/signal.h#L136
• https://github.com/illumos/illumos-gate/blob/f76886de6cd6914424d9f6c25bd9d93d87889269/usr/src/boot/sys/sys/signal.h#L117

Ability to reference outer-scope parameter?

I have often wished for the ability to directly reference an outer-scope parameter (bypassing any local parameters of the same name). This would be most useful for supporting functions that take the name of a parameter to be acted upon. Consider the following:

# Assign a value to the named association
# $1 => the name of the association to assign to
mkassoc() {
  local -A a_
  a_=( foo 1 bar 2 ) # Pretend this is useful
  : ${(PAA)1::="${(@kv)a_}"}
}
# This works
local -A a;  mkassoc a
# This doesn't — it only modifies a_ in the local scope
local -A a_; mkassoc a_

What i'd like is some feature — whether an expansion flag or an option to typeset — that tells zsh to use the outer-scope version of the parameter.

I was able to get this working for parameter reads pretty easily (only like five lines of code changed), but parameter writes (as in the above example) are more difficult because those are performed by name only. Many parts of the code seem to have the expectation that they'll be able to access the correct parameter by name, in fact.

The only way i can think of to get around that (besides fucking with the whole API) is maybe like making a clone of the original parameter but with some special prefix to the name that can't be used by a real parameter (like outer/). But that feels shitty. idk.

In the mean time, there are only two feasible work-arounds that i know of:

1. Prefix or suffix all of the function's local variables by a string of characters that's unlikely to be used in an outer scope. In order to allow two such functions to call each other, the strings would have to be pretty unique — maybe the first four characters of the base32 of the raw output from a hash of the function name. Even then, the function would still break if it called itself recursively.

2. Use a function EXIT trap (in combination with some very careful quoting) to set the outer-scope parameter after the function has returned to the calling scope. The setting of the parameter itself should be quite reliable, but you would still have issues if you needed to do anything with the parameter from inside the function (like check to see if it's defined).

Completion menu order doesn't always respect locale

(workers/42292)

The way completion possibilities are sorted in the menu has been bothering me,
but i was inspired to actually look into it today when i noticed that one
particular command was being sorted the way i wanted.

Here is an (abbreviated) example of the behaviour i see:

% export LC_ALL=C
% pgrep -<TAB>
-F -G -L -P -U -a -d -f -g -i -l -n -o -q -t -u -v -x

% export LC_ALL=en_GB.UTF-8
% pgrep -<TAB>
-F -G -L -P -U -a -d -f -g -i -l -n -o -q -t -u -v -x

% export LC_ALL=C
% pkill -<TAB>
-F -G -L -P -U -a -f -g -i -n -o -q -t -u -v -x

% export LC_ALL=en_GB.UTF-8
% pkill -<TAB>
-a -f -F -g -G -i -L -n -o -P -q -t -u -U -v -x

pkill is doing what i want — it respects my locale's collation settings. pgrep
is not — it always uses C/ASCII sorting.

The difference seems to be that pkill has 'grouped' options — i.e., multiple
options with the same description. In this case, the grouped ones are the
names of the signals, which all have the description 'signal'. If you
unset signals so that the function can't generate those options, pkill loses
its grouped options and it (mis)behaves like pgrep.

I tracked this down as far as cd_prep(). When there are grouped options, it
calls eltpcmp(), which in turn calls strcoll(). When there aren't grouped
options... well, it doesn't do that. Had to stop looking there.

The inconsistency affects many different completions. For example, ls and top
respect the locale, but ping and typeset do not. It's really irritating now that
i've noticed it.

no_unset doesn't apply in arithmetic contexts

(unposted)

bash:

$ ( set -u; (( foo++ )); echo $foo )
bash: foo: unbound variable

zsh:

% ( set -u; (( foo++ )); echo $foo )
1

I didn't really mention this to anyone, but i looked at it briefly and the fix didn't seem trivial. I'd have to see it again though.

Make (k) and (v) flags work with arrays

(workers/19572, workers/42950)

zsh actually has the (k) flag for this, but it only works on associations. zsh
also supports the ! syntax (which bash 3 borrowed from ksh93) when in ksh-
emulation mode, but since it's effectively an alias for (k) it also works only
on associations. To be clear, in both bash 4 and ksh93 it works as expected on
both.

I feel like the intuitive thing would be for (k) and (v) to work on arrays
equivalently to the way they work on associations. In other words...

% arr=( foo bar baz )
% print ${(k)arr}
1 2 3
% print ${(kv)arr}
1 foo 2 bar 3 baz

Looking at the manual, it seems to me that the current effect of those two flags
on arrays is actually undefined (except when used in conjunction with a
subscript), so... would that not be a reasonable change to make, if someone were
inclined?

Looks like Oliver was considering exactly that, once upon a time:
http://www.zsh.org/mla/workers/2004/msg00226.html

Peter pointed out that this isn't really necessary, at least not for what the OP had asked about, since you can use {1..$#arr} to enumerate array indexes. But the (k)/(v) behaviour from associations would still be useful in some cases with arrays — for example, it would be nice to have "${(@kv)arr}" just work as expected, without needing to hack around it.