Consider changing when optional options become available, and how they're presented, in usage

[larryhastings]

Consider changing when optional options become available, and how they're presented, in usage

[larryhastings]

I have been marinating on this for a while. And now I think I might be changing my mind.

Back in the day, options were required to come before (positional) arguments on command-lines. You couldn't say fgrep foo -i, you had to say fgrep -i foo. Allowing the former is a modern relaxation of the old rules.

So maybe "painful" isn't the word to describe the boundary condition described above. It's just a fact of life.

If my example above:

def o2(a, *, verbose=False): ...

@app.command()
def command2(arg, *things: o2): ...

had usage like this:

command2 arg [[-v|--verbose] a]...

where you were required to put option -v before the positional argument a, that doesn't seem all that bad.

To be precise: Appeal would require the options to be specified on the command-line before the last positional argument mapping to the args parameter. If your code looked like this:

def o3(a, b, c, *, verbose=False): ...

@app.command()
def command3(arg, *things: o3): ...

then usage would look like this:

command3 arg [[-v|--verbose] a b c]...

but in actuality you could specify the -v before a, after a but before b, or after b but before c. Just as long as it was before c. If you specified -v after c, it would map to the next invocation of o3().

So in this command-line:

% myscript.py command3 arg a b -v c

we'd call o3() once with verbose=True.

As a reminder, in this new world order, options in an optional group get optimistically lazy-mapped to the command-line when we complete all the positional arguments from the previous group.

--

I worry I'm not explaining this well, so I'm going to walk you--the reader--through it all the way.

When Appeal parses the command-line, it's building function calls. At at the point that Appeal runs out of input, if the function call is valid, Appeal is happy. If the function call is invalid, Appeal is unhappy and throws an exception.

Let's say Appeal is in the middle of parsing this command-line:

% myscript command3 myarg ...

myscript was the program the user ran, from their shell; the rest are arguments to that program, which Appeal is parsing. So far Appeal has seen two things, command3 and myarg. (The ellipses there indicate "the rest of the input, whatever it is".) From this, Appeal has started building a call to command3(), and has filled its arg parameter with myarg. At this exact moment, the function is callable--if there was no more input, Appeal would be happy and could successfully call command3().

Internally, Appeal has finished command3's first argument group, which is a mandatory group. All it contained was one positional argument, arg. The next argument group is optional, and it maps one option (-v) and three positional arguments. So if there was another positional argument on the command-line, Appeal would say "okay, we're building a call to o3() now`, and it would require a total of three positional arguments.

At this exact moment--where we haven't noticed whether or not there's another positional argument on the command-line--Appeal will now provisionally map -v to the command-line, by which I mean it will add -v to its internal list of options that are valid right now. But that wouldn't necessitate calling o3(). If there were no more arguments, Appeal will say "oh I guess we're not calling o3() after all, that's fine", and it'd be happy and call command3() as per the above.

Anyway. After Appeal provisionally maps -v to the command-line, the next thing it will do is pull out the next argument from the command-line argument iterator. Which, if we want to be all-encompassing here, could be one of four things:

1. The command-line could already be exhausted, and there are no more arguments of any description. In which case, Appeal is happy, and it can call command3(), and it doesn't mind that it's not ever going to call o3(). (The things argument to command3() will be an empty iterable.)
2. The next command-line argument doesn't start with a dash, in which case it's a positional argument. Appeal will say "okay, I have to build a call to o3()". This argument will be the first argument to o3(), and Appeal is gonna need two more if we're going to have a valid command-line. (The result of the call to o3() will be passed in to command3() as an entry in the things iterable.)
3. The next command-line argument could be -v, in which case Appeal will also say "okay, I have to build a call to o3()". Appeal is gonna need three more arguments to fill in for this call to o3(), for this to be a valid command-line.
4. The next command-line argument starts with a dash, but isn't -v or --verbose, in which case the command-line is illegal. (I told you we were covering all bases.)

So finally, here's the funny side-effect of these semantics. In this world, the following command-line:

% myscript.py command3 arg a b c -v

is invalid, because Appeal wants to call o3() twice. By the time Appeal sees the -v, it's already built the first call to o3()--that one's all done and ready to go. The -v it sees at the end there tells Appeal to build a second call to o3(), and we don't have any more positional arguments to use to fill it in, and so Appeal is unhappy and throws an exception. And now we need to figure out how to intelligibly communicate that to the user in a short string we can display before usage.

--

There are really three possibilities for how to handle options in repeating optional groups (*args):

1. They're provisionally bound before the first positional parameter in the repeating optional group, and get rebound after we see the last provisional parameter. (This is the scenario I described in this post in this issue.)
2. They're always bound after the first positional parameter in the repeating optional group, and only get rebound after we see the first provisional parameter in the next instance of that repeating optional group. (This is the scenario I described in my previous post in this issue.)
3. Like 2., except that the first time we bind them provisionally before the first positional parameter as per 1.

I can see the benefits of each of these. I don't expect to do 3, because I think consistency is probably better than the mild convenience afforded by its inconsistency. And, as I said at the top of this message, I'm now tilting away from 2 and towards 1.

[larryhastings]

Done!

[larryhastings]

And for the record, I went with the early-binding (and therefore early-un-binding). I think the mild surprise folks might experience on the rare occasion an option is un-bound (or re-bound) earlier than they expected is minuscule, compared to the big win of options being available early the way folks would normally expect.