larryhastings/appeal

Command-line parsing library for Python 3.

appeal's Issues

Figure out how to communicate the concept of "argument groups" better in usage

Consider this program:

import appeal
app = appeal.Appeal()
def int_float(i: int, f:float):
    return (i, f)
@app.command()
def fgrep(value, i_f:int_float=None):
    ...
app.main()

The fgrep command now requires either one or three positional arguments:

• One positional argument sets value to that argument.
• Three positional arguments sets value, and then i and f for int_float.

But two positional arguments is illegal. If you specify two positional arguments, you'll get a usage error like this:

Error: fgrep requires 2 arguments in this argument group.

This is correct, but it's meaningless to the user... unless they already understand how Appeal thinks. How else would they know what an "argument group" is?

I wanna fix it. But I'm not sure how. So, this issue is a place to put blue-sky proposals. What would be the platonic ideal error message for Appeal to issue here? If we (I) can come up with something that communicates the concept clearly to the user, by golly I'll figure out how to make it work.

Wrong command use response is too unspecific

When I a use command wrongly Appeal bascially tells me "you did use the command wrong". It would be nice to be more specific about what exactly I did wrong.

E.g. with the following script

#script.py
import appeal

app = appeal.Appeal()

@app.command()
def f(a:int):
    pass


if __name__ == "__main__":
    app.main()

when I run python script.py f "not an int" Appeal returns

usage: script.py command

Commands:

    f
    help            Print usage documentation on a specific command.

It would be nice to get this + something like "parameter a needs to be of type int but got passed as type str"

Appeal uses equals-signs-in-f-strings, but that's a 3.8 feature and Appeal supports 3.6+

I just installed appeal 0.5.1 into a venv created using Python 3.7.4 on Windows (using pip install appeal). I can't seem to import it, and the error message makes no sense to me:

Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import appeal
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<fstring>", line 1
    (options=)
            ^
SyntaxError: invalid syntax
>>>

appeal steals all stdout (bug?)

First of all, I really like appeal!

I wanted to use cog to update the help text in the README of a CLI that I build with appeal. For this I just wanted to capture stdout. However, appeal seems to capture all stdout and does not give it back.

Here is a little debug script.

import io
from contextlib import redirect_stdout

from my_package.__main__ import main  # this is the appeal cli

# main()  # if this is enabled, appeal steals all output. Should this be the case?

print("before")
temp_out = io.StringIO()
with redirect_stdout(temp_out):
    print("hello")
    # main()  # this is really what I want to capture to get the help text
print("after")

print(temp_out.getvalue())

Output:

before
after
hello

If I uncomment the first main(), I only get the help of main, nothing else.
If I uncomment the second main(), I only get

before

I'm not sure if this is the intended behavior or if there is a better way to do what I wanted to do or if this is a bug.

---

Note: with click I normally use the CLIRunner to conveniently captures stdout and then update the readme:

import cog
from click.testing import CliRunner
runner = CliRunner()
result = runner.invoke(my_cli.cli, ["--help"])
cog.out(resut.output)

Typing loguru logger - AttributeError: 'NoneType' object has no attribute 'get_signature'

Hi love the approach of appeal, came here from hearing the "Python Bytes" Podcast 🎧 .
I ran into a problem, that seems on the first sight pretty self-explanatory (and it might as well is 🤷 ).
The error is: 'NoneType' object has no attribute 'get_signature'
Below is a MRE of the error. It stems from giving the type loguru.logger (which is an instance of loguru._Logger Logger class) to the function parameter log.

MRE

import appeal
from loguru import logger

app = appeal.Appeal()


@app.command()
def hello_world_logger(log: logger):
    log.info("Hello World")


if __name__ == "__main__":
    app.main()

The Error

   File "/.../python3.10/site-packages/appeal/argument_grouping.py", line 138, in __init__
     self.converter = Function(callable, default, name, collapse_degenerate=collapse_degenerate, signature=signature)
   File "/.../python3.10/site-packages/appeal/argument_grouping.py", line 172, in __init__
     for name, p in signature(parameter).parameters.items():
   File "/.../python3.10/site-packages/appeal/__init__.py", line 2277, in signature
     signature = cls.get_signature(p)
 AttributeError: 'NoneType' object has no attribute 'get_signature'

In search of a solution

I tried to circumvent it by changing the type hint from loguru import _logger and give the parameter log the type _logger.Logger,
resulting in ImportError: cannot import name '_loguru' from 'loguru' (.../python3.10/site-packages/loguru/__init__.py)
This was not to surprising, as the intention to have the module (_loguru) private was obvious from the prefix _ , but I was not even aware that it is possible to have submodules not importable. However this is more related to loguru, then to appeal, back to the topic:

Is there any way to make appeal work with the loguru.logger type hint?

Passing global options to commands

Hello,

Say I want to handle the command line from the README example:

% ./script.py --debug add --flag ro -v -xz myfile.txt

What's the intended way for the add command get info about the --debug option?

(Is too early to ask questions like this?)

"--" can not be passed as argument

Why does python script.py f "--" not work?

# script.py
import appeal

app = appeal.Appeal()

@app.command()
def f(a):
    print(f"Received the following parameters:")
    for key, value in locals().items():
        print(f"  name: {key}")
        print(f"    value: {value}")
        print(f"    type: {type(value)}")

if __name__ == "__main__":
    app.main()

Type annotation of "bool" causes error producing usage docs

This example doesn't use type hints:

@app.command()
def foo(name, *, verbose = False):
    print(f"Hello, {name}!")
    if verbose:
        print("Here, have some verbosity")

And the output of help works ok:

usage: main.py command

Commands:

    foo
    help            Print usage documentation on a specific command.

Adding a type hint of bool like so:

@app.command()
def foo(name: str, *, verbose: bool = False):
    print(f"Hello, {name}!")
    if verbose:
        print("Here, have some verbosity")

causes a traceback.

Consider changing when optional options become available, and how they're presented, in usage

Something I've been pondering. Consider this Appeal API:

def optional_stuff(a, b, c, *, verbose=False, ignore_case=False): ...

@app.command()
def command(arg, stuff: optional_stuff=None): ...

Currently that would be rendered in usage as:

command arg [a [-v|--verbose] [-i|--ignore-case] b c ]

That is, a, b, c, -v, and -i are all in one "optional group". The options -v and -i only become available--only become "mapped"--once the user specifies the second positional argument, a. And yes, this really is the command-line API Appeal would create; see the Recursive Converters section of the Appeal docs to understand why.

This is consistent and understandable, but... it's also a little weird. I guess this is kind of a new command-line metaphor, having options that only become available after a certain number of positional parameters. But having them only become available after the first argument in the optional group? It's weird, right? It's not just me?

So, if we don't want that, what do we want? If we could start over and do anything, what's the usage string and command-line behavior of our dreams? I've convinced myself it's this:

command arg [[-v|--verbose] [-i|--ignore-case] a b c ]

That is, the optional part looks like conventional command-line usage, but with square brackets around it: options are shown first, then positional arguments.

But it's only fair to show this to the user if it actually works this way. If we show the user this usage string, they would quite reasonably expect this command to work:

% myscript.py command blah -v -i x y z

Can be made to work? Certainly. It means mapping the optional options at the judicious time (after arg is consumed), but not instantiating the call to optional_stuff until any of the options or arguments is specified. A little tricky but not impossible. Should it be made to work? It seems fine. If they specify -v or -i, they have to specify the three optional arguments a b and c. The hardest part seems like it'll be crafting an error message that gets this idea across to the user in an understandable way.

But this gives rise to a painful boundary condition when combined with *args:

def o2(a, *, verbose=False): ...

@app.command()
def command2(arg, *things: o2): ...

Currently this would be presented in usage as:

command2 arg [a [-v|--verbose]]...

That is, you can specify additional as as many times as you like, and each one can be followed by a -v. Completely unambiguous.

If we early-map optional options in this case, then what happens if the user runs this?

% myscript.py command2 meh first -v

Is this -v paired with the first instance of o2 (the one that gets called with a=first), or is it a preemptive option passed in to a second instance of o2 that the user never completes? It's kind of ambiguous.

In practice, it wouldn't be ambiguous--it'd consistently be one or the other. Either -v would be (re-)mapped before a was consumed, every time, or it would be (re-)mapped after a was consumed, every time. And since we're permitting -v to be used before a, then -v would have to be mapped before a was consumed, which means in the above command-line the -v would be passed in to the second call to o2(), which is incomplete because the user doesn't provide a second a. So this command-line is invalid--which I think the user would find surprising.

So I propose: we early-map optional options when they don't repeat, but we skip the early-mapping when they do repeat. I don't think that's amazingly wonderful, exactly,; it's a little inconsistent. But overall I think it minimizes unpleasantness and surprises to the user, and it's unambiguous.

There's one more thing to consider. Maybe it would be tidier if, for *args optional options, we display them in usage last. Consider:

def o3(p, q, r, *, verbose=False, ignore_case=False): ...

@app.command()
def command3(arg, *detritus: o3): ...

Which usage string is nicer?

1. command3 arg [p [-v|--verbose] [-i|--ignore-case] q r]...

2. command3 arg [p q r [-v|--verbose] [-i|--ignore-case]]...

I think 2. is prettier.

Note that I don't actually propose delaying mapping those optional options until the end. Between you and me, they'll still be mapped after the first optional argument (in this case p). It's just the usage string that we're tweaking.

Improve the error message generated when a required parameter appears after a converter with a VAR_POSITIONAL parameter

Thank's for writing appeal it's a cool package, and the README is a pleasure to read 🥇

I've come across an issue with using the pathlib.Path type, since I wanted to specify path arguments with the rich pathlib API, and I encountered this awkward issue.

Example:

import pathlib

import appeal

app = appeal.Appeal()


@app.command()
def paths_exist(path1: pathlib.Path, path2: pathlib.Path):
    exists_result1 = "exists" if path1.exists() else "does not exist"
    print(f"path1 {exists_result1}")
    exists_result2 = "exists" if path2.exists() else "does not exist"
    print(f"path2 {exists_result2}")


if __name__ == "__main__":
    app.main()

Adding an annotation in the first argument raises an error:

$ python cli.py

Traceback (most recent call last):
  File "./appeal_test/cli.py", line 17, in <module>
    app.main()
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 4203, in main
    sys.exit(self.process(args))
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 4179, in process
    return self.help()
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 4045, in help
    appeal.usage(usage=True, summary=True, doc=True)
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 4001, in usage
    usage_str, summary_str, doc_str = self.render_docstring(commands=self.commands, override_doc=docstring)
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 3897, in render_docstring
    usage_str, split_summary, doc_sections = self.compute_usage(commands=commands, override_doc=override_doc)
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 3310, in compute_usage
    child.analyze()
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 4084, in analyze
    self._analyze_attribute("_global")
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 4076, in _analyze_attribute
    program = charm_compile(self, callable)
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 1398, in charm_compile
    program = cc.compile(callable, default, is_option=is_option)
  File "./venv/lib/python3.10/site-packages/appeal/__init__.py", line 1255, in compile
    pg = argument_grouping.ParameterGrouper(callable, default, signature=signature)
  File "./venv/lib/python3.10/site-packages/appeal/argument_grouping.py", line 451, in __init__
    self.required, self.optional = pgf.analyze()
  File "./venv/lib/python3.10/site-packages/appeal/argument_grouping.py", line 263, in analyze
    return self.third_pass()
  File "./venv/lib/python3.10/site-packages/appeal/argument_grouping.py", line 232, in third_pass
    raise ValueError(f'Required parameter "{breadcrumb}.{p}" found after VAR_POSITIONAL parameter "{found_var_positional}"')
ValueError: Required parameter "paths_exist.path2" found after VAR_POSITIONAL parameter "paths_exist.path1.args."

Removing the first annotation, and only keeping the annotation in the 2nd argument (and possibly any next params) doesn't raise an error but changes the argument to appear as [args]... in the usage line (even though it's a single argument, and not a *arg.

Using appearl 0.5 from pypi on python 3.10

Traceback is not useful when a command is missing arguments

When running a command without its required arguments, the following traceback is shown:

$ python api.py create
Traceback (most recent call last):
  File "/Users/nuttab01/Projects/bbc/newslabs-highlights/cli/api.py", line 111, in <module>
    app.main()
  File "/Users/nuttab01/.virtualenvs/highlights/lib/python3.9/site-packages/appeal/__init__.py", line 4203, in main
    sys.exit(self.process(args))
  File "/Users/nuttab01/.virtualenvs/highlights/lib/python3.9/site-packages/appeal/__init__.py", line 4195, in process
    result = self.execute(commands)
  File "/Users/nuttab01/.virtualenvs/highlights/lib/python3.9/site-packages/appeal/__init__.py", line 4161, in execute
    result = command.execute()
  File "/Users/nuttab01/.virtualenvs/highlights/lib/python3.9/site-packages/appeal/__init__.py", line 2296, in execute
    return self.callable(*self.args, **self.kwargs)
TypeError: create() missing 2 required positional arguments: 'title' and 'in_time'

My thoughts are that this is a perfectly normal expected error for the user to make, and that while the exception message provides useful context, the traceback is just noise. At this point they just need to be reminded of the usage for this command, and informed of which arguments were missing.

It feels like this should just be handled and wrapped up in a simple error message. Perhaps the exception message could be reformatted slightly to be more about the parser's expectations than the Python function's expectations.

Suppress ValueError in "convert" phase, convert to usage exception

Currently, if a user runs a program using Appeal:

import appeal
app = appeal.Appeal()
@app.global_command()
def main(a: int=0, b: str=''):
    print(f"main a={a} b='{b}'")
app.main()

and supplies a bad parameter to a converter:

% python bad.py 33.5 abc

they're rewarded with a traceback and a ValueError. Appeal should catch these and intelligently print a usage error. (That was the whole point of separating the "convert" phase from the "execute" phase, after all.)

Bug: README mentions missing document for how to write documentation

https://github.com/larryhastings/appeal#writing-help mentions the document appeal/notes/writing.documentation.txt which does not exist.