Giter Site home page Giter Site logo

felixschl / neodoc Goto Github PK

View Code? Open in Web Editor NEW
227.0 7.0 9.0 2.53 MB

Beautiful, hand-crafted commandline interfaces for node.js

Home Page: https://felixschl.github.io/neodoc

License: MIT License

PureScript 84.03% JavaScript 3.68% Python 11.56% Makefile 0.12% Shell 0.60%
docopt parser nodejs argv command line parsing

neodoc's Introduction

<neodoc>

Beautiful, handcrafted command line interfaces

NPM package Build Status Build Status (appveyor) Join the chat at https://gitter.im/felixSchl/neodoc

About | Features | Installation | Usage | Language overview | License | Playground new

preview



About

<neodoc> is a revised implementation of the docopt language for node. In brief, it offers a unique way to author command lines by writing the command line's help text first and then deriving a matching parser from it, which can then be applied to user input. The advantages are numerous:

  • No boilerplate - just write your help-text
  • Full control over beautiful, hand-crafted help texts
  • Documentation comes first - hence your users come first
  • Documentation is always right - your help-text is necessarily correct
  • Version-controlled help-text - the help-text becomes a regular part of your codebase

This implementation features error reporting, both for users and developers, reading values from environment variables, type coercion and much more. For an (in-)comprehensive comparison to the original, click here. To take neodoc for a ride, click here.

A note to potential adopters and contributors: Neodoc is divided into two distinct parts — parsing the specification and parsing the argv, given the specificiation. Theoretically, the origin of the specification does not matter and the argv parser could be used standalone as it offers a more "correct" parse than most cli parsers out there, since it parses the input guided by the specification, rather than parsing the input and then matching it to the specification. See the "Features" section below. If neodoc finds adoption, I would not be surprised to see projects mimicking a yargs-like interface that use the neodoc parser, even though it somewhat defies the original idea of docopt.

Features

  • Derive command line interface from help text
  • Helpful error messages for both developers and users
  • Options-first parsing to compose large programs (see git example)
  • Fallback to alternate values: Argv -> Environment -> Defaults -> Empty
  • Convenient, concise and widely accepted POSIX-style syntax
    • -f[=ARG], --foo[=ARG] options
    • <arg>, ARG positionals
    • clone, pull, etc. commands
    • [<arg>] optional groupings
    • (<arg>) required groupings
    • [-f ARG] POSIX-style flags
    • -f[=ARG]... repeating elements
    • -- end of options separator
    • - stdin marker
    • 99% compatible with a typical git <command> --help output
    • Full overview of the language →
  • Stop parsing at any option and collect successive input as the argument to that option. Similar to -- but for named options (and their aliases).
  • Specification parsing (help-text parsing) is separated from argv parsing and can be used for other projects outside of neodoc. Work is underway to make the argv parser usable from JS as well.
  • Count repeated flags
  • Parses values into primitive JS types (bool, string, number)
  • Correct and smart argument parsing. For example, neodoc has absolutely no problem parsing this input: tar -xvzfsome-dir/some-file, given a specification of: usage: tar [-xvzf FILE] while other parses would not know that the option stack ends at -f and falsly parse this as -x -v -z -f -s -o -m -e=-dir/some-file at best.

Installation

npm install --save neodoc

Usage

neodoc.run(help | spec, opts)

Parse and apply the given docopt help text. Alternatively, pass the output of neodoc.parse. If no options are provided, apply it to process.argv and process.env. The result is a mapping of key -> value, where the key is the canonical form of the option and its alias, if available.

Options:

  • opts.dontExit - Do not exit upon error or when parsing --help or --version. Instead throw and error / return the value.
  • opts.env - Override process.env
  • opts.argv - Override process.argv
  • opts.optionsFirst - Parse until the first command or <positional> argument, then collect the rest into an array, given the help indicates another, repeatable, positional argument, e.g. : [options] <ommand> [<args>...]
  • opts.smartOptions - Enable parsing groups that "look like" options as options. For example: [-f ARG...] means [-f=ARG...]
  • opts.stopAt - Stop parsing at the given options, i.e. [ -n ]. It's value will be the rest of argv.
  • opts.requireFlags - Require flags be present in the input. In neodoc, flags are optional by default and can be omitted. This option forces the user to pass flags explicitly, failing the parse otherwise.
  • opts.laxPlacement - Relax placement rules. Positionals and commands are no longer solid anchors. The order amongs them, however, remains fixed. This implies that options can appear anywhere.
  • opts.versionFlags - An array of flags that trigger the special version behavior: Print the program version and exit with code 0.
  • opts.version - The version to print for the special version behavior. Defaults to finding the version of the nearest package.json file, relative to the executing main module. Note that disk IO is only performed if opts.versionFlags is non-empty and opts.version is not set.
  • opts.helpFlags - An array of flags that trigger the special help behavior: Print the full program help text and exit with code 0.
  • opts.repeatableOptions - Allow options to be repeated even if the spec does not explicitly allow this. This "loosens" up the parser to accept more input and makes for a more intuitive command line. Please note: repeatability is still subject to chunking (use opts.laxPlacement to relax this further).
  • opts.transforms.presolve - an array of functions to be called prior to "solving" the input. This function takes the spec as it's only parameter. At this point, the spec is mostly untouched by neodoc with the exception of smart-options which runs as a fixed transform prior to user-provided callbacks if smart-options is true. Transforms that need to be aware of option stacks and [...-options] references should run here as this information is lost during the solving transforms.
  • opts.transforms.postsolve - an array of functions to be called after "solving" the input, just prior to passing the spec to the arg-parser. This function takes the spec as it's only parameter. At this point, the spec has been fully solved, expanded and canonicalised.
  • opts.allowUnknown - Collect unknown options under a special key ? instead of failing. Useful to send an unknown subset of options to another program.

For example:

#!/usr/bin/env node

const neodoc = require('neodoc');

const args = neodoc.run(`
usage: git [--version] [--help] [-C <path>] [-c <name=value>]
           [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
           [-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
           [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
           <command> [<args>...]
`, { optionsFirst: true, smartOptions: true });

if (args['<command>'] === 'remote') {
    const remoteArgs = neodoc.run(`
    usage:
        git remote [-v | --verbose]
        git remote add [-t <branch>] [-m <master>] [-f] [--tags|--no-tags]
                        [--mirror=<fetch|push>] <name> <url>
        git remote rename <old> <new>
        git remote remove <name>
        git remote [-v | --verbose] show [-n] <name>...
        git remote prune [-n | --dry-run] <name>...
        git remote [-v | --verbose] update [-p | --prune] [(<group> | <remote>)...]
    `, { argv: ['remote'].concat(args['<args>']), smartOptions: true })

    // ...
} else { /* ... */ }

See the examples folder for a more sophisticated version of the above example.

neodoc.parse(help, opts)

Parse the docopt text and derive the specification along with some meta information. The specification is the canonical representation of the CLI as described by it's help text and can be used for building parsers etc. The output is a plain JS object and can be serialized. The output can further be passed to neodoc.run. This avoids neodoc having to parse and solve the original help text again, since parsing JSON is a order of magnitude faster to parse.

Language overview and terminology

This section gives an overview over the neodoc cli specification language. Keywords are highlighted.

The over-arching format could be described as follows:

Usage:  <program> [<argument>...] [| <argument> [<argument>...]]
[ [or:] <program> [<argument>...] [| <argument> [<argument>...]]
]*

[options:
    [<argument> [<description and meta tags>]
    ]*
]*

Where <argument> may be any of the arguments described in the following subsections.

A full example:

usage: git fetch [options] [<repository> [<refspec>...]]
   or: git fetch [options] <group>
   or: git fetch --multiple [options] [(<repository> | <group>)...]
   or: git fetch --all [options]

options:
    -v, --verbose         be more verbose
    -q, --quiet           be more quiet
    --all                 fetch from all remotes
    -a, --append          append to .git/FETCH_HEAD instead of overwriting
    --upload-pack <path>  path to upload pack on remote end
    -f, --force           force overwrite of local branch
    -m, --multiple        fetch from multiple remotes
    -t, --tags            fetch all tags and associated objects
    [...]

1. Arguments

At the heart of the language are command line arguments. There are three fundamental types of arguments: options, positional arguments and commands. Options are arguments that start with either a single or a double dash ('-'), commands are literal matches of a certain string and positionals constitute everything else. Read on below for more detail on each argument type.

1.1. Options

Options are those arguments that start with either one or two dashes. They are referred to as "short" and "long" options respectively throughout this document and the source code of neodoc.

Options may take an potentially optional option-argument. Options that do not are referred to as flags. Options that do specify an option-argument but declare it as being optional may behave as flags if an argument could not be consumed at runtime.

The following is true for all options:

  • Options may take an optional "option-argument"
  • Options may be repeated using ...
  • Adjacent options are not fixed in position: -a -b is equivalent to -b -a. Likewise, -ab is equivalent to -ba. This also holds true for options that take option-arguments.
  • Options that are repeated collect values into an array
  • Flags that are repeated count the number of occurrences
  • Flags that are not repeated simply yield true if present
  • Flags and options with an optional option-argument can always be omitted from the input. They simply won't yield anything in the output value mapping.
  • Options may alias a short (one-character) form with a long form, e.g.: -v, --verbose
  • Options that take an argument can specify a [default: <value>] in the option section as fallback.
  • Options that take an argument can specify a [env: <key>] in the option section as fallback.
1.1.1. Long options

Long options are lead by two dashes and may take an potentially optional option-argument.
For example:

  • --long <ARG> the option-argument is loosely bound
  • --long <ARG> the option-argument is loosely bound and optional. (#55)
  • --long=<ARG> the option-argument is explicitly bound
  • --long[=<ARG>] the option-argument is explicitly bound an optional
  • [--long <ARG>] the option-argument is explicitly bound via the smart-options setting
  • [--long [<ARG>]] the option-argument is loosely bound via the smart-options setting and optional

Note that all of the above forms could be followed by a ..., indicating that this option may appear one or more times. The repeated occurrence does not necessarily need to be adjacent to the previous match. Repeated occurrences are collected into an array or into a count if the option qualifies as a flag.

Note that successive dashes are allowed: --very-long-option.

1.1.2. Short options

Short options are lead by one dash and may take an potentially optional option-argument. A short option is a one character identifier, but can be "stacked".

For example:

  • -a <ARG> the option-argument is loosely bound to -a
  • -a=<ARG> the option-argument is explicitly bound to -a
  • -a<ARG> the option-argument is explicitly bound to -a
  • -aARG the option-argument is loosely bound to -a
  • -a [<ARG>] the option-argument is loosely bound to -a. (#55)
  • -a=<ARG> the option-argument is explicitly bound to -a
  • -a[=<ARG>] the option-argument is explicitly bound to -a an optional
  • [-a <ARG>] the option-argument is explicitly bound to -a via the smart-options setting
  • [-a [<ARG>]] the option-argument is loosely bound to -a via the smart-options setting and optional

Note, however that only the last option in the "option stack" may actually bind an argument:

  • -abc is equivalent to -a -b -c
  • -abc <ARG> is equivalent to -a -b -c <ARG>
  • -abc=<ARG> is equivalent to -a -b -c=<ARG>
  • -abc[=<ARG>] is equivalent to -a -b -c=<ARG>

...essentially nothing changes when options are stacked. Key is that only the last option in the stack may bind and consume arguments.

Again, note that all of the above forms could be followed by a ..., indicating that this option may appear one or more times. It is important to note that the repeatability is assigned to all options in the stack! Repeated occurrences are collected into an array or into a count if the option qualifies as a flag (hence for all but the last options in the stack).

1.1.3. Option-arguments

Option-arguments are arguments bound to options. If an option is said to take an option argument that is not optional, any attempt to match an option without the argument will result in an immediate parse error. Should an option-argument be declared optional and not matched during parsing, it may be treated as a flag and be substituted.

1.1.3.1. Option-argument bindings
  • "loose" binding: the option-argument is in adjacent position, but needs to be confirmed in the 'options' section. Should confirmation not take place, the adjacent argument is treated as a positional.
  • "explicit" binding: the option-argument is explicitly bound due to a lack of whitespace, an equal sign or through 'smart-options'.
1.1.4 The option secion

The option section gives a chance to add more information about options, such as their default value, their alias or their backing environment variable. Furthermore, options appearing in the option section may also indicate if the option is supposed to be repeatable or not. There is more information on this topic in section "1.7 - References - [options]".

  • An alias is assigned via -v, --verbose
  • A default value is assigned via [default: value]
  • An environment variable is assigned via [env: MY_KEY]

For example:

options:
    -f, --foo BAR  This is foo bar. [env: FOO_BAR] [default: 123]

The text is pretty flexible and can be arranged as the author pleases. For example:

options:
    -f, --foo BAR...
        This is foo bar.
        [env: FOO_BAR] [default: 123]

1.2. Positionals

Positionals are arguments that do not lead with any dashes. The position of their occurrence matters and options are "bounded" by them in that an option declared before an positional argument may not occur after that positional. (#24) Positional arguments are distinguished from commands by being either enclosed in angled brackets or being all upper case.

For example:

  • <ARG> is a positional element named <ARG>
  • ARG is a positional element named ARG
  • [<ARG>] is an optional positional element named <ARG>
  • [ARG] is an optional positional element named ARG
  • [<ARG>]... is an optional positional element named <ARG> that repeats
  • <ARG>... is a positional element named <ARG> that repeats

Positional arguments either yield a single value if not repeated or an array of values if repeated. Note that contrary to options, repetition must occur directly adjacent to the previous match. (#24)

1.3. Commands

Commands are a specialized form of positionals that require to be matched literally, including casing. All other rules that apply to positionals apply to commands. They yield a boolean indicating their presence or a count indicating the number of their occurrences if repeated.

For example:

  • command must be matched with input "command" on argv
  • command... must be matched on ore more times with input "command" on argv

1.4. EOA - end-of-arguments

The EOA (end-of-arguments) is understood as the separator between known and unknown arguments. The eoa is typically -- but any option can become one by using the 'stop-at' setting.\o

For example:

  • --
  • -- ARGS
  • -- ARGS...
  • [-- ARGS...]
  • [-- [ARGS...]]
  • ...and so on — they all have the same meaning.

1.5. Stdin marker

The stdin flag is a special, unnamed short-option: -. It's presence indicates that the program should be reading from standard input.

1.6. Groups

Groups are the only recursive argument type. Groups describe one or more mutually exclusive sets of arguments called "branches". At least one branch needs to yield a successful parse for the group to succeed.

For example:

  • (foo | bar qux) means either match command foo or command bar directly followed by command qux.
  • [foo | bar qux] means either match command foo or command bar directly followed by command qux, but backtrack on failure and ignore the group.
  • (foo | bar qux)... means either match command foo or command bar directly followed by command qux, repeatedly. During repetition another branch can be matched, so this is valid: foo bar qux bar qux foo. The output is: { "foo": 2, "bar": 2, "qux": 2 }.

The following is true for all groups:

  • Groups can be repeated: ...
  • Groups can be optional using brackets: [ foo ]
  • Groups can be required using parenthesis: ( foo )
  • Groups must not be empty
  • Groups must contain 1 or more branches
  • Groups succeed if at least one branch succeeds
  • Multiple successful branch matches are weighted and scored
1.6.1. Matching branches

Branches describe multiple mutually exclusive ways to parse a valid program. Branches can appear at the top-level or in any group. Since branches are mutually exclusive, only one branch can ever succeed. If multiple branches succeed, the highest scoring winner is elected. Generally, the depth of the parse within the branch (that is how deep into the branch the parse succeeded) as well as the weighting of the matched arguments matters. Arguments that were substituted by values in environment variables, or by their defaults or empty values, will have a lower ranking score than those that were read from argv.

1.7. References - [options]

This is not a real argument and not part of the canonical specification. It is used to indicate that the entire "options" section should be expanded in it's place. Since this approach lacks information about the relation between options, options are all expanded as optional and are exchangeable with adjacent options (#57). One exception to this rule is where an option that is defined in the option section also appears directly adjacent to the [options] reference tag.

For example:

usage: prog [options] (-f | -b)
options:
    -f foo
    -b bar

This program won't accept the input -f -b as -f and -b are declared mutually exclusive from one another.

Likewise:

usage: prog [options] --foo ARG
options:
    -f, --foo ARG

Here, --foo won't be expanded again and hence remain required.


Deviations from the original

This implementation tries to be compatible where sensible, but does cut ties when it comes down to it. The universal docopt test suite has been adjusted accordingly.

  • Better Error reporting. Let the user of your utility know why input was rejected and how to fix it
  • Optional arguments. Neodoc understands --foo[=BAR] (or -f[=<bar>]) as an option that can be provided either with or without an argument.
  • Alias matches. If --verbose yields a value, so will -v (given that's the assigned alias).
  • Flags are optional by default. There's arguably no reason to force the user to explicitly pass an option that takes no argument as the absence of the flag speaks — the key will be omitted from the output. This is also the case for flags inside required groups. E.g.: The group (-a -b) will match inputs -a -b, -ab, -ba, -b -a, -b, -a and the empty input. To disable this behaviour, enable options.requireFlags (see neodoc.run).
    Please note that the default behaviour may change in a future version of neodoc — refer to #61.
  • All arguments in a group are always required. This is regardless of whether or not the group itself is required - once you start matching into the group, all elements that are indicated as required have to be matched, either by value or via fallbacks.
    For example:
    Usage: prog [<name> <type>]
    will fail prog foo, but pass prog foo bar. The rationale being that this is more general, since if the opposite behaviour (any match) was desired, it could be expressed as such:
    Usage: prog [[<name>] [<type>]]
  • No abbreviations: --ver does not match --verbose. (mis-feature in the original implementation)
  • There is no null in the resulting value map. null simply means not matched - so the key is omitted from the resulting value map.
  • Smart-options. Options can be inferred from groups that "look like" options: Usage: foo [-f FILE] would then expand to Usage: foo [-f=FILE]
  • Environment variables. Options can fall back to environment variables, if they are not explicitly defined. The order of evaluation is:
    1. User input (per process.argv)
    2. Environment variables (per [env: ...] tag)
    3. Option defaults (per [default: ...] tag)
  • Stricter document layout. Neodoc imposes more restrictions on the format of the help text in order to achieve certain goals, such as:
    • Neodoc allows associating option aliases over newlines:

      options:
       -f,
       --foo this is foo
      
    • Neodoc does not require 2 spaces between option and argument. Instead, only those arguments that visually "look like" arguments are considered for binding (i.e. all-caps: ARG and in-angles: <arg>):

      options:
       -f,
       --foo ARG
      

      Should there be any ambiguity, the option can also be explicitly bound:

      options:
       -f,
       --foo=ARG
      

      The same is true for optional arguments:

      options:
       -f,
       --foo [ARG]
       -b,
       --bar[=ARG]
      
    • Neodoc is more conservative on learning about options in order to prevent subtly introducing options:

      usage: prog [options]
      
      options:
      
      --foo this is foo, and it is similar to some-command's
              --bar in that it does qux.
      

      Here, the author is talking about --bar in another context, so it should not be considered an option to the program. Neodoc figures this out based on indentation of the previous description start.

    • Neodoc allows interspersing spaces in between usage layouts:

      usage:
        prog foo bar
      
        prog qux woo
      

      it is important to not that in this format (above), identation of the layout is required. or:

      usage: prog foo bar
      
         or: prog qux woo
      
    • Neodoc requires an Options: section title in order to parse any options (see #76 for discussion). note: I am interested in a proposal on how to lift this requirement.

License

<neodoc> is released under the MIT LICENSE. See file LICENSE for a more detailed description of its terms.

neodoc's People

Contributors

felixschl avatar jimt avatar nicholasjpaterno avatar rgrannell1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

neodoc's Issues

Add option to not match aliases

I have observed varying behaviour of git -h and git --help, so there's a usecase for not matching both, but only matching explicitly provided args.

Do not provide fallbacks for mutually exclusive options

The problem:

Usage: foo (-a | -b)

Options:
  -a FILE  [default: qux]

Then:

  • $ prog -a baz produces: {"-a": baz, "-b": false} (good)
  • $ prog -b produces: {"-a": qux, "-b": true} (bad: I cannot tell which one was user-provided)

Should be:

  • $ prog -a baz produces: {"-a": baz} (good)
  • $ prog -b produces: {"-b": true} (good)

Allow flags to repeat

Allow flags to repeat regardless if they have a .... This way, flags can cancel each other out, which is useful for scripting.

This turned out to be not so easy because it needs to cover more than a simple .... I.e. options should be able to re-occur anywhere in the input if they've been met before (i.e. validated).

Work is tracked on https://github.com/felixSchl/neodoc/tree/feature/repeatable-options.

...dev notes:

  • Wrong error messages in some cases:

    usage: prog [-a] [-b] [-c] [-d] [-e] [-f] -h <path>...
    

    yields "missing -a" for input "-h" w/ opts.requireFlags.

Expose running from a spec (w/o parsing the spec)

This would allow users to use the argv parser on it's own by providing a spec.
The spec could then even be expanded to have custom error messages etc.
This could also allow people to parse the spec using neodoc and then run alterations on top and pass it back to neodoc for parsing the argv against it.

work is tracked on: feature/parse-spec

  • Implement a fromForeign function to read a spec
  • alter neodoc.run to take either a spec object or a string: neodoc.run(spec | string, options). This will be a FFI/Foreign nightmare, but is the mast javascript friendly approach. Maybe the routing could happen on JS level in lib/docop.js.

Allow [--] to denote optional --

Currently this resolves in a parse error, probably due to a bug.

Note, however, that -- is always optional anyway, so [--] is just for emphasis.

Expand groups for more intuitive matching

Given this specification:

usage: prog [-io] -f

This will currently fail:

$ prog -i -f -o
Unmatched option: -o

The reason is that -i matches [-io] in part, consuming it when -f fails. Then -f matches -f and finally -o is left hanging.

As far as I can see, a simple rewrite fixes this:

usage: prog [-i] [-o] -f

or:

usage: prog -iof

It is essentially about reducing groups to singleton groups in order to avoid swallowing input via fallbacks... needs more thought.

Add support for ranges -[0-9]

Add ranges support: -[0-9] would expand to -0123456789 (or -1, -2 ...).
For a usecase, see here

The hardest part is speccing this up properly and keeping it general to expand
upon, e.g. to add support for other substitutions later on etc.

Compile to bash 3

Using the solved specification, compile a CLI parser written in bash.

Autocompletion

It should be fairly doable to match the current input and then upon success (or failure) use the longest match to offer choices to complete the next thing.

Problems:

  • Deal with user typing in string. Or do we have to? In bash, typing "-- is the same as typing --.

  • There may be more than one branch that qualifies. Show all relevant options.

  • Need to provide hooks, so users can bring their own completion function for option arguments and arguments.

  • We need to know "where" we are in a branch after completion stops:

    foo --bar QUX --foo --moo
    $ # -> [--bar]
    $ --bar # -> []
    $ --bar "qux" --foo # -> [--moo]
    $ --bar "qux" --foo --moo # -> []

Render errors / warnings

Render error or warning messages for the compilation of the docopt string.

These should render the entire docopt string or a reasonable subset with a squiggly line underlining the locations:

Usage: foo --bar
    foo --qux --bar
    ^^^^^^^^^^^^^^^
Warning: Usages not aligned!

Add <command> section

Similar to [option], add a special section for commands:

Usage: foo <command>

Available commands:
  bar  lorem ipsum
          foo
  qux lorem ipsum

This extends docopt significantly.

This is a bit tricky to parse, but I think the most intuitive would be to mark the indentation level of the first
word after the actual command name. Then only treat text as part of the description of the command currently being parsed if it equal or more indented than the mark.

Indicate repeatbility in option descriptions

This covers the following usecase:

Usage: foo [options] [<command> [<args>...]]
Options:
  -v, --verbose ... Set the verbosity level. Repeat the flag for higher
                    verbosity, e.g. `-vvv`. 

Then:

  • $ foo -v yields { "-v": 1, "--verbose": 1}
  • $ foo -vv yields { "-v": 2, "--verbose": 2}

Also, further scenarios cover:

Usage: foo [options] [<command> [<args>...]]
Options:
  -v..., --verbose Set the verbosity level. Repeat the flag for higher
                   verbosity, e.g. `-vvv`. 
Usage: foo [options] [<command> [<args>...]]
Options:
  -v..., --verbose... Set the verbosity level. Repeat the flag for higher
                      verbosity, e.g. `-vvv`.
Usage: foo -v... [<command> [<args>...]]
Options:
  -v..., --verbose... Set the verbosity level. Repeat the flag for higher
                      verbosity, e.g. `-vvv`.

Here, the -v... occuring in the Usage section, has to adopt the ...:
In line with commit ec73c7a, this should fail with a mismatch
solve error:

Usage: foo -v [<command> [<args>...]]
Options:
  -v..., --verbose... Set the verbosity level. Repeat the flag for higher
                      verbosity, e.g. `-vvv`.

Start by writing failing tests on a feature branch.

Update dependencies

Update dependencies to versions that use one of the new purescript features: data constructor aliases. This allows to pattern match lists using : instead of Cons x ..., and it's a blessing for nested tuples.

Prompt for missing options

Explore the idea of adding a [prompt] tag to an option, indicating that if the option has been omitted from the command line and no environment variable substitution has been found, it will prompt the user if at a TTY.

I don't imagine that be too easy, though, since all of neodoc's processing is happening in pure code and adding these side effects will need to happen somewhere in the impure entry module.

Actually, I feel like this might not even be neodoc turf. It should be easy enough to implement this on user-level using some other library or a neodoc companion library maybe.

Stricter matching of option arguments in usage with description

Currently,

Usage: foo --bar

Options:
  -b, --bar <QUX> <- `--bar` is said not to have arguments above, should it merge?

Expands to:

Usage: foo --bar

Rather than:

Usage: foo --bar=<QUX>

This brings up an interesting point: Should neodoc accept the input or reject it? On the one hand, it would
make the usage section more correct, however, on the other hand it would also make it more verbose. However, it should be disallowed to associate stacked short options (expect for the last one) with option arguments specified in the description for POSIX compatibility:

Error:

Usage: foo -bf

Options:
  -b <a>  ... <- `-b` is specified as a stacked option above, cannot take arguments!
  -f <b> ... <- `-f` is said not to have arguments above, should it merge?

POSIX compat

Neodoc is already mostly POSIX compatible, but there are a few things that can be improved:

  • "Options-first" on by default (guideline 9)
  • Required option arguments be separate from the option: -f BAR
    • Update notation in usage syntax (disallow no spaces)
  • Optional option arguments be the same argument as the option: -fBAR
    • Update notation in usage syntax (disallow spaces)
  • Denote options in singleton groups: [-f BAR] means [-f=BAR] (no
    option description required)
  • Allow passing negative numbers as option arguments #39
  • Implement guideline 8 (also see 11)

Use parser for other projects?

Hi @felixSchl!

Thanks for the awesome project. I had a quick question about it's internals (Sorry if this is a obvious question, I've never used PureScript.)

Is there a way to use the docopt parse for other projects?

Here's where I'm coming from, in case there's an easier way:

We've been discussing over at shelljs/shelljs a plugin system, which for us would be aimed at allowing people to easily impliment custom shelljs commands. (shelljs/shelljs#391)

I would really like to find a way for users to easily declare that (for example) they'd like to use git, and be able to simply run a command like shell.external('git'), and then be able to do shell.git.commit({ message: 'foo'});, and have shelljs "bashify" it, to execute git commit --message "foo".

So far, I think there are mostly two options:

  1. Specify what commands you want: (ex. shell.use(shellify('git')('commit', 'add', 'push'), then shell.git.commit/add/push work, but not shell.git.merge.)
  2. Use proxies to dynamically make methods (ex. shell.git.foobar) available. Then, you don't even have to declare your commands. (ex. shell.mycmd.foo('blah') would just automagically work, assuming it's in your $PATH. Similar to a real shell.)

But then, I had an idea: What if we did option 1, but we could detect possible commands from help output? So, for example, you run shell.external('git'), and we automagically run git help through neodoc, and use that to define shell.git[whatever]

However, I don't think there's a way to do that with the current neodoc API, so I thought I'd inquire a little more.

Any ideas on how to accomplish this, or maybe just a slap in the face and a stern warning that I've gone completely insane?

Thanks so much!

cc @nfischer, @levithomason.

Add support for a mode where positionals are not positional

This is for convenience to the user:

usage: git branch [options] [-r | -a] [--merged | --no-merged]
   or: git branch [options] [-l] [-f] <branchname> [<start-point>]
   or: git branch [options] [-r] (-d | -D) <branchname>...
   or: git branch [options] (-m | -M) [<oldbranch>] <newbranch>

e.g. git branch -m, this all works:

git branch -m foo bar
git branch foo -m bar
git branch foo bar -m

This makes it easy to add another flag at then end should the user forget one.

Rewrite options parser

The current lib/parse/meta/options parsers are pretty wild, especially the description parser.
They need to be re-written and still satisfy the tests/meta/options test suite.

  • Tame description parser - add comments where needed to illustrate the mechanics.
  • Do not reset the parser's position's index to 0! This will make it difficult to later render error messages later.

Provide fromREADME entrypoint

Provide a convenient "fromREADME" entrypoint that would use
a regular expression along the lines of

/^```(?:docopt|neodoc)\n((?:\s|.)+?(?=^```))/m

to extract the first occurrence of a docopt block from the given README
(or markdown document in general).

Provide warnings

All parsers sit atop the RWS monad, where currently the writer is Unit. We can swap it out to accumulate warnings related to parsing the spec and present it to the developer if warnings are enabled (or not disabled).

As a first step, warnings don't have to show the source spans, but that can be added, too later.

Implement --help and --version

The developer will be able to specify the option that will trigger the --help and --version convenience functions, with fallbacks to --help and --version.

  • Implement --help. If matched, print the docopt text. (feature/help)
  • Implement --version. If matched, print either the npm package version or a user-provided version string

Custom EOA option

Allow marking any option as the EOA ("--"). Usecase example here is the --nodejs option in coffeescript and livescript.

Fail at solving when arguments mismatch

Refer to issue #13: If the option is said to have a an argument in the options section, but an adjacent arg is found but the name does not match, fail solving with a arguments-mismatch error.

Scanner is incredibly slow

The scanner code is incredibly slow, mostly because of this.

Find a way to improve the speed or fall back to regular expressions if impossible to do so.

Add support for negative numbers in adjacent position

🚧... this issue itself is still a work in progress...🚧

NOTE: Negative values are already supported for options, however currently they need to be bound using an equal sign: --foo=-1. This issue summarises the work that would need to be done to support negative values in the adjacent position, as well as for positional arguments. A proposal needs to be worked out that describes a fair subset of possible scenarios and how to handle them.


Add support for negative numbers as option arguments in adjacent position (No explicit association via '='). Further, add support for negative numbers for positionals.

Case 1 - no matching options in spec:

Usage: move <x>

There is no ambiguity here:

  • $ move -1 => {"<x>": -1}

Case 2 - matching options in spec

Usage: move -1 <x>

This could result in ambiguous parses. $ move -1 could mean:

  1. {"<x>": -1}
  2. Error: Expected <x>. Here -1 is consumed by the -1 option, leaving <x> w/o a value.

In order to achieve case 2, nothing needs to be done. This is definitively the lowest hanging fruit.
As for case 1 - this would require a substantial change to the parser in order to prioritize adjacent groups over one another, where positional arguments would always have a higher priority than options (but only options that look like negative values??? What about stacked options?). This is a can of worms, however, as I feel as there are just so many permutations to consider. Suddenly DOS-styled switches / don't seem so stupid anymore.

Case 3/a - option arguments

Usage: foo [-1 ARG]

Must handle:

  • $ foo -1 -1 => {"-1": -1}

Case 3/b - option arguments w/ repeating option

Usage: foo [-1 ARG...]

Must handle:

  • $ foo -1 -1 => {"-1": [-1]}

Case 3/c - optional option arguments w/ repeating option

Usage: foo [-1 [ARG...]]

Must handle either:

  • $ foo -1 -1 => {"-1": [-1]}
  • $ foo -1 -1 => {"-1": [true, true]}

... this is by far not exhaustive and needs more though. In saying that, I must admit I am really put off by the idea of adding this support, considering negative values are already supported for options at the very least using the explicit = or placing the value directly adjacent to an options. I could imagine, perhaps supporting a limited subset of scenarios, such as matching negative numbers as positionals and arguments, if and only if there is no option that takes the same form. This would constitute a suitable middleground and gives the help-text author enough flexibility to design around this.

Consider [-o <arg>] syntax

In git and in the POSIX utility conventions document, some options and their arguments are denoted as [-o <arg>]. Currently docopt reads this as [ FLAG POSITIONAL ]. To overcome this, one would need to write either an option section, associating <arg> with -o explicitly, or change the input slightly to: [-o=<arg>] or -o<arg>.

How about accommodating for this common usecase and associate options that are

1.) left most in a group
2.) have only one adjacent argument, which is must be a positional argument (<arg> or ARG)

This would need to work for a variety of input:

Input Canonical
[-o ARG] [-o=ARG]
[-oif ARG] [-o -i -f=ARG]
[--input FILE] [--input=FILE]
[-o [ARG]] [-o[=ARG]]
[-o (ARG)] [-o=ARG]
[-oif [ARG]] [-o -i -f[=ARG]]
[-oif (ARG)] [-o -i -f=ARG]
[--input [FILE]] [--input[=FILE]]
[--input (FILE)] [--input=FILE]
(-o ARG) (-o=ARG)
(-oif ARG) (-o -i -f=ARG)
(--input FILE) (--input=FILE)
(-o [ARG]) (-o[=ARG])
(-o (ARG)) (-o=ARG)
(-oif [ARG]) (-o -i -f[=ARG])
(-oif (ARG)) (-o -i -f=ARG)
(--input [FILE]) (--input[=FILE])
(--input (FILE)) (--input=FILE)

...plus both `FILE` *and* `<file>` variations

Improve error message for unknown options

Currently, options that were not matched against the spec yield "Unmatched option: -x", but we could check against the complete spec if option "-x" ever appears and based on that we could say "Unknown option: -x".

Support '[-i | --input]' notation

This is an extension to the smart-opts introduced in recent commits that would allow for the following notation:

Usage: foo [ -d | --debug ] [ -f | --file PATH ]

This would be equivalent to:

Usage: foo [-df PATH]
Options:
  -d, --debug
  -f, --file PATH

The biggest problem is speccing this up right...

Gameplan:

  • [-f [ARG[...]] | --file [ARG[...]]]
  • The argument names must either be omitted on one side or match
  • Repeatability will consolidated using logical or

But what about?

  • [-xvf [ARG[...]] | --file [ARG[...]]] ... I guess so?
  • [-x | -f | --file] ... no?
  • [[-f] | --file]?

Pure11 - neodoc as utility

Explore compiling neodoc using pure11 and exposing a utility surface that, given some docopt input, as well as user input, echoes the parsed values in various formats, making neodoc available anywhere outside of node. The utility surface could be documented using neodoc, which is as close to "self-hosted" as neodoc will ever be able to be :)

Ideally, the output of the pure11 compile is a tiny, cross-platform utility that can easily be wrapped in other languages and consumed from bash and other shells.

Update:

To find all bower components that have a FFI component:

ls -lA1 bower_components/purescript-*/src/**/*.js | cut -d'/' -f1-2 | uniq

Currently this list is 32 packages. Each of these will have to have a FFI implementation for pure11:

package pure11 FFI version
bower_components/purescript-aff
bower_components/purescript-arrays
bower_components/purescript-assert
bower_components/purescript-console
bower_components/purescript-datetime
bower_components/purescript-debug
bower_components/purescript-eff
bower_components/purescript-either
bower_components/purescript-exceptions
bower_components/purescript-exists
bower_components/purescript-foldable-traversable
bower_components/purescript-foreign
bower_components/purescript-functions
bower_components/purescript-generics
bower_components/purescript-globals
bower_components/purescript-integers
bower_components/purescript-lazy
bower_components/purescript-lists
bower_components/purescript-maps
bower_components/purescript-math
bower_components/purescript-maybe
bower_components/purescript-node-buffer
bower_components/purescript-node-fs
bower_components/purescript-node-path
bower_components/purescript-node-process
bower_components/purescript-node-streams
bower_components/purescript-nullable
bower_components/purescript-prelude
bower_components/purescript-spec
bower_components/purescript-st
bower_components/purescript-strings
bower_components/purescript-unsafe-coerce

De-dupe [options] upon inlining

De-dupe adjacent options after inlining [options] in order to avoid marking these options as repeated.

Usage: foo [options] --input FILE

Options:
  -i, --input FILE  Show this help text and exit

Currently, this expands to:

Usage: foo [--input FILE] --input FILE

But it should expand to:

Usage: foo --input FILE

It should do so by preferring explicit arguments over those that are expanded, hence in this case, the
user demanded the --input option be required, so this property will be inherited.

This would need to be a change to the Solver implementation, during canonicalisation.

Maybe, before expansion, [option]s are moved to the leftmost free element:

source target
<a> -h [options] -b <c> <a> [options] -h -b <c>
<a> (-h <b>) [options] -b <c> <a> (-h <b>) [options] -b <c>
((-h <b>) is not considered free due to <b> being positional).

Then, during a second pass, dedicated to expansion of [option]s, [options]s are submerged into adjacent options from left to right by recursing until the Cons (U.Reference _) _ pattern is not matched, returning a new, alternated (submerged) list.

Ambiguous fallbacks not resolved across groups

Failing test case

r"""
Usage: foo
  or: foo (--input=FILE)
Options: --input [env: INPUT]
"""

$ INPUT=bar prog
{"--input": "bar"}

Fails with: Unexpected output: - empty - (incorrectly matches first usage branch, rather than second)

This is because fallback scores are only kept for one group parse and then discarded, they must somehow be propagated until they reach a terminal group parse.

This should be a fairly easy fix in

genParser x@(D.Group optional bs repeated) canSkip = do
vs <- concat <$>
let mod = if optional then P.try >>> P.option mempty else \p -> p
parser = if repeated then goR else singleton <$> go
in mod parser
return $ score 0 vs
where
goR :: Parser (List (List ValueMapping))
goR = do
{score, result} <- unScoredResult
<$> genBranchesParser bs
false
optsFirst
-- always allow skipping for non-free groups.
(not (D.isFree x) || canSkip)
if (length (snd result) == 0)
then return $ singleton (snd result)
else do
xs <- goR <|> pure Nil
return $ (snd result) : xs
go :: Parser (List ValueMapping)
go = if length bs == 0
then return mempty
else do
snd <<< _.result <<< unScoredResult
<$> genBranchesParser bs
false
optsFirst
-- always allow skipping for
-- non-free groups.
(not (D.isFree x) || canSkip)
.

goR must somehow fold <> and go must return the full result ({ score, result }).

Usage-section syntax - "Not part of section"

Allow for NOT PART OF SECTION below as found in the official docopt tests:

Usage: prog --foo
       prog --bar
NOT PART OF SECTION

Currently, an explicit new line is required in order to be able to enforce proper alignment:

Usage: prog --foo
       prog --bar

NOT PART OF SECTION

This is because of ambiguities, where a usually valid, but badly aligned usage
line would be falsely interpreted as NOT PART OF SECTION.

Not sure what's the best approach here? Ideas:

  1. Drop enforcing proper alignment
  2. Parse badly aligned usage lines as NOT PART OF SECTION and simply sell as
    user error? I think this would be a bad developer experience.
  3. Break away from official docopt syntax?
  4. Use (2) and issue warning to developer?
  5. Take a reasonable guess. Run the meta.usage.line parser over the badly indented line and use it if the parse succeeds, or not if not.

Indicate requiredness in option descriptions

This covers the following usecase:

Usage: foo [options] [<commands> [<argv>...]]

Options
  -i, --input FILE  The file to read input from [required]

Does this make sense or should the user enforce this based on the output of
neodoc himself? Because it would be required for all possible usage branches.

Consider [--[no-]tags] flag negation support

Example:

Usage: build [-w|--watch]
  • $ ./build +w => --watch: false
  • $ ./build --no-watch => --watch: false
  • $ ./build -w +w --no-watch --watch +w => --watch: false

Changes:

  • Introduce an update to option notation: --[[no-]<name>]. Here the [no-] indicates that this
    option may be negated: --name and --no-name. This change impacts the spec parser as well
    as the arg parser. The option will be introduced as an "opt-in" feature and won't break backwards
    compat. A second option to apply this behavior to all options implicitly is added as well, which defaults to true.

To be done:

  • Implement spec lexer and parser updates to allow explicit notation in usage- and option-description-sections
  • Implement lexer and parser updates in arg parser
  • Fix #32
  • Add a IsNegated (= Boolean) field to the OptionAlias constructor
  • Collapse values during evaluation / reduction
    • For non-repeating flags, the output should be false (not omitted)
  • Use this behavior implicitly for all flags and options that take optional args (configurable)
  • Expand --[no-]foo into --foo, --no-foo (currently just does --no-foo which sort of implies --foo.

Dev Notes:

  • Probably most "correct" way would be to change the Tuple x <$> ... parser behavior to let the actual argument parser yield a different parsed arg... src/Neodoc/ArgParser/Parser.purs Another way would be to inject the negated versions into the description (at cost of perf, but most easy to reason about).

Issues to fix

  • This should be prohibited: --[no-]foo=ARG (negatable options may not take required arguments). solved by implying neg/pos
  • Remove the -/+ syntax, it's very unconventional

Add a mode that collects unknown options

Add a mode that, instead of failing a parse, collects unknown options into an array with a special key: '?'.

The usecase is passing unknown options to another program.

Consider repeating option arguments w/o repeating the option

Essentially:

usage: --foo=BAR...

could be run as:

  • $ prog --foo foo bar qux which would yield { "--foo": ["foo", "bar", "qux"] }.

currently, it needs to be run as:

  • $ prog --foo foo --foo bar --foo qux which would yield { "--foo": ["foo", "bar", "qux"] }.

What makes this non-trivial is that we don't know when to stop pulling in values.

I think we could:

  • Stop pulling in values when another option is met
  • Stop pulling in values if value falls out of options "choices" (NYI)
  • Stop pulling in values if "n" number of required args have been parsed (NYI)

purescript example

It would be nice to see a purescript usage example along side the JS one.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.