microsoft/knack

Knack - A Python command line interface framework

Knack

---

_                     _

| ___ __ __ _ ___| | __
/ __| |/ /
  <| | | | (_| | (__| <

__| __,__|_

A Command-Line Interface framework

Installation is easy via pip:

pip install knack

Knack can be installed as a non-privileged user to your home directory by adding "--user" as below:

pip install knack --user

---

Note

The project is in initial development phase. We recommend pinning to at least a specific minor version when marking knack as a dependency in your project.

---

Usage

import sys
from collections import OrderedDict

from knack import CLI, ArgumentsContext, CLICommandsLoader
from knack.commands import CommandGroup


def abc_str(length=3):
    import string
    return string.ascii_lowercase[:length]


class MyCommandsLoader(CLICommandsLoader):
    def load_command_table(self, args):
        with CommandGroup(self, 'abc', '__main__#{}') as g:
            g.command('str', 'abc_str')
        return OrderedDict(self.command_table)

    def load_arguments(self, command):
        with ArgumentsContext(self, 'abc str') as ac:
            ac.argument('length', type=int)
        super(MyCommandsLoader, self).load_arguments(command)


mycli = CLI(cli_name='mycli', commands_loader_cls=MyCommandsLoader)
exit_code = mycli.invoke(sys.argv[1:])
sys.exit(exit_code)

# $ python mycli.py abc str
# "abc"

# $ python mycli.py abc str --length 5
# "abcde"

# $ python mycli.py abc str --length 100
# "abcdefghijklmnopqrstuvwxyz"

More samples and snippets are available at examples.

Documentation

Documentation is available at docs.

Developer Setup

In a virtual environment, install the requirements.txt file.

pip install -r requirements.txt
pip install -e .

Run Automation

This project supports running automation using tox.

pip install tox
tox

Real-world uses

• Azure CLI: The Azure CLI 2.0 is Azure's new command line experience for managing Azure resources.
• VSTS CLI: A command-line interface for Visual Studio Team Services (VSTS) and Team Foundation Server (TFS). With the VSTS CLI, you can manage and work with resources including pull requests, work items, builds, and more.
• Service Fabric CLI: A command-line interface for interacting with Azure Service Fabric clusters and their related entities.

Do you use knack in your CLI as well? Open a pull request to include it here. We would love to have it in our list.

Release History

See GitHub Releases.

Contribute Code

This project has adopted the Microsoft Open Source Code of Conduct.

For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

If you would like to become an active contributor to this project, please follow the instructions provided in Contribution License Agreement.

License

Knack is licensed under MIT.

knack's Issues

[arguments.py] ValueError does not take keyword arguments

https://github.com/Microsoft/knack/blob/master/knack/arguments.py#L60

We should not call ValueError as a keyword argument.

Similar to #60

Up-level command group and argument context helpers from CLI 2.0

These features are used extensively in Azure CLI and are generic enough to be used in Knack-based projects.

Should be able to get verbosity level from CLILogging

determine_verbose_level requires args and as it is only called as part of configure, it should be an internal level method. However, whatever value is determined in configure should be saved as a property so I can access it simply using ctx.logging.verbose_level or something similar. Now I'm trying to jump through hoops to get this piece of information that should be readily available.

Conditionally add `enum34` dependency for Python 2.7

extras_require={
        ':python_version=="2.7"': ['enum34'],
    },

Knack doesn't support validators

This is a crucial feature that Azure CLI relies upon, is bizarrely not supported by argparse, and really should be part of Knack. We should include it. Currently I have to monkey patch CommandInvoker.execute to reintroduce this functionality.

...

However, while adding this feature to Knack, please, please, P-L-E-A-S-E including both of the following:

1. Argument level validators (the kind currently supported in AzureCLI)
2. Command level validators (NOT supported in AzureCLI)

If a command has a validator, then any argument level validators that would ordinarily be applied due to the argument registry aliasing, blah blah, would be ignored. A command can have at most ONE command validator or many argument level validators. The single command-level validator would allow you to sequence multiple validators, which is often necessary. The current way, of attaching a validator to a random property and making sure to strip any other ones, falls under the category of hacky workaround so we shouldn't give knack this handicap.

Help system: improve error message when type specified is wrong

If you create a help entry for type group that should be type command, you get a very ugly stack track and no indication of what went wrong. It would be better if we could catch that error and display a message that would help the author identify and correct the problem.

Add support for datetime.date in util function

See Azure/azure-cli#4671

Support an interactive mode built-in

Consider adding an interactive mode like az interactive from Azure CLI.

docs link on PyPI doesn't work

The docs link on your PyPI page doesn't work - it gives a 404 error.

Knack 0.4.5 changes behavior with --debug

In knack 0.4.4, using just the cli name would print the welcome message. Now, if the user provides name --debug or name --verbose then the welcome message will no longer be displayed. This was unintentional and the next release should revert the behavior.

CLICommandsLoader Recommended Changes

Based on ongoing conversion work in CLI 2.0, consider the below listed changes to CLICommandsLoader. The purpose of these would be to simplify the command authoring process, as the author need only focus on the public methods of the CommandLoader class.

• 1. Make "create_command" private. This should be reached by invoking "command" on a command group. This method is currently public and presents an alternate, non-favored way of creating commands. get_op_handler and user_confirmed also seem to be internal implementation details and might be better of being private.
• 2. Allow __init__ to take **kwargs and save them in the loader's instance. This is used by the CLI to accomplish various scenarios that eliminate coding redundancy (for example, settings the default resource_type in a module).
• 3. Expose public methods argument_context and command_group that wrap the import and instantiation of ArgumentContext and CommandGroup respectively. This makes it easier to instantiate these things because the author doesn't have to remember to import these classes and allows the author to simply focus on the public interface of the command loader to accomplish everything (s)he needs.
• 4. Remove "CommandSuperGroup" as it is a nebulous concept that came from a nebulous concept in CLI 2.0. Module_name can be upleveled to the CLICommandsLoader metadata, and "operations_tmpl" can be downleveled to the CommandGroup.
• 5. Add a CLICommandType class which is a reusable bundle of kwargs, analogous to CLIArgumentType. Allow CommandGroup to accept this type, overriding its values based on any additional kwargs. These group_kwargs would then serve as the baseline for any subsequent calls to group.command(...)
• 6. ...and in the spirit of improvement, maybe "CLICommandModule" would be a more intuitive class name. It's definitely how I mentally think of it.

Error message color does not have enough contrast with default console backgrounds

The vsts cli is built on knack and we have received several customer complaints that the color of red used does not provide enough contrast with the standard console backgrounds, and is therefore hard to read. Is it possible to adjust the default error message color to a brighter red that provides more contrast? It would also be good to provide a mechanism for the consuming application to choose the colors. This way we can implement a way customers can configure this in our application if they don't like the default.

Actual feedback from customer:
The error messages are printed in dark red. That's very hard to read.

testsdk folder: should be excluded from the pypi packaging

To me, it is not used in production code, then we should not include it in the package.
I came into this when looked through the files in the CLI msi installer

ensure the config directory exists in CLIConfig constructor

A follow up to Azure/azure-cli#5001

Should be able to access the command loader and similar objects.

Currently there is only a property on the CLI object for "command_loader_cls". This makes it very difficult to access methods that require a command loader (for example, registering commands!).

Documentation deficiencies

Most public facing methods and constructors lack docstrings that explains what the various kwargs do. This makes it very hard to use anything other than those described in the examples, which are very simple.

In arguments.py TypeError: AttributeError does not take keyword arguments

  File "/Users/debekoe/Repos/derekbekoe-azure-cli/env/lib/python3.6/site-packages/knack/arguments.py", line 78, in __getattr__
    raise AttributeError(message=name)
TypeError: AttributeError does not take keyword arguments

Arguments _NAMED_ARGUMENTS cannot be changed

This variable, set in CLICommandArgument, is a tuple. As such, I cannot subclass CLICommandArgument and change the value. I need to add 'id_part' to the list, but due to the immutable nature of Python tuples, I cannot do this. If this were changed to a simple list, there wouldn't be an issue.

Chinese Characters returned from command are transformed to unicode

Example:

`import sys
from collections import OrderedDict

from knack import CLI, ArgumentsContext, CLICommandsLoader
from knack.commands import CommandGroup

def abc_str(length=3):
thisdict = {
"brand": "Ford",
"model": "生活很糟糕",
"year": 1964
}
return thisdict

class MyCommandsLoader(CLICommandsLoader):
def load_command_table(self, args):
with CommandGroup(self, 'abc', 'main#{}') as g:
g.command('str', 'abc_str')
return OrderedDict(self.command_table)

def load_arguments(self, command):
    with ArgumentsContext(self, 'abc str') as ac:
        ac.argument('length', type=int)
    super(MyCommandsLoader, self).load_arguments(command)

mycli = CLI(cli_name='mycli', commands_loader_cls=MyCommandsLoader)
exit_code = mycli.invoke(sys.argv[1:])
sys.exit(exit_code)`

===============================================
Running:
python .\knackSample.py abc str
Returns:
{
"brand": "Ford",
"model": "\u751f\u6d3b\u5f88\u7cdf\u7cd5",
"year": 1964
}

===============================================

Issue is that the chinese character (生活很糟糕) turned to "\u751f\u6d3b\u5f88\u7cdf\u7cd5"

Do not include tests in the built package

Improve error related strings

Not sure where's the best place to put this, but we would like a better way to capture 400 and 500 error responses from lower libraries like urrllib3. Right now on 500 responses we see the following unhelpful message accompanied by a stack-trace:

Caused by ResponseError('too many 500 error responses',))

It would be nice if there was some callback we could register to handle these errors more gracefully

_command_handler needs to be extensible

In CLI 2.0 we expose the no_wait parameter in the _command_handler method of the create_command method in CLICommandsLoader. This method is not customizable, so I had to copy and modify the entire implementation from the base class.

Move azure.cli.core.commands.parameters.get_enum_type into Knack

Transfer from: Azure/azure-cli#7089

https://github.com/Azure/azure-cli/blob/dev/src/command_modules/azure-cli-redis/azure/cli/command_modules/redis/_params.py#L12

Support for passing in JSON as text files

Hello,

Is there currently any support for taking a file for JSON parameter inputs? If not, are there any plans for it?
It's quite a pain to escape quotes, etc, when passing in JSON inputs in command line :)

Thanks!

Allow validators to use `cmd` like CLI 2.0

Often needed when a call inside a validator would require the cli_ctx

How does this compare to click ?

There is already a Click CLI framework... how will this be better\different? Maybe MSFT can contribute to that project... https://github.com/pallets/click

I'm not involved in Click, I just use it and would rather see one thing made awesome than have two meh things. ;-)

table_transformer should accept a JMESPath string

This CLI 2.0 feature should be upleveled to Knack.

__init__ methods should type-check context

As I do the conversion work, I ran into a situation where there are two commands, one which has a context of type CLI and another CLICommandsLoader. Since I don't know what context it is truly intended to have, I don't know which is correct and which is wrong.

Especially given that "context" is such a nebulous term, this type check should really be in place to ensure the framework is used correctly.

ArgumentContext.ignore should mangle the options list for ignored arguments

If I have:

with ArgumentContext('foo') as c:
  c.argument('arg1', options_list=['--this'])

with ArgumentContext('foo bar') as c:
   c.ignore('arg1')
   c.argument('arg2', options_list=['--this'])

I will get an error because, although arg1 would be ignored, argparse sees a name conflict between arg1 and arg2. If the ignore method automatically changed the options_list to something else --__ARG1 for instance, then it won't produce the conflict and avoids me having to do this:

with ArgumentContext('foo') as c:
  c.argument('arg1', options_list=['--this'])

with ArgumentContext('foo bar') as c:
   c.argument('arg1', ignore_type, options_list=['--literally-anything-else'])
   c.argument('arg2', options_list=['--this'])

Support no group name for CommandGroup

In https://github.com/Microsoft/knack/blob/master/knack/commands.py#L301

command_name = '{} {}'.format(self.group_name, name) if self.group_name else name

Built-in support for multiple command loaders

Knack supports only a single command loader which makes it tricky to use in scenarios like Azure CLI which has several modules. Knack should work in a consistent way for single or multiple command modules.

Add missing imports to main README.md

import sys
from collections import OrderedDict
from knack import CLI, CLICommandsLoader, CLICommand, CommandSuperGroup

Migration process and pain points

I am working on converting just the Redis commands to use Knack. The process, as best I can tell, is something like this:

1. wrap the contents of commands.py in a method load_command_table. Import this method in the CommandsLoader class in the __init__.py file.
2. wrap the contents of params.py in a method load_arguments. Import this method in the CommandsLoader class in the __init__.py file.

For old style registration
3a. Add self. to each cli_command entry
3b. Add client_factory= to every command because this is now a kwarg instead of a positional arg.
3c. Add self. to each register_cli_argument entry

For new style registration
3e. Replace any uses of create_service_adapter with a simple template string expected by Knack.
3f. Add self as the 2nd parameter in each ServiceGroup entry.
3g. Swap the template parameter and the client factory parameter (now 3rd and 4th).
3h. For each s.group call, add self as the second parameter.
3i. Custom commands, because they now require a separate client factory, must be broken out into separate ServiceGroups.
3j. More to come...

4. Add cli_ctx as an argument to the signature of all client factory methods and custom methods.
5. Update all references to get_sdk or supported_api_version to use self.ctx, the client's CLI reference.

Text wrap width

Just curious, is there any plan for bubbling up the width setting for TextWrapper in _print_indent, or otherwise making it customizable? Thanks :)

Introspection.py method "extract_args_from_signature" contains dead CLI 2.0 code

https://github.com/Microsoft/knack/blob/master/knack/introspection.py#L71

This makes reference to the no_wait_param which is used by CLI 2.0 to expose --no-wait and is pretty CLI specific. Regardless, all of this code is dead because there's no good way to pass anything into this method, so the default branch is always triggered.

My recommendation would be to remove this code.

Table output does not honor OrderedDict

There's a test failure in CLI 2.0 that is testing a table formatter (ComputeListSkusScenarioTest). The formatter is a callable that produces a list of OrderedDicts. However, the resulting table output is not in the same order that the values are added to the OrderedDict.

Can't customize help sufficiently by subclassing CLIHelp

To migrate CLI 2.0, I needed to be able to customize the help output logic. However, I found that subclassing CLIHelp did not allow me to do this--it basically only lets you customize the privacy statement and welcome message. Instead I had to customize the CLIParser, override the format_help method (which I had to do anyways) and then implement my own version of show_help, a free-floating method. Essentially the encapsulation on CLIHelp should be improved.

Argparse triggers the call to format_help, but it would make sense if that simply delegated into a method within CLIHelp, so you could simply override those methods you want in your custom class. The process would be much more intuitive for command authors.

Need way to pass a CLI context into a command

In order to access a CLI instance's config file without stuffing that CLI into a global symbol that can be imported, Knack needs a way of passing the CLI context into a command invocation. Currently, to covert the az configure command, I am using the following incredibly hacky workaround:

def _app_context_factory(self, *args):
  return self.ctx

def load_command_table(self, args):
  self.cli_command(__name__, 'configure', '...#handle_configure', client_factory=_app_context_factory)

This is clearly not using the client factory kwarg for its intended purpose. I recommend we either:

1. expose a flag to pass in the CLI context, also requiring the custom command has that parameter (or supports kwargs) OR
2. Have a reserved parameter name which, if seen during introspection, will automatically fill with the application context. This is the approach I would prefer.

knack 0.5.1: global 'cmd' ignore breaks Azure CLI multi-loader implementation.

To accept knack 0.5.1, the CLI had to uplevel the implementation of CommandLoader's load_arguments method and remove the global ignore of cmd. This is done elsewhere in the CLI.

Change logic of validator execution

Currently, if a command has a command validator, none of the argument validators that may exist are executed.

My feedback from applying this to the command validators in CLI 2.0 is that I would suggest the following change (pseudocode):

# execute any argument validators first
for validator in cmd.argument_validators:
   validator(args)
# execute the command validator last
if cmd.command_validator:
   cmd.command_validator(args)

The current way, everything has to be sequenced in the command validator, if it where that validation is sequenced doesn't matter (in CLI 2.0 think tags). With the proposed change, you would validate all the things for which sequencing doesn't matter and then validate those things for which sequence does matter. It allows for more flexibility while still preserving the clarify of the command validator.

Missing telemetry or at least extensibility hooks to inject telemetry

Azure CLI core contains telemetry that could theoretically be part of this repo. As it stands, the CLI has telemetry code build into various methods which means I can't convert those parts to this package without throwing away that customization.

Shouldn't have to subclass CLIHelp to customize privacy_statement and welcome_message

As shown in the example (and what I actually had to do) to customize these messages you have to create a help class that inherits from CLIHelp and sets these parameters. It seems I should be able to pass these into CLI init, which would not require me to create this derived class.

Retrieving help text in Python

Is there currently a way to validate that all commands and arguments have documentation through as seen through -h? If not, are there suggested implementations for this validation and/or will this be potentially added in future releases?

Thanks!

ArgumentsContext register & register_alias are exactly the same

Consider removing one of them.

Eliminate redundant ways of doing the same thing

Currently, knack allows you to register commands with either of these options:

# method 1
self.cli_command(...)

# method 2
with CommandSuperGroup(...) as sg:
   with CommandGroup(...) as g:
     g.command(...)

Argument registration likewise exposes two syntactically very different ways of doing the same thing. Neither method is clearly better--they are both just different. However, this has resulted in confusion for external partners as to what the difference is and what they should use. While it doesn't take hours to tell them "they are the same and use whichever you prefer", ideally there would only be one right way to do things, and by this point we should have a better idea what that is (my guess is it would be a third implementation that combines the strengths and mitigates the weaknesses of both existing approaches).

Since this is supposed to be a framework that exposes the simplest way to do the right thing for external partners, we should strive to commit to only a single way of registering commands and arguments, especially give that no implementation can be migrated to the new framework without modification.

Add test for docstring splitting

This functionality was carried over from CLI 2.0 but not the test. It ensures that the docstring

""" This is line one. This is line two. """

Gets parsed by the introspection methods into:

short-summary: This is line one.
long-summary: This is line two.

Help file parameter entries are very strict towards the allowed entry name.

Now, if you have the options, --source and -s, you need to enter the parameter as --source -s.
NOT: --source or -s or -s --source.

We might want to consider making this more lenient.

Should be able to pass Custom CommandInvoker class to CLI

CLIHelp support command_help_file_cls

Currently, you have to override show_help for this and that shouldn't be required.

CLICommandsLoader should accept a "command_cls" argument

The create_command method inside CLICommandsLoader is what actually does the job of created CLICommand objects. The problem is, if I subclass CLICommand (which I have to do to reintroduce support for configured defaults) then I also have to subclass CLICommandsLoader and override the create_command implementation with one that is 99.99% identical and differs only by a single line.

Instead, it would be nicer if I could pass a command_cls kwarg to the init method so that I can have it instantiate my customized version without needing to duplicate the code.