airlift/airline

Java annotation-based framework for parsing Git like command line structures

Airlift

Airlift is a framework for building REST services in Java.

This project is used as the foundation for distributed systems like Trino (formerly PrestoSQL).

Airlift pulls together stable, mature libraries from the Java ecosystem into a simple, light-weight package that lets you focus on getting things done and includes built-in support for configuration, metrics, logging, dependency injection, and much more, enabling you and your team to ship a production-quality web service in the shortest time possible.

Airlift takes the best-of-breed libraries from the Java ecosystem and glues them together based on years of experience in building high performance Java services without getting in your way and without forcing you into a large, proprietary framework.

Getting Started

• Overview
• Getting Started
  • Then see Next Steps

Reference

• Configuration
• Lifecycle/Bootstrapping
• TBD - Concurrency
• TBD - Database Pooling
• TBD - Discovery
• TBD - Events
• TBD - HTTP server
• HTTP client
• TBD - Packaging
• TBD - Logging
• TBD - Tracing
• TBD - Maven BOM
• TBD - Jackson/JSON

Recipes

• How do I ... do conditional binding based on config
• How do I ... serve static HTML files
• How do I ... package my service
• TBD - How do I do ... ?

airline's Issues

Synopsis for group help is missing positional arguments

Example:

$ git help remote
...
    git [-v] remote show [-n]

$ git help remote show
...
    git [-v] remote show [-n] [--] [<remote>]

NPE when running "help" when argument is marked as hidden

https://github.com/airlift/airline/blob/0.7/src/main/java/io/airlift/airline/GlobalUsageSummary.java#L67-L69 converts hidden options to nulls.

That list is used just below that in appendWords which throws a NPE.

Interactive command line

I can't use airline to build a personal interactive command line, can I?
Classic example of such:

php -a;
python;
mysql;

Then you type all the commands within and it works.

I know it doesn't make sense for all the applications which provide an interactive terminal, but it would be very convenient for the ones that just want to be a subset of what terminal commands are...

Edit: less than 1 minute later after writing this post, i re-read the README and i believe this is actually possible. :) (and -_- on me)
Will confirm soon.

NPE when group has no default command

builder.withGroup("foo")
    .withDefaultCommand(SomeCommand.class)    // <--- NPE if this line is missing
    .withCommands(SomeCommand.class);

Exception in thread "main" java.lang.NullPointerException
    at io.airlift.command.model.MetadataLoader.loadCommand(MetadataLoader.java:79)
    at io.airlift.command.Cli$1.apply(Cli.java:83)
    at io.airlift.command.Cli$1.apply(Cli.java:80)
    at com.google.common.collect.Iterators$8.transform(Iterators.java:860)
    at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
    at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:266)
    at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:223)
    at io.airlift.command.Cli.<init>(Cli.java:79)
    at io.airlift.command.Cli.<init>(Cli.java:40)
    at io.airlift.command.Cli$CliBuilder.build(Cli.java:243)

Show default values when showing help

It would be nice to show the default values for all options when help is displayed.

Does not compile

Just forked the project, ran mvn clean install and got the following

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
objc[85520]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
    at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
Caused by: java.lang.RuntimeException: Class java/util/UUID could not be instrumented.
    at org.jacoco.agent.rt.internal_5d10cad.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:138)
    at org.jacoco.agent.rt.internal_5d10cad.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:99)
    at org.jacoco.agent.rt.internal_5d10cad.PreMain.createRuntime(PreMain.java:51)
    at org.jacoco.agent.rt.internal_5d10cad.PreMain.premain(PreMain.java:43)
    ... 6 more
Caused by: java.lang.NoSuchFieldException: $jacocoAccess
    at java.lang.Class.getField(Class.java:1703)
    at org.jacoco.agent.rt.internal_5d10cad.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:136)
    ... 9 more
FATAL ERROR in native method: processing of -javaagent failed
/bin/sh: line 1: 85520 Abort trap: 6           /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/bin/java -javaagent:/Users/frenaud/.m2/repository/org/jacoco/org.jacoco.agent/0.6.2.201302030002/org.jacoco.agent-0.6.2.201302030002-runtime.jar=destfile=/Users/frenaud/projects/me/airline/target/jacoco.exec -jar /Users/frenaud/projects/me/airline/target/surefire/surefirebooter5947238246567954039.jar /Users/frenaud/projects/me/airline/target/surefire/surefire8523588895070006246tmp /Users/frenaud/projects/me/airline/target/surefire/surefire_04622422368411571876tmp

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.676 s
[INFO] Finished at: 2015-10-07T05:05:23-07:00
[INFO] Final Memory: 22M/383M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.14:test (default-test) on project airline: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.14:test failed: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ?
[ERROR] Command was/bin/sh -c cd /Users/frenaud/projects/me/airline && /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/bin/java -javaagent:/Users/frenaud/.m2/repository/org/jacoco/org.jacoco.agent/0.6.2.201302030002/org.jacoco.agent-0.6.2.201302030002-runtime.jar=destfile=/Users/frenaud/projects/me/airline/target/jacoco.exec -jar /Users/frenaud/projects/me/airline/target/surefire/surefirebooter5947238246567954039.jar /Users/frenaud/projects/me/airline/target/surefire/surefire8523588895070006246tmp /Users/frenaud/projects/me/airline/target/surefire/surefire_04622422368411571876tmp
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

My java:

java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Java Server Overloaded

When I run the program java file overloaded.

ParseOptionConversionException is not transporting the cause

I have the following problem: my CLI application has commands accepting an argument, which I provide a class, ClusterID having a constructor taking a String, for parsing and validating the argument.

    @Arguments(required = true, description = "Cluster ID ...")
    public ClusterID cluster;

While this is working fine for the happy case, ClusterID's constructor would raise an exception with a proper message if something has failed.

I'd like to report that message to the user, but I get an io.airlift.airline.ParseOptionConversionException only, without the cause.

io.airlift.airline.ParseOptionConversionException: cluster: can not convert "098" to a ClusterID
	at io.airlift.airline.TypeConverter.convert(TypeConverter.java:78)
	at io.airlift.airline.Parser.parseArg(Parser.java:265)
	at io.airlift.airline.Parser.parseArgs(Parser.java:255)
	at io.airlift.airline.Parser.parse(Parser.java:70)
	at io.airlift.airline.Cli.parse(Cli.java:121)
	at io.airlift.airline.Cli.parse(Cli.java:108)
	at io.airlift.airline.Cli.parse(Cli.java:103)
	at Main.main(Main.java:53)

I'd appreciate TypeConverter.java#L75 would not ignore the Throwable but pass in on io.airlift.airline.ParseOptionConversionException as cause.

Automatically show help when user types invalid command?

When the user enters a command that doesn't exist (e.g. mis-typed a command), an exception is thrown.

Exception in thread "main" io.airlift.airline.ParseArgumentsUnexpectedException: Found unexpected parameters: [blah, something]
at io.airlift.airline.Cli.validate(Cli.java:148)
at io.airlift.airline.Cli.parse(Cli.java:116)
at io.airlift.airline.Cli.parse(Cli.java:97)

Since the end-user mistyping a command is an obvious use case, I'm assuming I've done something wrong. I've set Help as the default command, but I guess that isn't it.

    Cli.CliBuilder<Runnable> builder = Cli.<Runnable>builder("muse")
        .withDescription("Muse command-line tools")
        .withDefaultCommand(Help.class)
        .withCommands(Help.class)
        .withCommands(implementors);
    Cli<Runnable> muse_parser = builder.build();
    muse_parser.parse(args).run();

My fix thus far is to catch the exception and run the Help command by re-parsing an empty command line. That seems like a hack. new Help().run() threw an exception, too, so that wasn't any better.

I feel like I'm missing an obvious solution, since none of the examples include catching an exception.

TIA!
Chris

Version 0.7 requires Java 1.7

Is there any reason that the 0.7 artifact requires java 1.7?

Upgrade to Guava 24.1+

Google Guava versions 11.0 through 24.1 are vulnerable to unbounded memory allocation in the AtomicDoubleArray class (when serialized with Java serialization) and Compound Ordering class (when serialized with GWT serialization). An attacker could exploit applications that use Guava and deserialize untrusted data to cause a denial of service. Could you upgrade guava to version 24.1 or above?

https://github.com/google/guava/wiki/CVE-2018-10237

Support command factories

Allow clients to supply their own factory to instantiate commands

Update README.md for latest 0.7 (currently says 0.6)

At https://github.com/airlift/airline/, it says the latest release is 0.6 (and that is what the code examples are for, I presume). However, on the release page and on maven central the latest release is 0.7.

The github README.md page should be updated to indicate this.

Dependency on guava

Thanks a lot for the library.
Surprisingly there aren't many java CLI libraries designed for usage for commands with parameters. IMHO this is the best one.

The only disadvantage I found is dependency on guava:

• for tiny projects it fills wrong to add 2Mb guava jar just for CLI
• "enterprise" projects will usually have dependency on guava already, but not necessarily the same version as airline

Do you think it makes sense to copy into project only those guava bits which are used?
I could do this myself, just wanted to ask your opinion.

Information about Inheritance and Composition

I see that part of the documentation is still in TBD. I probably wont need a full fledged documentation, but some hints would be very helpfull.

I have a number of options that I want to use for several (but not all) commands, and the list of relevant options is different for each command.

ParseArgumentsUnexpectedException although following usage

java -jar app-jar-with-dependencies.jar help a

NAME
        app a -

SYNOPSIS
        app [(-n <number> | --number <number>)] a

OPTIONS
        -n <number>, --number <number>

java -jar app-jar-with-dependencies.jar a --number 1

Exception in thread "main" io.airlift.airline.ParseArgumentsUnexpectedException: Found unexpected parameters: [--number, 1]
    at io.airlift.airline.Cli.validate(Cli.java:148)
    at io.airlift.airline.Cli.parse(Cli.java:116)
    at io.airlift.airline.Cli.parse(Cli.java:97)
    at my.App.main(App.java:23)```

Text book example:

public class App {

  public static void main(String[] args) {
    CliBuilder<Runnable> builder = new Cli.CliBuilder<Runnable>("app")
      .withDescription("The app cli")
      .withDefaultCommand(Help.class)
      .withCommands(Help.class, A.class);

    Cli<Runnable> parser = builder.build();
    parser.parse(args).run();
  }

  public static abstract class Abstract implements Runnable {

    @Option(name = {"-n", "--number"}, type = OptionType.GLOBAL)
    public int number;
  }

  @Command(name = "a")
  public static final class A extends Abstract {

    @Override
    public void run() {
      System.out.println(number);
    }

  }
}

--help breaks when using required options when using singleCommand()

If I have:

@Option(name = { "--sourceIndexes"}, description = "Source indexes to read from.", required = true)
public List<String> sourceIndexes;

and then do --help it tells me:

io.airlift.airline.ParseOptionMissingException: Required option '--targetIndex' is missing

... but if I remove required=true

I'll get the proper help output.

which is not fun because for our use case this breaks airline. Though I might see if I can work around this.

Improve error messages on bad command

I just ran

java_stuff server overlord

and got

Exception in thread "main" io.airlift.command.ParseArgumentsUnexpectedException: Found unexpected parameters: [overlord]
    at io.airlift.command.Cli.validate(Cli.java:148)
    at io.airlift.command.Cli.parse(Cli.java:116)
    at io.airlift.command.Cli.parse(Cli.java:97)
    at io.druid.cli.Main.main(Main.java:24)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Which is a pretty poor error message for a user to see. Something like, "server overlord is unknown, known options are XXX" would be what I would like users to see.

Add support for char type

Add char to TypeConverter.convert

This should check or a string of length 1 or a standard Java escaped character (e.g., '\t', '\n') including unicode escapes ('a' is '\u0061')

add optional example & discussion sections to the commands

Having a discussion section for a longer explanation of the functionality of the command, or related, but not required information, might be useful in some cases. Try "git help remote" for an example of this.

Further, having a section of example command invocations would be very help for many users.

Cli.commandFactory is never used?

The member variable Cli.commandFactory is never used?
Generally, user call the method Cli.parse(String ...), the CommandFactory instance that installed by Cli.CliBuilder.withCommandFactory(factory) should be used instead of create a new one.

Another optimization point is that you don't have to create a new CommandFactory object every time it's called，CommandFactory instance can be created once on the calling of Cli.CliBuilder.build().

Support named positional arguments

There should be a way to specify that exactly two arguments are required:

git remote add <name> <url>

Currently, this must be done as List<String>, which requires the user code to do the min/max checking, and doesn't allowing mixed types, such as String and URI.

Show help when missing a required parameter

When I call "command --help" when missing a required parameter specified with required=true, all I see is a stacktrace like this:

Exception in thread "main" io.airlift.command.ParseOptionMissingException:
 Required option '-rev' is missing

When specifying the help parameter the help should be displayed no matter if required parameters are missing or not.

I used the code shown on the readme page:

public static void main(String... args) {
  MyCommand command = SingleCommand.singleCommand(MyCommand.class).parse(args);

  // cancel further processing if help is displayed
  if (command.help.showHelpIfRequested()) {
    return;
  }

  command.run();
}

The exception is thrown in the parse() method naturally. Because the Help can only be triggered AFTER the command object is created, I cannot show the help if required parameters are missing. Is there an existing solution to this?

Regards,
Tom

Option.allowedValues() does not appear to be respected

The Option annotation has a allowedValues() that can be set but this does not appear to actually get honoured by the parser

The only place this is used is when OptionMetadata is populated by MetadataLoader and then the only place that OptionMetadata.getAllowedValues() is used is in the Parser where it only gets used to display an error message

Allow mutually exclusive required options

As this option in picocli, it would be nice to have a way to provide mutually exclusive options :

https://picocli.info/#_mutually_exclusive_options

I will make a PR for this one

Support sub-groups

git-flow is a nice add-on to git, which uses sub-groups or chains of groups, for example, you can do something like git flow feature start newFeatureName or git flow release finish theReleaseName

I think having these sub groups or chains of groups together makes for a natural CLI, but does not seem to be possible as Cli assumes there is only top level groups -- Cli.withGroup produces a GroupBuilder which itself does not allow another group.

Help class NPE

So far this is all I have. Will update with more details / simpler reproducable example

public static void main(String[] args) throws Exception {
Cli.CliBuilder builder = Cli.builder("hio")
.withDefaultCommand(Help.class)
.withCommands(Help.class,
InputBenchmarkCmd.class,
TailerCmd.class,
ConfOptionsCmd.class);
Cli cli = builder.build();
cli.parse(args).run();
}

nitayj@nitay-fb hive-io-3 (h3|● 6✚ 36) $ java -jar hive-io-exp-cmdline/target/hive-io-exp-cmdline-0.8-SNAPSHOT-jar-with-dependencies.jar help
Exception in thread "main" java.lang.NullPointerException
at io.airlift.command.UsagePrinter.appendWords(UsagePrinter.java:105)
at io.airlift.command.GlobalUsageSummary.usage(GlobalUsageSummary.java:73)
at io.airlift.command.GlobalUsageSummary.usage(GlobalUsageSummary.java:52)
at io.airlift.command.Help.help(Help.java:45)
at io.airlift.command.Help.help(Help.java:38)
at io.airlift.command.Help.run(Help.java:25)
at com.facebook.hiveio.cmdline.Main.main(Main.java:34)

Conflicting arguments, although they don't conflict

I have a command with two arguments:

@Arguments( title = "recordType", description = "Record type", required = true )
public String recordType;

@Arguments( title = "id", description = "Record id", required = true )
public long id;

when building a CLI for that command I get exception:

java.lang.IllegalArgumentException: Conflicting arguments definitions: ArgumentsMetadata{title='recordType', description='Record type', usage='', required=true, accessors=[DumpPropertyChain.recordType]}, ArgumentsMetadata{title='id', description='Record id', usage='', required=true, accessors=[DumpPropertyChain.id]}
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:145)
    at io.airlift.airline.model.ArgumentsMetadata.<init>(ArgumentsMetadata.java:48)
    at io.airlift.airline.model.MetadataLoader$InjectionMetadata.compact(MetadataLoader.java:257)
    at io.airlift.airline.model.MetadataLoader$InjectionMetadata.access$600(MetadataLoader.java:242)
    at io.airlift.airline.model.MetadataLoader.loadInjectionMetadata(MetadataLoader.java:112)
    at io.airlift.airline.model.MetadataLoader.loadCommand(MetadataLoader.java:86)
    at io.airlift.airline.model.MetadataLoader$1.apply(MetadataLoader.java:70)
    at io.airlift.airline.model.MetadataLoader$1.apply(MetadataLoader.java:67)
    at com.google.common.collect.Iterators$8.transform(Iterators.java:799)
    at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
    at com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:301)
    at com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.java:691)
    at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:275)
    at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:226)
    at io.airlift.airline.model.MetadataLoader.loadCommands(MetadataLoader.java:66)
    at io.airlift.airline.Cli.<init>(Cli.java:77)
    at io.airlift.airline.Cli.<init>(Cli.java:40)
    at io.airlift.airline.Cli$CliBuilder.build(Cli.java:246)
    ...

The two arguments doesn't conflict. Instead this looks to be an issue at https://github.com/airlift/airline/blob/master/src/main/java/io/airlift/airline/model/ArgumentsMetadata.java#L48 where that condition looks to be negated... i.e. it will always report conflict if there are more than one argument defined.

help to include allowedValues, and respect multi-line descriptions

Part one...

I have an option where I define allowed values, but help doesn't show what those allowed values are.

    @Option(name = { "--persist" }, allowedValues = { "disabled", "auto", "rebind", "clean" }, title = "persistance mode", description = "The persistence mode.")

Help just shows:

    --persist <persistance mode>
        The persistence mode.

Part two...

I want to include in the description what each of these allowed values means. It looks like my only option is to include that in description, so I wrote the following code:

    @Option(name = { "--persist" }, allowedValues = { "disabled", "auto", "rebind", "clean" },
            title = "persistance mode",
            description = "The persistence mode. Possible values are:\n"+
                    "disabled: will not read or persist any state; \n"+
                    "auto: will rebind to any existing state, or start up fresh if no state; \n"+
                    "rebind: will rebind to the existing state, or fail if no state available; \n"+
                    "clean: will start up fresh (not using any existing state)")

However, this discarded my new-line characters to just split the lines at 79 characters, wherever that happened to occur (presumably due to the code in https://github.com/airlift/airline/blob/master/src/main/java/io/airlift/airline/UsagePrinter.java append).

    --persist <persistance mode>
        The persistence mode. Possible values are: disabled: will not read
        or persist any state; auto: will rebind to any existing state, or
        start up fresh if no state; rebind: will rebind to any existing
        state, or fail if no state available; clean: will start up fresh
        (not using any existing state)

Note I can't work around this by padding the lines with spaces, because UsagePrinter. append has split on white space, trimming each word.

group-aware help

With

builder.withGroup("subcommand")
    ....
    .withCommand(Help.class);

running

x subcommand help

should print subcommand-specific help, just like x help subcommand does.

This would play nicely when group has .withDefaultCommand(Help.class).

Feature Request: Support for Kotlin types

at a minimum just List

Currently get compile warnings

w: Main.kt: (53, 16): This class shouldn't be used in Kotlin. Use kotlin.collections.List or kotlin.collections.MutableList instead.