wildfly-extras/creaper

Small library for JBoss AS 7 and WildFly management with a slight bias towards testing

creaper's Issues

having to always update the ManagementVersion enum is too brittle

When referring to certain management version, commands use the ManagementVersion enum which contains all known versions. This is required so that commands can differentiate between application server releases and accomodate.

The same ManagementVersion enum is, at the same time, returned from management clients when the user asks for server version. And if the server is too new, it's "unknown" and version discovery fails with an exception. Since version discovery is performed eagerly (during management client creation), this basically prevents using older Creaper with newer server versions.

This design decision was made early and never revisited. It is now clear, in the hindsight, that it was a bad decision and must be changed.

HTTP transport

It would be useful to have support for HTTP transport, in addition to the native management protocol. Some operations can't be performed over the HTTP transport, though -- notably, anything related to CLI (i.e., all invocations of executeCli and certain subset of invocations of execute, even if we were able to parse CLI strings). Meaning that some methods of OnlineManagementClient would throw UnsupportedOperationException with the HTTP transport when they would work perfectly fine with native protocol.

For that reason, I think that maybe HTTP transport could maybe be configured in a slightly annoying way. But maybe that doesn't make a lot of sense.

This also incorporates HTTPS, which should reuse the SSL configuration bits from #93.

default bootTimeout of 10 seconds is too small for managed domain

The OnlineOptions.bootTimeout timeout added as a part of #43 has a default value of 10 seconds. That is perhaps reasonable for standalone server, but definitely too small for managed domain.

Two possible solutions:

• adjust the default value so that it's suitable for both standalone server and managed domain
• keep the default value as is for standalone server and introduce a new, higher default value for managed domain

the AddModule command should workaround WFCORE-1082

The CLI operation module add generates the module.xml file automatically if it isn't provided. However, it generates the <main-class> element wrongly, using the attribute value instead of name. This is fixed since WildFly Core 2.0.0, but the AddModule command is useful way before that. So the AddModule class should workaround the problem. A simple textual replace of the wrong string would be enough.

Add AddDatasource command for MariaDB

MariaDB is new open source database based on MySQL and it would be convenient to have a command for it (as we have for MySQL as well).

The difference between MariadDB and MySQL datasource is just small. There is mainly different package namespace.

investigate providing a Creaper Launcher

If we could have a launcher that would seamlessly support both embedding WildFly and launching an external WildFly process (using WildFly Launcher), that would be super-awesome. If not, it would still be useful to investigate both:

https://github.com/wildfly/wildfly-core/tree/master/launcher
https://github.com/wildfly/wildfly-core/tree/master/embedded

It would have to be a separate module (creaper-launcher). AFAIK, it's not possible to support AS7.

add CliScript command

The existing CliFile command is able to load the CLI script from a File or a classpath resource. Would be good to have a similar command that would accept a String containing the entire CLI script. Then, CliFile could be reimplemented on top of that.

Deprecating CliFile could be considered, but it's probably not worth it.

Builders in child AddDatasources commands should return the child builder not the one from parent

Currently when I create a AddDatasource command for a database - let's say AddOracleDatasource and I decide to use some of builder methods from parent (e.g. addConnectionProperty) then such method returns by fluent api the class of AddDatasource.Builder and not class of the child builder (e.g. AddOracleDatasource.Builder).

We need to apply generics magic (I'm sorry not the magic but F-bounded type polymorphism science) to get the builder from the child in our flow api.

We will need two generic parameters - one of type of the Builder and the second one of type the result command.

remove the uberjar

This is required for #20 -- with the client libraries being <scope>provided</scope>, there's no meaningful way of building an uberjar. Besides, I'm not aware of any uberjar users, so killing it will also remove the maintenance burden (even though it's really minimal).

SSL support

The OnlineOptions class has to be extended in order to:

• support configuring SSL options
• actually using them when establishing connection to the server

unify deprecations

Currently, some elements are marked deprecated using the @Deprecated annotation, some use the @deprecated javadoc tag. This should be unified: both an annotation and the javadoc tag must be present.

Create online commands for deploying/undeploying

There should be created command, which will allow to easily create deployment operation, which can be used for both: server in standalone and in domain mode.

Administration.reload doesn't finish when server is in state "restart-required"

When the server is in state restart-required and the user (mistakenly) calls Administration.reload (or a variant thereof), the method never returns and instead times out. The cause is that this method waits until the server gets into state running, which never happens. After starting, it gets directly into state restart-required. This should be accounted for -- the reload operation did finish successfully and the server did successfully start.

add version checks to relevant offline commands

Some offline commands are version-specific and should have version checks. This wasn't possible before, but now that offline version discovery is implemented, we should add the checks retroactively.

deprecate OnlineOptions.serverType and replace with .protocol

The serverType name is slightly misleading -- the major thing it affects is the communication protocol (remoting or http-remote). When localDefault() is used, it also affects the port number, but that is kinda implied by the protocol. So a method called protocol (and a new enum called Protocol or ManagementProtocol) would be more appropriate.

Add support for batch command

It is not possible to execute batch and if-statement commands now, e.g.

batch
:whoami
run-batch

if (outcome != success) of /system-property=hello:read-resource
/system-property=hello:add(value=world)
end-if

I suggest to use method CommandContext#handle(String line). This method supports also comments. Then it will be easy to load commands from file.

can't connect to server when using old JBoss AS 7 client dependencies

When using the latest AS7 client dependencies available in Maven Central (i.e., 7.2.0.Final), Creaper fails early because it's using a variant of the ModelControllerClient.Factory.create method that was only added later. Should investigate if workaround is possible.

remove the debugging code in Administration

The Administration code contains a bit of debugging code (together with an implementation of "sneaky throw") that is no longer relevant and could/should be removed.

remove deprecated elements

For the 1.0 release, we want to remove all the deprecated features.

investigate running PMD automatically as a part of the build process

https://pmd.github.io/
https://maven.apache.org/plugins/maven-pmd-plugin/

add Address.extension

update the WildFly 10 version in the testsuite

The test suite uses an older version of WildFly 10 because the tests for patching commands don't understand patch streams. See https://issues.jboss.org/browse/WFCORE-865 for more info. The tests should be updated.

exception thrown when calling Operations.removeIfExists from command

When calling Operations.removeIfExists outside of an OnlineCommand, it returns false correctly when the desired management resource doesn't exist. However, when it's called from an OnlineCommand, it throws an exception. The reason is that inside a command, the management client is wrapped in a proxy that detects failed management operations and throws exceptions automatically, so that commands don't have to do that.

This is an important convenience that needs to be retained. However, sometimes it's simply expected that an operation will fail (not only for Operations.removeIfExists but also for Operations.exists and one could argue that this can happen for user-constructed operations as well). So there needs to be a way to communicate such intent from inside the command. If we were on Java 8, a lambda would be natural, but on Java 6, this is simply too unwieldy.

Thanks to @olukas for discovering the bug!

add Administration.shutdown

It's useful from time to time.

Also the respective variants should be added to DomainAdministration.

Security realm command should allow defining realm without truststore

Not always user wants to use both side ssl authentication and wants to define only keystore with server certificate.

support for "switching" default host/profile for managed domain

Currently, when creating Online|OfflineOptions for managed domain, one can specify a default host and/or default profile. This choice is then set in stone, which forces users to either completely avoid the convenience of transforming standalone operations to domain, or to create separate clients for each host/profile. Both options are unfriendly, the second additionally causes more overhead.

The API could be reworked a bit to address the issue. The default host/profile wouldn't be set in the Online|OfflineOptions. Instead, one would be able to create a host- and/or profile-specific Online|OfflineManagementClient from the "default" one by calling .forProfile or .forHost. These specific clients would delegate to the original management client and only perform the transformation.

If possible, this should be backwards-compatible. That's hard, though, because the default host and profile is currently publicly accessible from Online|OfflineOptions.

Thanks to @akostadinov for the basic idea of using a wrapper!

OnlineManagementClient.executeCli should support "reload"

Currently, the reload and shutdown CLI commands are not supported in Creaper, as documented in the executeCli method documentation. They can't work because the CLI implementation of them require a CLI-specific implementation of ModelControllerClient, but Creaper uses the "standard" one.

Creaper could detect them and provide an alternative implementation (using Administration), but at least in case of shutdown, it might require parsing (not sure about reload).

investigate integration with WildFly Configuration API

https://github.com/wildfly-swarm/wildfly-config-api

Maven Central synchronization

For some reason, Creaper artifacts are not synchronized to Maven Central even though they should. At least I believe they should. This must be resolved before 1.0.0.

offline variant of the patching commands

Commands for patching only have an online variant. An offline variant would be very useful too. This requires some kind of support for running "local CLI commands" in the offline mode.

empty Batch should work fine

Currently, submitting an empty composite operation via Operations fails with "WFLYCTL0155: steps may not be null". This is because the steps field is not initialized, so if no step is added, it ends up undefined.

This can be tested using new Operations(client).batch(new Batch()), but it in the real world, it happens by constructing the batch incrementally.

add OfflineManagementClient.executeCli(String)

Some local CLI operations can be run offline. For that, we'd need something like OfflineManagementClient.executeCli(String). That shouldn't be terribly hard to add.

The only thing is that I'd like to have a unified CliException for both online and offline CLI, yet the existing CliException is in the ...online package. Not sure how to deal with that. Probably just give up this goal and use a different exception, e.g. OfflineCliException.

Creaper should be able to wait when connections are being refused

When the server is really just starting and the management endpoint isn't up yet (i.e., connections will get refused), Creaper will unconditionally throw an exception. There should be a configurable timeout for this situation as well. It probably shouldn't be the existing OnlineOptions.connectionTimeout that is being passed to the ModelControllerClient.Factory, it should have a different name (but what?).

the Add[XA]DataSource commands should reload if required when replacing existing data source

The replaceExisting() method is enough to express the consent with reloading the server when required.

builder for local CLI operations

By "local CLI operations", I mean things like ls, patch apply etc. I think that the CLI calls them "commands", but Creaper uses the word "command" for something different.

The builder should handle things like escaping spaces, quotes, file paths etc. See PatchingConversions and AddModule.apply for some existing work in this area. Ideally, these two places would use the new API.

client libraries should be <scope>provided</scope>

The client libraries (*-controller-client, *-cli and also wildlfy-patching) from JBoss EAP 6.4 are currently propagated to all Creaper users (as transitive dependencies). This might have been useful a long time ago, but it's more of a hassle now. Users should provide a correct version of all the client libraries. Hence, <scope>provided</scope>.

better documentation for client library dependencies

Current documentation in the README assumes that users will always use both core and commands. That doesn't necessarily have to be true. The README should clearly document what dependencies are needed for core and what are for commands only.

Specifically, with WildFly 10, if I only want to use core and hence don't include the dependency on wildfly-patching, the CLI will fail to initialize because JBoss LogManager is missing. This wasn't necessary in previous versions, it's caused by the newly added commands for embedding a server into the CLI.

ManagementClient.online should declare more checked exceptions

This is so that all exceptions from OnlineOptions.createModelControllerClient don't have to be wrapped in IllegalStateException. It is source-incompatible, so it has to be done in a major release.

mgmt operation can be rejected due to server boot in progress also during reload/restart

The fix for #43 made sure that connecting to the server that is currently booting will wait until boot is finished. The same situation can occur during server reload/restart.

can't execute /core-services=management/access=authorization:... in managed domain

Creaper always prepends /host=... to addresses beginning with /core-service=... in managed domain. That, however, is wrong, there are legitimate uses of /core-service=... addresses in managed domain (e.g. RBAC).

Currently, if there's no default host set, attempting to execute operation on a /core-service=... address will end up with an exception. The solution of this problem will relax this check -- if there's no default host, just execute the operation without changes.

Thanks to @cypo721 for discovering the issue!

The test AddUndertowListenerOnlineTest.addHttpsConnector_commandSucceeds fails on WildFly 10.0.0.CR5

When moving to WildFly 10.0.0.CR5 (see commit Ladicek@87c25e7898), the test AddUndertowListenerOnlineTest.addHttpsConnector_commandSucceeds starts failing with this error:

09:14:31,186 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-3) MSC000001: Failed to start service jboss.undertow.listener.test-listener: org.jboss.msc.service.StartException in service jboss.undertow.listener.test-listener: Failed to start service
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: WFLYUT0079: No SSL Context available from security realm. Either the realm is not configured for SSL, or the server has not been reloaded since the SSL config was added.
    at org.wildfly.extension.undertow.HttpsListenerService.startListening(HttpsListenerService.java:117)
    at org.wildfly.extension.undertow.ListenerService.start(ListenerService.java:138)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
    ... 3 more

09:14:31,195 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 2) WFLYCTL0013: Operation ("add") failed - address: ([
    ("subsystem" => "undertow"),
    ("server" => "default-server"),
    ("https-listener" => "test-listener")
]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.undertow.listener.test-listener" => "org.jboss.msc.service.StartException in service jboss.undertow.listener.test-listener: Failed to start service
    Caused by: java.lang.IllegalStateException: WFLYUT0079: No SSL Context available from security realm. Either the realm is not configured for SSL, or the server has not been reloaded since the SSL config was added."}}

Run this command to observe the problem (using Java 8): mvn clean test -Dtest=AddUndertowListenerOnlineTest#addHttpsConnector_commandSucceeds -DfailIfNoTests=false

The purpose of this issue is investigation: is it a problem in the test, or did we find a problem in WildFly? (My guess is that it's a problem with the test, there really is no SSL configuration in ApplicationRealm, but I'm far from an expert on this.)

commands for configuring system properties

Especially in managed domain, there are multiple places where system properties can be configured, see https://docs.jboss.org/author/display/WFLY10/General+configuration+concepts#Generalconfigurationconcepts-SystemProperties

configuration backup/restore in online mode

The ConfigurationFileBackup set of offline commands facilitate taking a backup of the server configuration and later restoring it. It would be very useful to have the same possibility in the online mode, e.g. using the existing snapshot function. This would most probably require some kind of support from the server; I didn't file a feature request yet, so I'm filing this issue so that I don't forget :-)

JMX transport

It would be useful to have support for JMX transport, in addition to the native management protocol. Compared to the HTTP transport (per #94), this is from my experience much more involved (though not more complex), so this has a lower priority. I expect the limitations of the JMX transport to be similar to the limitations of the HTTP transport or worse.

add RELEASE_PROCEDURE.md

The release procedure isn't very complicated, but it should be documented anyway.

if the server is booting when OnlineManagementClient connects, it should wait until it's running

It's possible for ManagementClient.online(...) to end up with an exception like this: java.lang.AssertionError: Expected success, but operation failed: {"outcome" => "failed","failure-description" => "WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available"}

This is caused by the client trying to perform a management operation while the server is still booting. The client should detect this situation and wait until the server is running (using some reasonable timeout, e.g. 20 seconds).

remove the "sneaky throw" mechanism from the Administration class

This is a follow-up on #79. Removing the "sneaky throw" mechanism from the Administration class requires adding checked exceptions to some methods (they should have been there since the beginning, in fact...). That is source-incompatible, so it has to be done in a major release.

add a DataSources entrypoint

Similarly to the Logging or Security entrypoints, there should be a DataSources entrypoint that would provide easier access to datasource commands builders. In addition to that, a simplified variant of the datasource commands builders should be added -- these simplified builders would only accept a small subset of the possible options and would provide a way to create the "full" builder. The DataSources entrypoint would primarily create these simplified builders.

ManagementProtocol constants are not consistent with real protocol names

The enum values in ManagementProtocol are REMOTING and HTTP_REMOTE, yet the protocol names (as can be seen in the same enum) are remote and http-remoting. This is confusing. The bad names should be deprecated and replaced by the consistent ones (i.e., REMOTE and HTTP_REMOTING).

implement version discovery in the offline case

The current version discovery is completely missing for the offline case. This must be supported for the 1.0 release.