googlecloudplatform/appengine-plugins

A client Java library to manage App Engine Java applications for any project that performs App Engine Java application management. For example, the Maven, Gradle and Eclipse App Engine plugins, custom user tools, etc.

Google App Engine Plugins Core Library

IMPORTANT: This library is used by Google internal plugin development teams to share App Engine related code. Its use for any other purpose is highly discouraged and unsupported. Visit our App Engine documentation for more information on Google supported clients for App Engine administration.

Requirements

This library requires Java 1.8 or higher to run. This library requires Maven and Java 1.8 or higher to build.

You must also install the Cloud SDK command line interface (gcloud). If it isn't installed yet, follow these instructions.

You must also install the app-engine-java component:

gcloud components install app-engine-java

Supported operations

The library implements the following operations:

• Deploy an application to standard or flexible environment
• Stage an application for deployment
• Run an application on the local server synchronously or asynchronously

How to use

Build the library using the "mvn clean install" command at the repository root directory, where the pom.xml file is located. This produces the appengine-plugins-core-version-SNAPSHOT.jar file in the "target" directory that you can add to your application's class path.

To deploy a new version, a client calls the library in the following way:

// All operations require the cloud SDK
CloudSdk cloudSdk = new CloudSdk.Builder().sdkPath(some-path).javaHome(some-java-home).build();


// Operations that use gcloud command line are initiated by the Gcloud object.
Gcloud gcloud = Gcloud.builder(sdk).setMetricsEnvironment("asdf","12").setCredentialFile(some-file).build()

// Similarly for AppCfg and the DevAppServer
AppCfg appcfg = AppCfg.builder(sdk).build()
DevServers localRun = DevServers.builder(sdk).build()

// Operations are started as processes, access to these processes is handled
// by implementations of ProcessHandler. To continue to interface with processes
// as before, use LegacyProcessHandler.
ProcessHandler handler = LegacyProcessHandler.builder()...configureHandler...build();
gcloud.newDeployment(handler).deploy(myDeployConfiguration);

SDK Manager

This library provides a mechanism for installing, adding components to, and updating the Cloud SDK. The operations are intended to run asynchronously, either on an executor or through mechanisms provided by an IDE.

// Create a new Managed SDK instance
ManagedCloudSdk sdk = ManagedCloudSdk.newManagedSdk("123.123.123") // SDK fixed at version.
ManagedCloudSdk sdk = ManagedCloudSdk.newManagedSdk() // 'LATEST' sdk, can be updated.

// Implement the listener interface to listen to operation output
MessageListener listener = new MessageListener() {...};

// Always check if operations are needed before running them
if (!sdk.isInstalled()) {
  sdk.newInstaller().install(listener);
}

// use SdkComponent to reference a Cloud Sdk component
if (!sdk.hasComponent(SdkComponent.APP_ENGINE_JAVA)) {
  sdk.newComponentInstaller().installComponent(SdkComponent.APP_ENGINE_JAVA, listener);
}

// updates will only occur on 'LATEST' sdks
if (!sdk.isUpToDate) {
  sdk.newUpdater().update(listener);
}

// You can then create an SDK from a managed SDK instance
new CloudSdk.Builder().sdkPath(sdk.getSdkHome())...;

appengine-plugins's Issues

Enable license header checking

See: GoogleCloudPlatform/app-maven-plugin/issues/15

Improve Cloud SDK location detection

Use something like a which command to find the gcloud installation.

Enable Google code style checking

Configure Travis to Deploy Snapshots on commit to master branch

remove version from stage

It gets stripped out anyway by the staging command

Set quiet flag at the SDK level

Several commands accept this flag and none should run in interactive mode.

Add support for passing in arbitrary flags to actions that invoke gcloud

Ensure artifacts have the correct file permissions

After staging a flexible style project, we need to ensure the artifact is readable by everyone or the user launching it inside the docker wont be able to run it. Deploys fail to verify the vm is running and an 502 is presented when trying to hit the url.

Automatically resolve Cloud SDK path when none is provided

CloudSdk class should use the PathResolver to automatically find the Cloud SDK path when one is not provided.

Update README.md to reflect the new API

Project flag needs to be available for all commands

This parameter spans multiple commands. While it probably doesn't apply to all commands, I think it should be available to all of them in the same way that format is.

Revamp README when new design is pushed

The current version is outdated.

Should AppEngineSdk be merged with CloudSdk?

Since App Engine SDK can now be bundled with Cloud SDK, there doesn't seem to be a need for the users to provide a Cloud SDK path and an App Engine SDK path separately. That being the case, should we just merge AppEngineSdk with CloudSdk, and simply move the AppEngineSdk.runCommand() to CloudSdk?

Add tests for Args utility

Use File instead of Path everywhere

Maven and Gradle do not support automatic parameter binding to Path.

Enable FindBugs plugin and fix any violations

Update CloudSdkAppEngineModuleService to use non-deprecated gcloud commands

Several of these gcloud preview app modules _ commands have been deprecated and migrated to gcloud preview app services

Allow passing in auth context to supported gcloud operations

For consumers of this API that can't rely on having to do gcloud auth login there needs to be a way to pass in the auth context.

An initial solution could be to support the --credential-file-override flag and require consumers to do the staging of the credential file themselves.

Copy all App Engine supported files from src/main/appengine for staging

...Not just app.yaml

Handle additional configuration files

How should we handle [index.yaml] [cron.yaml] [dispatch.yaml] [datastore-indexes.xml]?

Allow clients to destroy the running process

Both for sync or async.

Need to expose ability to kill a process that was started for consumers using a custom ProcessRunner.

Enable Travis and CLA checks

@patflynn let me know what else is there in terms of CI that needs to be taken care of.

AppEngineException should be RuntimeException

Add a Builder to the CloudSdk class

This would eliminate the numerous constructors.

Use Varargs for CloudSdk.Builder.add*

Implement standard GAE staging

We're currently depending on the App Engine SDK to stage standard GAE apps. (i.e., AppCfg.main()).

We should implement staging in the library, cutting the App Engine SDK dependency and also the SdkResolver one (which makes us set a system property with the GAE SDK root folder).

Look for Dockerfile in src/main/webapp/ by default for Compat projects

Provide default configuration implementations for every action

Package com.google.cloud.tools.app.config contains interfaces for each action's configuration.

Maven and Gradle will have implementations of their own, but the library should provide a default implementation for other clients which don't have any implementation. These implementations should probably use builder syntax.

e.g., DefaultDeleteConfigurationImpl.newDeleteConfiguration(requiredParam1, requiredParam2)
.optionalParam1(p1)
.optionalParam2(p2)

Add sources jar to build

Sources jar is needed for IDEs to provide documentation and debugging support.

make sure we allow setting the environment variable to identifying the deployment source

Create file logging output listener.

Create standard log to file listener.

--retain_upload_dir is missing from stage

Check for existence of copyright notice in files

For staging with dockerfile, dockerfile must be in exploded-war directory

Fix IntelliJ codestyle settings

Support for EAR based project staging and error messaging

Currently this lib does not support staging and deployment for EAR based projects. The user is shown an error message indicating this, but the message does not provide any workaround suggestions.

If it doesn't make sense to add in this support then perhaps we should expand the error message guiding the user: i.e. deploy the modules individually instead.

Motivation: the IJ legacy App Engine plugin does support ear plugins.

Make AppEngineException checked

This indicates an unpredictable environmental condition, not a program bug. This is exactly what checked exceptions were designed for.

Ensure integration tests pass

Identified scenarios that require validation:

Deploy normal Maven app with web.xml (guestbook)
Deploy normal Maven app with vm=true in appengine-web.xml (guestbook)
Deploy pure Java8 app without appengine-web.xml (sparkjava)
Deploy JAR and WAR apps, with/without appengine-web.xml
Test on Windows
Test with no authentication in gcloud
Test with no project set in gcloud
Test without gcloud

We could also test these scenarios in bugouts and new releases.

Use appcfg directly from the sdk reference instead of using a maven dependency

see https://github.com/GoogleCloudPlatform/gradle-appengine-plugin/blob/master/src/main/groovy/com/google/appengine/task/appcfg/AppConfigTaskTemplate.groovy#L73

This way we don't have to update the maven dependency like we are in #29

Copy all files from src/main/docker for staging

Not only the Dockerfile from src/main/docker should be copied, but all the files and subdir content if any, as these files can serve as context files for the docker image build step.

windows testing

Make sure everything works on windows.

• Gcloud actually runs
• All our pathing is OS agnostic
• Stage Flex sets the correct file permissions

When staging for standard environment, do not attempt to copy Dockerfile if it doesn't exist

Detect when Dev App Server has started

Expose underlying Process to allow destroying it

Configure pom.xml for Maven Central Snapshots pushing

Publish library as an artifact in Maven Central

Merge AppEngineSdk with CloudSdk

Followup to: #20

Add --format flag support

For some clients its useful to be able to control the output format of cammand via the --format flag. For example --format=json so that the output is easily parseable.

explore freeform flag configuration

Let users pass in an argsMap for obscure/new/hidden flags in configurations.

Add Maven instructions to README