justinpihony/gradle-aws-plugin-reboot

Gradle plugin to manage Amazon Web Services

gradle-aws-plugin-reboot's Issues

Add support for Gradle 7.0

Add support for Gradle 7.0 as usage of this plugin is blocked when using with Gradle 7+

Some problems were found with the configuration of task ':awsDeploy' (type 'SyncTask').
  - Type 'SyncTask' property 'acl' is missing an input or output annotation.

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.internal.execution.WorkValidationException: Some problems were found with the configuration of task ':awsDeploy' (type 'SyncTask').
  - Type 'SyncTask' property 'acl' is missing an input or output annotation.
    
    Reason: A property without annotation isn't considered during up-to-date checking.
    
    Possible solutions:
      1. Add an input or output annotation.
      2. Mark it as @Internal.
    
    Please refer to https://docs.gradle.org/7.0/userguide/validation_problems.html#missing_annotation for more details about this problem.
  - Type 'SyncTask' property 'bucketName' is missing an input or output annotation.
    
    Reason: A property without annotation isn't considered during up-to-date checking.
    
    Possible solutions:
      1. Add an input or output annotation.
      2. Mark it as @Internal.
    
    Please refer to https://docs.gradle.org/7.0/userguide/validation_problems.html#missing_annotation for more details about this problem.
  - Type 'SyncTask' property 'delete' is missing an input or output annotation.
    
    Reason: A property without annotation isn't considered during up-to-date checking.
    
    Possible solutions:
      1. Add an input or output annotation.
      2. Mark it as @Internal.
    
    Please refer to https://docs.gradle.org/7.0/userguide/validation_problems.html#missing_annotation for more details about this problem.
  - Type 'SyncTask' property 'metadataProvider' is missing an input or output annotation.
    
    Reason: A property without annotation isn't considered during up-to-date checking.
    
    Possible solutions:
      1. Add an input or output annotation.
      2. Mark it as @Internal.
    
    Please refer to https://docs.gradle.org/7.0/userguide/validation_problems.html#missing_annotation for more details about this problem.
  - Type 'SyncTask' property 'prefix' is missing an input or output annotation.
    
    Reason: A property without annotation isn't considered during up-to-date checking.
    
    Possible solutions:
      1. Add an input or output annotation.
      2. Mark it as @Internal.

Upgrade to latest version of Gradle

This project is built using Gradle 4.6 from Feb 28, 2018. It should be upgraded to Gradle 6.7.1+ to make use of the latest Gradle features.

So far I've spotted that the findbugs plugin no longer exists in newer Gradle versions.

Are there any other known issues?

java11

any chance you can rebuild with latest aws sdk so we have java11 support?

thanks

Add support for getting secrets from AWS secret manager

Would be nice to have api that would allow us to use secrets in our build

Mention breaking change in AWSLambdaMigrateFunctionTask

there is a breaking change when timeout has changed to lambdaTimeout without any warning notice

New Gradle settings need optional in many places

Now that it has the Input annotation it also requires an Optional annotation for many of the fields - as they were not required previously. Most are handled in the code itself...so maybe it is better to blanket them?

Upgrade to Gradle 7.3

This project is built using Gradle 7.1.1 . I have a Java 17 project (Which is the new LTS) which is supported in Gradle 7.3+.

Is there any blockers to prevent this from happening?

ResourceConflictException

I believe that this change is causing issues for this library:

https://aws.amazon.com/de/blogs/compute/coming-soon-expansion-of-aws-lambda-states-to-all-functions/

in the past few days, i have started getting:

The operation cannot be performed at this time. An update is in progress for resource: arn:aws:lambda:eu-west-2:700816333857:function:scheduleBroadcasts (Service: AWSLambda; Status Code: 409; Error Code: ResourceConflictException; Request ID: 58c23e92-d13b-4d60-8a65-728ab960446d; Proxy: null)

If I place "aws:states:opt-out" in my lambda description field, I can deploy again, but it looks like this opt-out will only last until dec 6th.

Is anyone else having similar issues?

thanks

tim

Add support for setting ECR permissions

Hello,

At the moment, there is a task for creating ECR repositories however there isn't a task for setting up the permissions.
The AWS SDK call would be https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/ecr/AmazonECRClient.html#setRepositoryPolicy-com.amazonaws.services.ecr.model.SetRepositoryPolicyRequest-

S3 insists I'm trying to use us-east-1

I have everything set to us-west-2 in my build.gradle file, but I keep getting the same error:

root@47874c042e0a:~/project/build_scripts# ./gradlew :infrastructure:disasterRecovery:createCFBucket

> Configure project :
--------------------------------------------------------------------------
  Current Environment: dev-staging-circleci
  Debug Mode: false, Time: 17:54:23
--------------------------------------------------------------------------

> Configure project :monitoring
/root/project/build/main/cftemplates/monitoring/generate_templates.js
Download https://plugins.gradle.org/m2/jp/classmethod/aws/reboot/gradle-aws-plugin-reboot/maven-metadata.xml
Download https://plugins.gradle.org/m2/jp/classmethod/aws/reboot/gradle-aws-plugin-reboot/0.43/gradle-aws-plugin-reboot-0.43.pom
Download https://plugins.gradle.org/m2/jp/classmethod/aws/reboot/gradle-aws-plugin-reboot/0.43/gradle-aws-plugin-reboot-0.43.jar

> Configure project :infrastructure:disasterRecovery
Creating bucket: us-west-2-main-dev-staging-circleci-dr-deploymentbucket

> Task :infrastructure:disasterRecovery:createCFBucket FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':infrastructure:disasterRecovery:createCFBucket'.
> The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-2' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: 800B103F796C126B; S3 Extended Request ID: VHow9cfoh1frrxgs5xAjOIDTMVOPV7rHPcGVj3ZdRLM5V1Vy+4DAisXq+D+0dwHDj1xbabwG04g=)

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s
1 actionable task: 1 executed
root@47874c042e0a:~/project/build_scripts#

Here's what my build.gradle file looks like:

...
import jp.classmethod.aws.gradle.s3.CreateBucketTask

...

aws {
  profileName = "default"
  region = "$dr_aws_region"
}

...

task createCFBucket(type: CreateBucketTask) {
  if(aws_environment == "staging" || aws_environment == "prod" || aws_environment == "dr") {
    println "Creating bucket: $dr_aws_region-$aws_vpc-$deploy_env-dr-deploymentbucket"

    group 'AWS'
    bucketName "$dr_aws_region-$aws_vpc-$deploy_env-dr-deploymentbucket"
    region "$dr_aws_region"
    ifNotExists true
  }
}

It prints out the println Creating bucket: us-west-2-main-dev-staging-circleci-dr-deploymentbucket, so I know that's fine. I tried replacing the variables with the plain strings to see if there was some weird variable substitution issue, but I got the exact same error. I'm clueless.

Maybe this is related? aws/aws-sdk-java#1451

Support configuration-cache for S3Sync

This plugin doesn't support configuration cache. There are two problems for S3Sync:

1. Easy to fix by injecting ObjectFactory and using ObjectFactory.fileTree().from(dir)
   

   gradle-aws-plugin-reboot/src/main/java/jp/classmethod/aws/reboot/gradle/s3/SyncTask.java

   Lines 131 to 138 in 072eedf

   	getProject().fileTree(source).visit(new EmptyFileVisitor() {
   	public void visitFile(FileVisitDetails element) {
   	es.execute(
   	new UploadTask(s3, element, bucketName, prefix, storageClass, acl, metadataProvider,
   	getLogger()));
   	}
   	});

2. Harder to fix
   

   gradle-aws-plugin-reboot/src/main/java/jp/classmethod/aws/reboot/gradle/s3/SyncTask.java

   Lines 112 to 113 in 072eedf

   	AmazonS3PluginExtension ext = getProject().getExtensions().getByType(AmazonS3PluginExtension.class);
   	AmazonS3 s3 = ext.getClient();

Not a big deal, very easy to just add --no-configuration-cache when doing an upload.

EDIT: P.S. Thanks for resurrecting this!

S3 sync gives stack trace but with success

import jp.classmethod.aws.reboot.gradle.s3.SyncTask

plugins {
    id("jp.classmethod.aws.reboot.s3") version "0.42"
}

tasks {

    register<SyncTask>("uploadCfts"){
        source = file("${projectDir}/infra/cfts")
        bucketName = "build-artifacts-us-east-1"
        prefix = project.name

    }
}

$ ls $PWD/build.gradle.kts
/Users/cbongiorno/dev/nexus/build.gradle.kts

$ ls $PWD/infra/cfts
00-nexus.cft            01-nexus-infra.cft      02-nexus-codedeploy.cft 03-nexus-policies.cft   04-nexus-sg.cft         05-nexus-ec2.cft        06-nexus-network.cft    07-ssl-cert.cft

Exception in thread "pool-3-thread-6" com.amazonaws.services.s3.model.AmazonS3Exception: The provided token is malformed or otherwise invalid. (Service: Amazon S3; Status Code: 400; Error Code: InvalidToken; Request ID: EC661917404D5E37; S3 Extended Request ID: bNS7SgKv9RA+g6r5Q34F72MxUy6oq0DcOf/mXvvPREReJRCB/VoUJC0lg22QtYojvYXzOmVewSE=), S3 Extended Request ID: bNS7SgKv9RA+g6r5Q34F72MxUy6oq0DcOf/mXvvPREReJRCB/VoUJC0lg22QtYojvYXzOmVewSE=

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.1.1/userguide/command_line_interface.html#sec:command_line_warnings
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)

        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
BUILD SUCCESSFUL in 3s
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
1 actionable task: 1 executed
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4914)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4860)
        at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:389)
        at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:5793)
        at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1786)
        at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1746)
        at jp.classmethod.aws.reboot.gradle.s3.SyncTask$UploadTask.run(SyncTask.java:216)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

For the record, the files aren't actually there.