Giter Site home page Giter Site logo

wata727 / packer-plugin-amazon-ami-management Goto Github PK

View Code? Open in Web Editor NEW
99.0 11.0 26.0 1.6 MB

Packer post-processor plugin for Amazon AMI management

License: Mozilla Public License 2.0

Go 97.01% Makefile 2.18% Shell 0.81%
packer aws post-processor ami-management

packer-plugin-amazon-ami-management's Introduction

packer-plugin-amazon-ami-management

Build Status GitHub release License: MPL 2.0

Packer post-processor plugin for Amazon AMI management

Description

This post-processor cleanups old AMIs and EBS snapshots after baking a new AMI.

Installation

Packer >= v1.7.0 supports third-party plugin installation by init command. You can install the plugin automatically after adding the required_plugin block.

packer {
  required_plugins {
    amazon-ami-management = {
      version = ">= 1.0.0"
      source = "github.com/wata727/amazon-ami-management"
    }
  }
}

See the Packer documentation for more details.

Usage

The following examples is a templates to keep only the latest 3 AMIs.

An example with defined option identifier

source "amazon-ebs" "example" {
  region        = "us-east-1"
  source_ami    = "ami-6869aa05"
  instance_type = "t2.micro"
  ssh_username  = "ec2-user"
  ssh_pty       = true
  ami_name      = "packer-example ${formatdate("YYYYMMDDhhmmss", timestamp())}"
  tags = {
    Amazon_AMI_Management_Identifier = "packer-example"
  }
}

build {
  sources = ["source.amazon-ebs.example"]

  provisioner "shell" {
    inline = ["echo 'running...'"]
  }

  post-processor "amazon-ami-management" {
    regions       = ["us-east-1"]
    identifier    = "packer-example"
    keep_releases = 3
  }
}

An example with defined option tags

locals {
  tags = {
    version    = 1.23
    department = "dev"
  }
}

source "amazon-ebs" "example" {
  region        = "us-east-1"
  source_ami    = "ami-6869aa05"
  instance_type = "t2.micro"
  ssh_username  = "ec2-user"
  ssh_pty       = true
  ami_name      = "packer-example ${formatdate("YYYYMMDDhhmmss", timestamp())}"
  tags          = local.tags
}

build {
  sources = ["source.amazon-ebs.example"]

  provisioner "shell" {
    inline = ["echo 'running...'"]
  }

  post-processor "amazon-ami-management" {
    regions       = ["us-east-1"]
    keep_releases = 3
    tags          = local.tags
  }
}

Configuration

Type: amazon-ami-management

Required:

  • identifier (string) - An identifier of AMIs. This plugin looks Amazon_AMI_Management_Identifier tag. If identifier matches tag value, these AMI becomes to management target.
  • tags (map of strings) - The tags to indetify AMI. This plugin uses search by tags. If tags matches AMI tags, these AMI becomes to management target. If identifier is set, this parameter is ignored.
  • keep_releases (integer) - The number of AMIs. This value is invalid when keep_days is set.
  • keep_days (integer) - The number of days to keep AMIs. For example, if you specify 10, AMIs created before 10 days will be deleted. This value is invalid when keep_releases is set.
  • regions (array of strings) - A list of regions, such as us-east-1 in which to manage AMIs. NOTE: Before v0.3.0, this parameter was region. Since 0.4.0, region is not used.

Optional:

  • dry_run (boolean) - If true, the post-processor doesn't actually delete AMIs.

The following attibutes are also available. These are optional and used in the same way as AWS Builder:

  • access_key
  • secret_key
  • assume_role
  • custom_endpoint_ec2
  • mfa_code
  • profile
  • skip_metadata_api_check
  • token

IAM Task or Instance Role

The post-processor requires additional permissions to work. Below is the difference from the minimum permissions required by Packer.

{
  "Version": "2012-10-17",
  "Statement": [{
      "Effect": "Allow",
      "Action" : [
+       "autoscaling:DescribeLaunchConfigurations",
        "ec2:AttachVolume",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:CopyImage",
        "ec2:CreateImage",
        "ec2:CreateKeypair",
        "ec2:CreateSecurityGroup",
        "ec2:CreateSnapshot",
        "ec2:CreateTags",
        "ec2:CreateVolume",
        "ec2:DeleteKeyPair",
        "ec2:DeleteSecurityGroup",
        "ec2:DeleteSnapshot",
        "ec2:DeleteVolume",
        "ec2:DeregisterImage",
        "ec2:DescribeImageAttribute",
        "ec2:DescribeImages",
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceStatus",
+       "ec2:DescribeLaunchTemplates",
+       "ec2:DescribeLaunchTemplateVersions",
        "ec2:DescribeRegions",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSnapshots",
        "ec2:DescribeSubnets",
        "ec2:DescribeTags",
        "ec2:DescribeVolumes",
        "ec2:DetachVolume",
        "ec2:GetPasswordData",
        "ec2:ModifyImageAttribute",
        "ec2:ModifyInstanceAttribute",
        "ec2:ModifySnapshotAttribute",
        "ec2:RegisterImage",
        "ec2:RunInstances",
        "ec2:StopInstances",
        "ec2:TerminateInstances"
      ],
      "Resource" : "*"
  }]
}

Developing Plugin

If you wish to build this plugin on your environment, you can use GNU Make build system. But this Makefile depends on Go 1.22 or more. At First, you should install Go.

$ make build

packer-plugin-amazon-ami-management's People

Contributors

dependabot-preview[bot] avatar dependabot[bot] avatar fernandoalex avatar fossabot avatar nitrocode avatar panthaaaa avatar stobias123 avatar sweetops avatar tchia04 avatar wata727 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

packer-plugin-amazon-ami-management's Issues

Plugin is not compatible anymore with packer >= 1.7.0

Packer 1.7.0 was release two days ago, and the plugin is not compatible anymore.

error initializing post-processor 'amazon-ami-management': The protocol of this
plugin (protocol version 4 and lower) was deprecated, please use a newer version
of this plugin.Or use an older version of Packer (pre 1.7) with this plugin.

Thanks

Can't get it to work in CodeBuild - any suggestions

Implementing a packer AMI pipeline in CodePipeline/CodeBuild and I am trying to implement your ami management plugin (don't want to introduce yet another lambda)
Packer 1.3.5
plugin: latest (0.6.1)
Installing via your script on Ubuntu 14 based build.
Installation works but it seems that somehow packer spaws the process in a way that doesn't work

Failed to initialize build 'amazon-ebs': error initializing post-processor 'amazon-ami-management': fork/exec /root/.packer.d/plugins/packer-post-processor-amazon-ami-management: no such file or directory 

Do you have any ideas where the problem might be? Tried chaining the command differently with no success. Local tests on Ubuntu/Mac worked perfectly.
Thanks!

Compatibility packer 1.10

Hi !

It's me again.

Packer 1.10 break this plugin with this error (it is working fine with 1.9.5):

Error: Unknown data type amazon-ami
  on gitlab-runner.pkr.hcl line 36:
  (source code not available)
known data sources: [http null hcp-packer-image hcp-packer-iteration]
Error: Unknown source type amazon-ebs
  on gitlab-runner.pkr.hcl line 92:
  (source code not available)
The source amazon-ebs is unknown by Packer, and is likely part of a plugin that
is not installed.
You may find the needed plugin along with installation instructions documented
on the Packer integrations page.
https://developer.hashicorp.com/packer/integrations?filter=amazon
Cleaning up project directory and file based variables

Thank you for your work !

Can't pull credentials when running in a container

Running in an ECS container, the Task Role and Instance Role should be able to be automatically assumed by this plugin (they work for example with the amazon-ebs builder).

However, relying on this yields a NoCredentialsError from the aws sdk lib.

Can't assume role in other account

I am using Packer 1.7.2 to build AMI in OU account by using assume_role but I can't use this plugin to manage AMIs in that account.

Does this plugin support assume_role block like packer?

unexpected EOF (again)

Using packer v1.5.6

==> Some builds didn't complete successfully and had errors:
--> ami: 1 error(s) occurred:

* Post-processor failed: unexpected EOF

==> Builds finished but no artifacts were created.
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: panic: runtime error: invalid memory address or nil pointer dereference
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1d397a2]
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin:
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: goroutine 117 [running]:
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: main.(*Cleaner).DeleteImage(0xc0000b46c0, 0xc0001ea600, 0x0, 0x1)
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: 	/home/runner/work/packer-post-processor-amazon-ami-management/packer-post-processor-amazon-ami-management/cleaner.go:140 +0x212
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: main.(*PostProcessor).PostProcess(0xc00031c800, 0x24bab20, 0xc000478b80, 0x24c1300, 0xc0004ac7e0, 0x24c1360, 0xc0004a4ae0, 0x1, 0x203000, 0x28, ...)
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: 	/home/runner/work/packer-post-processor-amazon-ami-management/packer-post-processor-amazon-ami-management/post-processor.go:100 +0x34e
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: github.com/hashicorp/packer/packer/rpc.(*PostProcessorServer).PostProcess(0xc0001e37c0, 0x1, 0xc0004d2010, 0x0, 0x0)
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: 	/home/runner/go/pkg/mod/github.com/hashicorp/[email protected]/packer/rpc/post_processor.go:109 +0x277
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: reflect.Value.call(0xc000045da0, 0xc00000eb60, 0x13, 0x21ccdc1, 0x4, 0xc0004c4f18, 0x3, 0x3, 0x0, 0x2f9ce00, ...)
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: 	/opt/hostedtoolcache/go/1.13.8/x64/src/reflect/value.go:460 +0x5f6
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: reflect.Value.Call(0xc000045da0, 0xc00000eb60, 0x13, 0xc00047e718, 0x3, 0x3, 0xc0004cc000, 0xc0004be030, 0xc)
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: 	/opt/hostedtoolcache/go/1.13.8/x64/src/reflect/value.go:321 +0xb4
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: net/rpc.(*service).call(0xc0001e3800, 0xc0000ab400, 0xc00036e9a0, 0xc00036e9b0, 0xc0000fca80, 0xc00031f9a0, 0x1e5bfe0, 0xc0004be050, 0x18a, 0x1e091c0, ...)
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: 	/opt/hostedtoolcache/go/1.13.8/x64/src/net/rpc/server.go:377 +0x16f
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: created by net/rpc.(*Server).ServeCodec
2020/06/16 06:48:47 packer-post-processor-amazon-ami-management plugin: 	/opt/hostedtoolcache/go/1.13.8/x64/src/net/rpc/server.go:474 +0x42b
2020/06/16 06:48:47 [INFO] (telemetry) ending amazon-ami-management
2020/06/16 06:48:47 [INFO] (telemetry) Starting post-processor manifest
2020/06/16 06:48:47 /Users/lbriggs/.packer.d/plugins/packer-post-processor-amazon-ami-management: plugin process exited
2020/06/16 06:48:47 [INFO] (telemetry) ending manifest
2020/06/16 06:48:47 Flagging to keep original artifact from post-processor 'manifest'

* Post-processor failed: unexpected EOF
2020/06/16 06:48:47 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2020/06/16 06:48:47 machine readable: ami,error []string{"1 error(s) occurred:\n\n* Post-processor failed: unexpected EOF"}

* Post-processor failed: unexpected EOF
==> Builds finished but no artifacts were created.
2020/06/16 06:48:47 [INFO] (telemetry) Finalizing.
2020/06/16 06:48:47 waiting for all plugin processes to complete...
2020/06/16 06:48:47 /usr/local/bin/packer: plugin process exited
2020/06/16 06:48:47 /usr/local/bin/packer: plugin process exited
2020/06/16 06:48:47 /usr/local/bin/packer: plugin process exited
2020/06/16 06:48:47 /usr/local/bin/packer: plugin process exited
make: *** [build] Error 1

Build fails with 'Could not find AWS region, make sure it's set'

I get a Could not find AWS region, make sure it's set. Not sure what's going on. The account I'm using for this has full access to everything with regards to AWS. I find it odd that it prints out amazon-ebs (amazon-ami-management): Processing in us-east-2 which looks good to me.

Here's my packer.json

{
  "variables": {
    "aws_access_key": "OMITTED",
    "aws_secret_key": "OMITTED",
    "build-region": "us-east-2",
    "packer-built-this-tag": "packer-built"
  },
  "sensitive-variables": [
    "aws_secret_key"
  ],
  "builders": [
    {
      "type": "amazon-ebs",
      "access_key": "{{user `aws_access_key`}}",
      "secret_key": "{{user `aws_secret_key`}}",
      "region": "us-east-2",
      "source_ami_filter": {
        "filters": {
          "virtualization-type": "hvm",
          "name": "ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*",
          "root-device-type": "ebs"
        },
        "owners": [
          "099720109477"
        ],
        "most_recent": true
      },
      "subnet_filter": {
        "filters": {
          "tag:purpose": "packer-builder"
        },
        "random": true
      },
      "tags": {
        "Amazon_AMI_Management_Identifier": "{{user `packer-built-this-tag`}}"
      },
      "instance_type": "t2.micro",
      "ssh_username": "ubuntu",
      "ami_name": "ubuntu 18.04 {{timestamp}}"
    }
  ],
  "post-processors": [
    {
      "type": "amazon-ami-management",
      "regions": [
        "us-east-2"
      ],
      "identifier": "packer-built",
      "keep_releases": "3"
    }
  ]
}

Here's the console out

amazon-ebs output will be in this color.

==> amazon-ebs: Prevalidating AMI Name: ubuntu 18.04 1565815383
    amazon-ebs: Found Image ID: ami-05c1fa8df71875112
    amazon-ebs: Found Subnet ID: subnet-0b0553751a39dbc5e
==> amazon-ebs: Creating temporary keypair: packer_5d547257-0bf6-67fc-46d0-e0fa8b6aa50c
==> amazon-ebs: Creating temporary security group for this instance: packer_5d54725a-2c86-fd5a-4491-31d3f145662d
==> amazon-ebs: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
    amazon-ebs: Adding tag: "Name": "Packer Builder"
    amazon-ebs: Instance ID: i-00861a33446c7aa96
==> amazon-ebs: Waiting for instance (i-00861a33446c7aa96) to become ready...
==> amazon-ebs: Using ssh communicator to connect: 13.58.56.59
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Stopping the source instance...
    amazon-ebs: Stopping instance
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating AMI ubuntu 18.04 1565815383 from instance i-00861a33446c7aa96
    amazon-ebs: AMI: ami-012b028a18497998a
==> amazon-ebs: Waiting for AMI to become ready...
==> amazon-ebs: Adding tags to AMI (ami-012b028a18497998a)...
==> amazon-ebs: Tagging snapshot: snap-0ac2d1f217342e95e
==> amazon-ebs: Creating AMI tags
    amazon-ebs: Adding tag: "Amazon_AMI_Management_Identifier": "packer-built"
==> amazon-ebs: Creating snapshot tags
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary security group...
==> amazon-ebs: Deleting temporary keypair...
==> amazon-ebs: Running post-processor: amazon-ami-management
    amazon-ebs (amazon-ami-management): Processing in us-east-2
Build 'amazon-ebs' errored: 1 error(s) occurred:

* Post-processor failed: Could not find AWS region, make sure it's set.

Permission denied with packer 1.5.0

Our CI-Runner updated to packer version 1.5.0 and the next packer build returned this error:

+ packer build dockerhost.json
Failed to initialize build 'amazon-ebs': error initializing post-processor 'amazon-ami-management': fork/exec /opt/packer: permission denied

We use this way to install packer and the post processor.

# Packer
RUN PACKER_VERSION="$(curl -fsSL https://releases.hashicorp.com/packer/ | grep -oPm 1 'packer_\K(\d\d?\.\d\d?\.\d\d?)(?=</a>$)')" && \
    wget -q -O /tmp/packer.zip https://releases.hashicorp.com/packer/${PACKER_VERSION}/packer_${PACKER_VERSION}_linux_amd64.zip && \
    unzip -q /tmp/packer.zip -d /opt/packer && \
    rm -rf /tmp/packer.zip
ENV PATH="/opt/packer:${PATH}"

# Packer AMI Management Post Processor
RUN PACKER_POST_PROCESSOR_AMAZON_AMI_MANAGEMENT_URL="$(curl -fsSL "https://api.github.com/repos/wata727/packer-post-processor-amazon-ami-management/releases/latest" | grep -o https://github.com.*linux_amd64.zip)" && \
    wget -q -O /tmp/packer_ami_post_processor_plugin.zip ${PACKER_POST_PROCESSOR_AMAZON_AMI_MANAGEMENT_URL} && \
    unzip -q /tmp/packer_ami_post_processor_plugin.zip -d /opt/packer && \
    rm -rf /tmp/packer_ami_post_processor_plugin.zip

Reverting back to packer 1.4.5 fixed the issue.

Multiple regions support

Hi,

Is it possible to have the post processor utilize {{user destination_regions}}, for example:

"post-processors":[{
"type": "amazon-ami-management",
"region": "{{user destination_regions}}",
"identifier": "misp-build",
"keep_releases": "1"
}
]

Since I don't specify regions in environment variables, I'd still like to avoid specifying regions in two locations of the packer template.

Split comma-separated value for `ami_regions`

Currently, Packer doesn't allow you to use JSON arrays inside of configuration files. This means that if I want to store ami_regions in a configuration file and pass it to Packer, I have to store it as, e.g.

{
  "ami_regions": "us-east-1,us-west-2"
}

The packer-post-processor-amazon-ami-management plugin does not interpret a comma-separated value as containing multiple regions. It would be great if it did!

wrong download url in readme

installation instructions in readme include the line:

wget https://<snip>/download/v0.6.2/packer-post-processor-amazon-ami-management_0.7.0_darwin_amd64.zip -P /tmp/

which has both 0.6.2 and 0.7.0, making it 404. Should probably be:

wget https://<snip>/download/v0.7.0/packer-post-processor-amazon-ami-management_0.7.0_darwin_amd64.zip -P /tmp/

panic with v1.4.0: type cty.Type has no exported fields

I'm not sure if this is a bug with packer or this plugin.

I've tried it with latest packer, also version 1.8.6. Both work with version 1.3.1 of this plugin.

Here's the relevant output with the stack trace:

# packer version
Packer v1.9.1

# packer init .
Installed plugin github.com/wata727/amazon-ami-management v1.4.0 in "/home/ubuntu/.config/packer/plugins/github.com/wata727/amazon-ami-management/packer-plugin-amazon-ami-management_v1.4.0_x5.0_linux_amd64"

# packer build .
panic: ConfigSpec failed: gob: type cty.Type has no exported fields [recovered]
	panic: ConfigSpec failed: gob: type cty.Type has no exported fields

goroutine 1 [running]:
log.Panic({0xc000a1e2e0?, 0x0?, 0xc000a1e340?})
	log/log.go:384 +0x65
github.com/hashicorp/packer/packer.(*cmdPostProcessor).checkExit(0x416250?, {0x295cdc0, 0xc000c8cdd0}, 0x0)
	github.com/hashicorp/packer/packer/cmd_post_processor.go:50 +0x7f
github.com/hashicorp/packer/packer.(*cmdPostProcessor).ConfigSpec.func1()
	github.com/hashicorp/packer/packer/cmd_post_processor.go:22 +0x39
panic({0x295cdc0, 0xc000c8cdd0})
	runtime/panic.go:884 +0x[21](https://github.com/ingatlancom/packer/actions/runs/5375889656/jobs/9752342914#step:5:22)3
github.com/hashicorp/packer-plugin-sdk/rpc.(*commonClient).ConfigSpec(0xc000c8af00)
	github.com/hashicorp/[email protected]/rpc/common.go:47 +0x297
github.com/hashicorp/packer/packer.(*cmdPostProcessor).ConfigSpec(0xc000a1e718?)
	github.com/hashicorp/packer/packer/cmd_post_processor.go:[25](https://github.com/ingatlancom/packer/actions/runs/5375889656/jobs/9752342914#step:5:26) +0x65
github.com/hashicorp/packer/hcl2template.decodeHCL2Spec({0x38e3ba0, 0xc000649530}, 0x0?, {0x7ffa3dddcb98?, 0xc000c7f8c0?})
	github.com/hashicorp/packer/hcl2template/decode.go:20 +0x39
github.com/hashicorp/packer/hcl2template.(*HCL2PostProcessor).HCL2Prepare(0xc000c94a80, 0xc000c94a50?)
	github.com/hashicorp/packer/hcl2template/types.hcl_post-processor.go:53 +0x41c
github.com/hashicorp/packer/hcl2template.(*PackerConfig).startPostProcessor(0xc0000e43c0, {{{0xc000632e07, 0xa}, {0xc000632e12, 0xc}}, {0x0, 0x0}, {0x38e3ba0, 0xc000648000}}, 0xc000564900, ...)
	github.com/hashicorp/packer/hcl2template/types.build.post-processor.go:86 +0x5f8
github.com/hashicorp/packer/hcl2template.(*PackerConfig).getCoreBuildPostProcessors(0xc0000e43c0, {{{0xc000632e07, 0xa}, {0xc000632e12, 0xc}}, {0x0, 0x0}, {0x38e3ba0, 0xc000648000}}, {0xc0006498f0, ...}, ...)
	github.com/hashicorp/packer/hcl2template/types.packer_config.go:556 +0x3f2
github.com/hashicorp/packer/hcl2template.(*PackerConfig).GetBuilds(0xc0000e43c0, {{0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, 0x0, 0x0, {0x0, ...}, ...})
	github.com/hashicorp/packer/hcl2template/types.packer_config.go:695 +0x17f7
github.com/hashicorp/packer/command.(*BuildCommand).RunContext(0xc0001657d0, {0x38e34a0?, 0xc00015c050}, 0xc00014a090)
	github.com/hashicorp/packer/command/build.go:115 +0x2bd
github.com/hashicorp/packer/command.(*BuildCommand).Run(0xc0001657d0, {0xc0000501a0, 0x1, 0x1})
	github.com/hashicorp/packer/command/build.go:41 +0xc5
github.com/mitchellh/cli.(*CLI).Run(0xc0000e4140)
	github.com/mitchellh/[email protected]/cli.go:[26](https://github.com/ingatlancom/packer/actions/runs/5375889656/jobs/9752342914#step:5:27)2 +0x5f8
main.wrappedMain()
	github.com/hashicorp/packer/main.go:265 +0xcc5
main.realMain()
	github.com/hashicorp/packer/main.go:52 +0xf5
main.main()
	github.com/hashicorp/packer/main.go:[38](https://github.com/ingatlancom/packer/actions/runs/5375889656/jobs/9752342914#step:5:39) +0x19

Remove only unused AMIs

Yesterday my team brought up another interesting issue where the packer post processor could potentially remove an AMI that an Auto Scaling Group's Launch Configuration is currently using. If that AMI gets deregistered and we scale up, the ASG will not be able to create an instance based on that AMI.

Would it be conceivable to have an additional option to only remove if the AMI is unused e.g. remove_if_unused ?

This doesn't seem possible to see in the AWS Console but there is a Reservations key in the describe-images awscli command which shows you where the AMI is used. If it is empty, it is unused according to cloud conformity.

aws ec2 describe-images \
	--region us-east-1 \
	--image-ids ami-3f708e52 \
	--query 'Images[*].BlockDeviceMappings[*].Ebs.SnapshotId'

That can be piped into jq '.Reservations[].Instances[].InstanceId' to get a list of instance ids.

Lately I've been waffling between the CloudCustodian project, this project, and creating a custom lambda. For now, I think only the last could currently solve this issue.

user variable for region

Hi again @wata727

I'm trying to run the post-processor with a user variable, as such:

"post-processors":[{
"type": "amazon-ami-management",
"region": "{{user destination_regions}}",
"identifier": "buildX",
"keep_releases": "1"
}
]

destination_regions is defined as:

{
"variables": {
"aws_access_key": "{{env AWS_ACCESS_KEY_ID}}",
"aws_secret_key": "{{env AWS_SECRET_ACCESS_KEY}}",
"destination_regions": "eu-west-1,eu-west-2,eu-west-3,eu-central-1,us-east-1,us-west-1,ap-northeast-1,sa-east-1",
"hostname": "buildXX"
},

I'm under the impression, however, that the post-processor cannot recognize this syntax and silently fails. Can that be the case?

Any tips on how I can make use of a variable in your post-processor part?

Thank you!

* Post-processor failed: unexpected EOF

Post-processor is failing while calling the your plugin. Its strange its working perfectly on my vagrant centos7 machine but failing while testing on RHEL 7 on AWS.


==> LinuxImage: Running post-processor: amazon-ami-management
Build 'LinuxImage' errored: 1 error(s) occurred:

* Post-processor failed: unexpected EOF

==> Some builds didn't complete successfully and had errors:
{
"variables": {
    "aws_region": "ap-southeast-1",
    "post": "amazon-ebs"
  },

  "description": "Amazon AMI's",
  "builders": [{
    "type": "amazon-ebs",
    "region":  "{{user `aws_region`}}" ,
    "vpc_id": "vpc-xxxxx",
    "subnet_id": "subnet-4xxxxxxx",
    "security_group_id": "xxxxxxxxxx",
    "source_ami": "ami-2cxxxxx",
    "instance_type": "t2.micro",
    "ssh_username": "ec2-user",
    "ami_name": "packer-example-{{timestamp}}",
    "tags": {
        "Amazon_AMI_Management_Identifier": "packer-example"
    }
  },
  {
  "type": "virtualbox-iso",
  "guest_os_type": "Ubuntu_64",
  "iso_url": "http://releases.ubuntu.com/12.04/ubuntu-12.04.5-server-amd64.iso",
  "iso_checksum": "769474248a3897f4865817446f9a4a53",
  "iso_checksum_type": "md5",
  "ssh_username": "packer",
  "ssh_password": "packer",
  "shutdown_command": "echo 'packer' | sudo -S shutdown -P now"
}
],
  "provisioners":[{
    "type": "shell",
    "inline": [
      "echo hostname",
      "sudo yum install git -y",
      "sudo yum install httpd -y"
    ]
  }],
  "post-processors":[{
    "type": "amazon-ami-management",
    "region": "ap-southeast-1",
    "identifier": "packer-example",
    "keep_releases": "1"
  }]
}

Dry Run

Thank you for putting the time in to make this post processor. I saw it in the Packer Book on page 121!

It would be nice to have a dry run feature to print what the post processor would remove if it was enabled.

Passing credentials through AWS_PROFILE + AWS_CONFIG_FILE + AWS_SHARED_CREDENTIALS_FILE environment variable fails (where packer supports them)

In my CI environment, I pass credentials to packer through the following environment variables, all supported by the AWS SDK.

AWS_PROFILE
AWS_CONFIG_FILE
AWS_SHARED_CREDENTIALS_FILE

Packer runs without error, picking the profile up from the credential / config files provided through the environment, but the post-processor fails:

2018-04-03T08:00:27.0174519Z 2018/04/03 08:00:26 packer-post-processor-amazon-ami-management: 2018/04/03 08:00:26 Running the post-processor
2018-04-03T08:00:27.0215799Z 2018/04/03 08:00:26 packer-post-processor-amazon-ami-management: 2018/04/03 08:00:26 Describing images
2018-04-03T08:02:28.8369251Z 2018/04/03 08:02:28 [INFO] (telemetry) ending amazon-ami-management
2018-04-03T08:02:28.8414930Z 2018/04/03 08:02:28 Deleting original artifact for build 'amazon-ebs'
2018-04-03T08:02:28.8448919Z 2018/04/03 08:02:28 packer: 2018/04/03 08:02:28 Deregistering image ID (ami-8d01a2f0) from region (us-east-1)
2018-04-03T08:02:30.3383783Z 2018/04/03 08:02:30 ui error: Build 'amazon-ebs' errored: 1 error(s) occurred:
2018-04-03T08:02:30.3417186Z 
2018-04-03T08:02:30.3461601Z * Post-processor failed: NoCredentialProviders: no valid providers in chain. Deprecated.
2018-04-03T08:02:30.3497528Z 	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2018-04-03T08:02:30.3524518Z 2018/04/03 08:02:30 Builds completed. Waiting on interrupt barrier...
2018-04-03T08:02:30.3550089Z 2018/04/03 08:02:30 machine readable: error-count []string{"1"}
2018-04-03T08:02:30.3579119Z 2018/04/03 08:02:30 ui error: 
2018-04-03T08:02:30.3602924Z ==> Some builds didn't complete successfully and had errors:
2018-04-03T08:02:30.3630403Z 2018/04/03 08:02:30 machine readable: amazon-ebs,error []string{"1 error(s) occurred:\n\n* Post-processor failed: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"}
2018-04-03T08:02:30.3653095Z 2018/04/03 08:02:30 ui error: --> amazon-ebs: 1 error(s) occurred:
2018-04-03T08:02:30.3663984Z 
2018-04-03T08:02:30.3679615Z * Post-processor failed: NoCredentialProviders: no valid providers in chain. Deprecated.
2018-04-03T08:02:30.3695811Z 	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2018-04-03T08:02:30.3709543Z 2018/04/03 08:02:30 ui: 
2018-04-03T08:02:30.3722535Z ==> Builds finished but no artifacts were created.
2018-04-03T08:02:30.3733868Z 2018/04/03 08:02:30 [INFO] (telemetry) Finalizing.
2018-04-03T08:02:30.3749163Z Build 'amazon-ebs' errored: 1 error(s) occurred:
2018-04-03T08:02:30.3756649Z 
2018-04-03T08:02:30.3772120Z * Post-processor failed: NoCredentialProviders: no valid providers in chain. Deprecated.
2018-04-03T08:02:30.3787153Z 	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2018-04-03T08:02:30.3792947Z 
2018-04-03T08:02:30.3806661Z ==> Some builds didn't complete successfully and had errors:
2018-04-03T08:02:30.3822017Z --> amazon-ebs: 1 error(s) occurred:
2018-04-03T08:02:30.3829326Z 
2018-04-03T08:02:30.3843821Z * Post-processor failed: NoCredentialProviders: no valid providers in chain. Deprecated.
2018-04-03T08:02:30.3858481Z 	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Post-processor failed: AuthFailure: AWS was not able to validate the provided access credentials

I've been trying to get the plugin to work on with my packer template. I'm using the Amazon EBS provider for my build and it works successfully with the assume role. However, when it comes to the post-provisioner stage, I get the following error saying invalid credentials when it is working on the builder.

source "amazon-ebs" "amazon-linux" {
  ami_name            = "${var.ami_prefix}-${var.type}-{{ timestamp }}"
  ami_regions         = var.ami_regions
  instance_type       = var.instance_type
  region              = var.primary_region
  source_ami          = data.amazon-ami.amazon-linux.id
  ebs_optimized       = true
  ena_support         = true
  enable_t2_unlimited = false

  ssh_interface        = "session_manager"
  ssh_username         = "ec2-user"
  pause_before_ssm     = "30s"
  iam_instance_profile = "packer-imagebuilder-ssm"

  vpc_filter {
    filters = {
      "tag:Name" : "packer-imagebuilder",
      "isDefault" : "false",
    }
  }

  subnet_filter {
    filters = {
      "tag:Name" = "packer-imagebuilder-private-${var.primary_region}a"
    }
    most_free = true
    random    = true
  }

  assume_role {
    role_arn     = var.deployment_role_arn
    session_name = "packer-imagebuilder"
  }
}

build {
  sources = [
    "source.amazon-ebs.amazon-linux"
  ]
  
  post-processor "amazon-ami-management" {
    regions       = concat([var.primary_region], var.ami_regions)
    identifier    = "${var.ami_prefix}-${var.type}-"
    keep_releases = 3

    assume_role {
      role_arn     = var.deployment_role_arn
      session_name = "packer-imagebuilder"
    }
  }
2023/01/04 12:39:50 [INFO] (telemetry) ending amazon-ebs.amazon-linux
2023/01/04 12:39:50 [INFO] (telemetry) Starting post-processor amazon-ami-management
==> amazon-ebs.amazon-linux: Running post-processor:  (type amazon-ami-management)
    amazon-ebs.amazon-linux (amazon-ami-management): Processing in af-south-1
2023/01/04 12:39:51 packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:51 Running the post-processor
2023/01/04 12:39:51 packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:51 [INFO] AWS Auth provider used: "EnvProvider"
2023/01/04 12:39:51 packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:51 [INFO] Attempting to AssumeRole arn:aws:iam::xxxxxx:role/packer (SessionName: "packer-imagebuilder", ExternalId: "")
2023/01/04 12:39:51 packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:51 Found region eu-west-1
2023/01/04 12:39:51 packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:51 [INFO] AWS Auth provider used: "AssumeRoleProvider"
2023/01/04 12:39:53 [INFO] (telemetry) ending amazon-ami-management
2023/01/04 12:39:53 Deleting original artifact for build 'amazon-ebs.amazon-linux'
2023/01/04 12:39:53 packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:53 error: AuthFailure: AWS was not able to validate the provided access credentials
2023/01/04 12:39:53 packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_arm64 plugin: 	status code: 401, request id: d36201a0-eded-457c-815c-af2f417870e4
2023/01/04 12:39:53 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:53 Deregistering image ID (ami-04bc46ae4f51f43db) from region (af-south-1)
2023/01/04 12:39:58 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:58 Deregistered AMI id: ami-04bc46ae4f51f43db
2023/01/04 12:39:59 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:59 Deleted snapshot: snap-00c57b2db9bc56025
2023/01/04 12:39:59 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:39:59 Deregistering image ID (ami-008353bfaf323c514) from region (ap-southeast-1)
2023/01/04 12:40:04 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:40:04 Deregistered AMI id: ami-008353bfaf323c514
2023/01/04 12:40:06 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:40:06 Deleted snapshot: snap-0bec51acca6efcde6
2023/01/04 12:40:06 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:40:06 Deregistering image ID (ami-0bdf2163907803ea4) from region (eu-west-1)
2023/01/04 12:40:09 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:40:09 Deregistered AMI id: ami-0bdf2163907803ea4
Build 'amazon-ebs.amazon-linux' errored after 11 minutes 38 seconds: 1 error(s) occurred:

* Post-processor failed: AuthFailure: AWS was not able to validate the provided access credentials
	status code: 401, request id: xxxxx

==> Wait completed after 11 minutes 38 seconds

==> Some builds didn't complete successfully and had errors:

--> amazon-ebs.amazon-linux: 1 error(s) occurred:
* Post-processor failed: AuthFailure: AWS was not able to validate the provided access credentials

	status code: 401, request id: xxxxxxx
==> Wait completed after 11 minutes 38 seconds
2023/01/04 12:40:10 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2023/01/04 12:40:10 machine readable: amazon-ebs.amazon-linux,error []string{"1 error(s) occurred:\n\n* Post-processor failed: AuthFailure: AWS was not able to validate the provided access credentials\n\tstatus code: 401, request id: * Post-processor failed: AuthFailure: AWS was not able to validate the provided access credentials
	status code: 401, request id: xxxxxxx

==> Builds finished but no artifacts were created.
xxxxxxx"}

* Post-processor failed: AuthFailure: AWS was not able to validate the provided access credentials
	status code: 401, request id: xxxxxx
==> Builds finished but no artifacts were created.
2023/01/04 12:40:10 packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64 plugin: 2023/01/04 12:40:10 Deleted snapshot: snap-013963d646f90692f
2023/01/04 12:40:10 [INFO] (telemetry) Finalizing.
2023/01/04 12:40:11 waiting for all plugin processes to complete...
2023/01/04 12:40:11 /Users/choon_ming_goh/.config/packer/plugins/github.com/wata727/amazon-ami-management/packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_arm64: plugin process exited
2023/01/04 12:40:11 /Users/choon_ming_goh/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64: plugin process exited
2023/01/04 12:40:11 /Users/choon_ming_goh/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.1.6_x5.0_darwin_arm64: plugin process exited

Keep AMIs for the last X days

Thanks a lot for the last one! This probably falls under "give an inch and they'll take a mile" but one more feature request. It would be nice to keep the AMIs for a number of days instead of (or in addition to) the number of images.

For instance

  "post-processors":[{
    "type": "amazon-ami-management",
    "regions": ["us-east-1"],
    "identifier": "packer-example",
    "keep_releases": "30"
  }]

^ keep the last 30 releases

or

  "post-processors":[{
    "type": "amazon-ami-management",
    "regions": ["us-east-1"],
    "identifier": "packer-example",
    "keep_releases_for": "90"
  }]

^ keep releases created within the last 90 days

Thanks again!

post processor fails with Unexpected EOF if any Launch Template in the account does not have an AMI ImageId

in cleaner.go it will iterate through every launch template in the account and every launch template version. From there if one of the old launch template versions does not have an associated imageId (i.e someone deregisters the image), the post processor will throw an Unexpected EOF.

The workaround is to delete the now invalid launch template version(s) and rerun. This is problematic because maybe the launch templates causing the issue may not your responsibility / may be completely unrelated to the packer builder.

To resolve the issue cleaner.go would need to catch this error (null imageId) and continue iterating over the launch templates. Since the ImageId is null it is safe to do so since obviously it does not match the imageId to be deleted.

Here is an example log for when this happens:

2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: panic: runtime error: invalid memory address or nil pointer dereference
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x11399f0]
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: goroutine 24 [running]:
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: main.(*Cleaner).setLaunchTemplateUsed(0xc000308000, 0x0, 0x0)
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:         /home/runner/work/packer-post-processor-amazon-ami-management/packer-post-processor-amazon-ami-management/cleaner.go:220 +0x200
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: main.NewCleaner(0xc00000a3c0, 0xc000574650, 0xe, 0xc000574690, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:         /home/runner/work/packer-post-processor-amazon-ami-management/packer-post-processor-amazon-ami-management/cleaner.go:62 +0x213
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: main.(*PostProcessor).PostProcess(0xc000304800, 0x18ba4c0, 0xc0001cc800, 0x18c0a20, 0xc00058abd0, 0x18c0a80, 0xc00056bc00, 0x7f34d472e640, 0x203000, 0xaa, ...)
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:         /home/runner/work/packer-post-processor-amazon-ami-management/packer-post-processor-amazon-ami-management/post-processor.go:80 +0x737
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: github.com/hashicorp/packer/packer/rpc.(*PostProcessorServer).PostProcess(0xc0001cd700, 0x1, 0xc000030020, 0x0, 0x0)
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:         /home/runner/go/pkg/mod/github.com/hashicorp/[email protected]/packer/rpc/post_processor.go:109 +0x277
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: reflect.Value.call(0xc00007bc80, 0xc00000eb48, 0x13, 0x15cd42d, 0x4, 0xc00008ef18, 0x3, 0x3, 0xc0004b9e78, 0x40541c, ...)
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:         /opt/hostedtoolcache/go/1.13.8/x64/src/reflect/value.go:460 +0x5f6
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: reflect.Value.Call(0xc00007bc80, 0xc00000eb48, 0x13, 0xc0004b9f18, 0x3, 0x3, 0xc0004085a0, 0xc0003f9200, 0xc0003f9288)
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:         /opt/hostedtoolcache/go/1.13.8/x64/src/reflect/value.go:321 +0xb4
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: net/rpc.(*service).call(0xc0001cd740, 0xc00009f400, 0xc000374868, 0xc000374870, 0xc0000e4a00, 0xc0003077c0, 0x125c4c0, 0xc0000380f0, 0x18a, 0x12092e0, ...)
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:         /opt/hostedtoolcache/go/1.13.8/x64/src/net/rpc/server.go:377 +0x16f
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin: created by net/rpc.(*Server).ServeCodec
2020/10/14 18:29:50 packer-post-processor-amazon-ami-management plugin:         /opt/hostedtoolcache/go/1.13.8/x64/src/net/rpc/server.go:474 +0x42b
2020/10/14 18:29:50 [INFO] (telemetry) ending amazon-ami-management
2020/10/14 18:29:50 Deleting original artifact for build 'amazon-ebs.ami-management'
2020/10/14 18:29:50 /home/ec2-user/.packer.d/plugins/packer-post-processor-amazon-ami-management: plugin process exited
2020/10/14 18:29:50 packer-builder-amazon-ebs plugin: Deregistering image ID (ami-XXXX) from region (REGION)
2020/10/14 18:29:50 packer-builder-amazon-ebs plugin: Deregistered AMI id: ami-XXXX
2020/10/14 18:29:50 packer-builder-amazon-ebs plugin: Deleted snapshot: snap-XXXX
2020/10/14 18:29:50 packer-builder-amazon-ebs plugin: Deleted snapshot: snap-XXXX
2020/10/14 18:29:50 ui error: Build 'amazon-ebs.ami-management' errored: 1 error(s) occurred:```

invalid memory address or nil pointer dereference

Hi!
I am getting below error when I try to use this plugin on eu-central-1. I also have parallel bake operations where I create exact copy of the same AMI in different locations. This problem only occurs on eu-central-1. Rest of the regions are working perfectly fine.

Here's the error log:

panic: runtime error: invalid memory address or nil pointer dereference
108 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1001cb7]
109 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 
110 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: goroutine 93 [running]:
111 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: main.(*Cleaner).setLaunchTemplateUsed(0xc0004da000, 0x0, 0x0)
112 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 	/Users/watanabekazuma/workspace/golang/src/github.com/wata727/packer-post-processor-amazon-ami-management/cleaner.go:220 +0x207
113 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: main.NewCleaner(0xc0004d41c0, 0xc00048c160, 0x13, 0xc000438280, 0xa, 0x100, 0x0, 0x0, 0xc0004856e0, 0x0, ...)
114 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 	/Users/watanabekazuma/workspace/golang/src/github.com/wata727/packer-post-processor-amazon-ami-management/cleaner.go:62 +0x221
115 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: main.(*PostProcessor).PostProcess(0xc0003e4000, 0x163bcc0, 0xc000304a00, 0x1641780, 0xc0003090a0, 0x16417e0, 0xc0003090c0, 0xc00009eb78, 0xc0003f6020, 0x2, ...)
116 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 	/Users/watanabekazuma/workspace/golang/src/github.com/wata727/packer-post-processor-amazon-ami-management/post-processor.go:88 +0x753
117 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: github.com/hashicorp/packer/packer/rpc.(*PostProcessorServer).PostProcess(0xc00030b470, 0x1, 0xc0004993e0, 0x0, 0x0)
118 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 	/Users/watanabekazuma/workspace/golang/pkg/mod/github.com/hashicorp/[email protected]/packer/rpc/post_processor.go:107 +0x243
119 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: reflect.Value.call(0xc00030ea80, 0xc000340148, 0x13, 0x140198a, 0x4, 0xc00009ef18, 0x3, 0x3, 0xc000328fc0, 0x120a220, ...)
120 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 	/usr/local/Cellar/go/1.12.7/libexec/src/reflect/value.go:447 +0x461
121 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: reflect.Value.Call(0xc00030ea80, 0xc000340148, 0x13, 0xc0004aaf18, 0x3, 0x3, 0xc00030e960, 0xc0004aaec0, 0x2)
122 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 	/usr/local/Cellar/go/1.12.7/libexec/src/reflect/value.go:308 +0xa4
123 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: net/rpc.(*service).call(0xc000304980, 0xc00034c3c0, 0xc00003fd30, 0xc00003fd40, 0xc000344900, 0xc0003f6040, 0x1107400, 0xc000438520, 0x18a, 0x10bc320, ...)
124 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 	/usr/local/Cellar/go/1.12.7/libexec/src/net/rpc/server.go:384 +0x14e
125 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: created by net/rpc.(*Server).ServeCodec
126 2019/12/30 12:50:39 packer-post-processor-amazon-ami-management: 	/usr/local/Cellar/go/1.12.7/libexec/src/net/rpc/server.go:481 +0x42b
127 2019/12/30 12:50:39 [INFO] (telemetry) ending amazon-ami-management
128 2019/12/30 12:50:39 ui error: Build 'apibalancer builder' errored: 1 error(s) occurred:
129 * Post-processor failed: unexpected EOF
130 2019/12/30 12:50:39 machine readable: error-count []string{"1"}
131 2019/12/30 12:50:39 ui error: 
132 ==> Some builds didn't complete successfully and had errors:
133 2019/12/30 12:50:39 machine readable: apibalancer builder,error []string{"1 error(s) occurred:\n\n* Post-processor failed: unexpected EOF"}
134 2019/12/30 12:50:39 ui error: --> apibalancer builder: 1 error(s) occurred:
135 * Post-processor failed: unexpected EOF
136 2019/12/30 12:50:39 ui: 
137 ==> Builds finished but no artifacts were created.
138 2019/12/30 12:50:39 /home/gitlab-runner/.packer.d/plugins/packer-post-processor-amazon-ami-management: plugin process exited
139 2019/12/30 12:50:39 Cancelling post-processor after context cancellation context canceled
140 2019/12/30 12:50:39 Error cancelling post-processor: connection is shut down
141 2019/12/30 12:50:39 [INFO] (telemetry) Finalizing.
142 2019/12/30 12:50:39 waiting for all plugin processes to complete...
143 2019/12/30 12:50:39 /usr/local/bin/packer: plugin process exited
144 2019/12/30 12:50:39 /usr/local/bin/packer: plugin process exited
145 2019/12/30 12:50:39 /usr/local/bin/packer: plugin process exited
146 2019/12/30 12:50:39 /usr/local/bin/packer: plugin process exited
147 2019/12/30 12:50:39 /usr/local/bin/packer: plugin process exited
148 !!!!!!!!!!!!!!!!!!!!!!!!!!! PACKER CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
149 Packer crashed! This is always indicative of a bug within Packer.
150 A crash log has been placed at "crash.log" relative to your current
151 working directory. It would be immensely helpful if you could please
152 report the crash with Packer[1] so that we can fix this.
153 [1]: https://github.com/hashicorp/packer/issues
154 !!!!!!!!!!!!!!!!!!!!!!!!!!! PACKER CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!
157
bash: line 82: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8): No such file or directory
00:00
158 bash: line 82: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8)
160 ERROR: Job failed: exit status 1

Post-processor failed: unexpected EOF

Hello,

As, i want to manage the AWS AMI retention, and after some research i came across this plugin, versions of packer and plugin are given below, but while executing the packer build command, at the stage of post processor [amazon-ami-management], unexpected EOF error is coming:

Version:

  1. Packer V1.0.0
  2. Packer post processor ami management V0.1.1

Packer configuration:
{
"description": "Amazon AMI's",
"builders": [{
"type": "amazon-ebs",
"region": "ap-southeast-1" ,
"vpc_id": "vpc-XXXXXXX",
"subnet_id": "subnet-XXXXXX",
"security_group_id": "sg-XXXXXXX",
"source_ami": "ami-XXXXXXX",
"instance_type": "t2.micro",
"ssh_username": "ec2-user",
"ami_name": "packer-example-{{timestamp}}",
"tags": {
"Amazon_AMI_Management_Identifier": "packer-example"
}
}
],
"post-processors":[{
"type": "amazon-ami-management",
"region": "ap-southeast-1",
"identifier": "packer-example",
"keep_releases": "2"
}]
}

Packer build logs:
amazon-ebs output will be in this color.

==> amazon-ebs: Prevalidating AMI Name...
amazon-ebs: Found Image ID: ami-2c95344f
==> amazon-ebs: Creating temporary keypair: packer_59214322-e1a5-b9ee-6d94-250b2e1e2841
==> amazon-ebs: Launching a source AWS instance...
amazon-ebs: Instance ID: i-0f06fbaa1ca05c7f9
==> amazon-ebs: Waiting for instance (i-0f06fbaa1ca05c7f9) to become ready...
==> amazon-ebs: Adding tags to source instance
amazon-ebs: Adding tag: "Name": "Packer Builder"
==> amazon-ebs: Waiting for SSH to become available...
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Stopping the source instance...
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating the AMI: packer-example-1495352098
amazon-ebs: AMI: ami-bf1691dc
==> amazon-ebs: Waiting for AMI to become ready...
^[[6~==> amazon-ebs: Adding tags to AMI (ami-bf1691dc)...
==> amazon-ebs: Tagging snapshot: snap-0b8c8ac4344792751
==> amazon-ebs: Creating AMI tags
amazon-ebs: Adding tag: "Amazon_AMI_Management_Identifier": "packer-example"
==> amazon-ebs: Creating snapshot tags
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Deleting temporary keypair...
==> amazon-ebs: Running post-processor: amazon-ami-management
Build 'amazon-ebs' errored: 1 error(s) occurred:

  • Post-processor failed: unexpected EOF

==> Some builds didn't complete successfully and had errors:
--> amazon-ebs: 1 error(s) occurred:

  • Post-processor failed: unexpected EOF

==> Builds finished but no artifacts were created.

Error:
image

Please let me know about this issue, what could be root cause for this?

Thanks in advance.

Regards,
Sandeep

Getting Failed getting the "github.com/wata727/amazon-ami-management" plugin error

I am using github.com/wata727/amazon-ami-management plugin in my github action pipeline to manage packer AMIs,
i have started getting below error

Run packer init aws-ms_windows.pkr.hcl
Failed getting the "github.com/wata727/amazon-ami-management" plugin:
1 error occurred:
* binary reported a pre-release version of 1.4.1-dev
Remote installation of pre-release plugin versions is unsupported.
This is likely an upstream issue, which should be reported.

image

Feature Request: make checking launch templates more configurable

We use terraform to manage our launch templates: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template

and it doesn't delete old versions ever. It would be nice if there was an option on this plugin to still delete amis that are part of launch templates, but not launch template verions that are used in ASGs, or the latest, or the default version of a launch template. Otherwise nothing ever gets deleted unless we go in manually and delete a bunch of old launch template versions.

It might be nice if that was completely configurable, options like:

launch_template_versions = { # launch templates versions to look at
  latest = 6  # do not consider amis from the most recent 6 versions of launch templates for deletion
  asg = true  # do not consider amis that are part of launch template versions configured in asgs for deletion
  all = true  # do not consider amis part of any launch template version for deletion
  default = true  # do not consider amis part of the default launch template version for deletion
}

This could default to:

launch_template_versions = {
  all = true
}

just to be on the safe side. I believe this is how it behaves currently.

I would probably personally set it to:

launch_template_versions = {
  latest = 3
  asg = true
}

to delete amis from all launch templates other than the most recent 3, and any that are part of asgs.

An alternative is for terraform to implement an argument to limit the number of versions of the launch template that are kept.

My current workaround is to put the ami id in the name_prefix so there's only ever one ami in any versions of the launch template:

resource "aws_launch_template" "main" {
  update_default_version = true
  name_prefix            = "some-descriptive-name-${data.aws_ami.image.image_id}"
  lifecycle {
    create_before_destroy = true
  }
  ...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.