martinbaillie/packer-plugin-ami-copy

Packer post-processor plugin for copying, encrypting, tagging a built AMI into other accounts.

packer-plugin-ami-copy

Description

This plugin fills a gap in a lot of AWS image bakery workflows where the source image built by any of Packer's Amazon builders (EBS, Chroot, Instance etc.) needs to be copied to a number of target accounts.

For each region:ami-id built, the plugin will copy the image and tags, and optionally encrypt the target AMI and wait for it to become active.

Installation

Using pre-built releases

Using the packer init command

Starting from version 1.7, Packer supports a new packer init command allowing automatic installation of Packer plugins. Read the Packer documentation for more information.

Manual installation

You can find pre-built binary releases of the plugin here. Once you have downloaded the latest archive corresponding to your target OS, uncompress it to retrieve the plugin binary file corresponding to your platform. To install the plugin, please follow the Packer documentation on installing a plugin.

You can and should verify the authenticity and integrity of the plugin you downloaded. All released binaries are hashed and the resulting sums are signed by my GPG key.

# Import my key.
curl -sS https://github.com/martinbaillie.gpg | gpg --import -

# Verify the authenticity.
gpg --verify SHA256SUMS.sig SHA256SUMS

# Verify the integrity.
shasum -a 256 -c SHA256SUMS

From Sources

If you prefer to build the plugin from sources you will need a modern Go compiler toolchain (Go 1.16+ ideally). If you are a Nix user, shell.nix can be of use here.

Clone the GitHub repository locally then run a go build from the root. Upon successful compilation, a packer-plugin-ami-copy plugin binary will be produced. To install it, follow the official Packer documentation on installing a plugin.

Usage

For more information on how to use the plugin, see the docs/ and examples/.

Configuration

Type: ami-copy

Required:

• ami_users (array of strings) - A list of account IDs to copy the images to. NOTE: you must share AMI and snapshot access in the builder through ami_users and snapshot_users respectively.

Optional:

• copy_concurrency (integer) - Limit the number of copies executed in parallel (default: unlimited).
• encrypt_boot (boolean) - create the copy with an encrypted EBS volume in the target accounts
• kms_key_id (string) - the ID of the KMS key to use for boot volume encryption. (default EBS KMS key used otherwise).
• ensure_available (boolean) - wait until the AMI becomes available in the copy target account(s)
• keep_artifact (boolean) - remove the original generated AMI after copy (default: true)
• manifest_output (string) - the name of the file we output AMI IDs to, in JSON format (default: no manifest file is written)
• tags_only (boolean) - if set to true, then the AMI won't be copied, but the tags will be duplicated on the shared AMI in the destination account.

packer-plugin-ami-copy's Issues

[Feature request] AMI Copy with encryption

Hello,
I recently saw that AWS now supports copying an AMI with a Customer Managed Key. This feature will be really useful in the future in order to use it in a Production environment

Copy tasks should check for AMIs to become available in target account(s)

At the moment the code will exit when Amazon returns success on the copy tasks. However, the AMI is most likely in a pending state, especially if there are EBS copies involved.

Is the AMI ID retained after the copy from account A to account B ?

Hi @martinbaillie . Thank you so much for creating this post processor. I found this while looking for post processors that could copy AMIs to a list of accounts and copy the tags of those AMIs to the accounts. And an extra benefit is encryption!

• When the AMIs are copied to the other accounts, are the AMI IDs different from the original account or are the same AMI IDs retained?
  • I'm guessing yes because it's a copy of an AMI.
• Are there additional packer IAM permissions to get this to work ? Does the user/role that runs the packer job require certain IAM permissions or role assumption for the other accounts ?
  • Edit: Nevermind, I see this other issue that mentions it #6 (comment)

The docs/ and examples/ link in the Readme return 404

Hi,
The docs/ and examples/ link here https://github.com/martinbaillie/packer-plugin-ami-copy#usage return a 404 error code. Can you fix this please!

Protocol issue with Packer 1.7

Getting the following when using the plugin w/ Packer 1.7

error initializing post-processor 'ami-copy': The protocol of this plugin (protocol version 4 and lower) was deprecated, please use a newer version of this plugin.Or use an older version of Packer (pre 1.7) with this plugin.

Unexpected EOF

I'm sure I'm just not configured correctly but I'm trying to copy over tags. I'm getting the below error message when ami-copy is executed, it's a little cryptic so I'm not sure what is wrong. Any thoughts? Below is a redacted snip of the code.

Error Message:

==> amazon-ebs.proxy: Running post-processor:  (type ami-copy)
Build 'amazon-ebs.proxy' errored after 18 minutes 11 seconds: 1 error(s) occurred:

* Post-processor failed: unexpected EOF

==> Wait completed after 18 minutes 11 seconds

==> Some builds didn't complete successfully and had errors:
--> amazon-ebs.proxy: 1 error(s) occurred:

* Post-processor failed: unexpected EOF

==> Builds finished but no artifacts were created.

proxy.pkr.hcl:

packer {
  required_plugins {
    ami-copy = {
      version = ">=v1.7.0"
      source  = "github.com/martinbaillie/ami-copy"
    }
  }
}

variable "aws_access_key_id" {
  type    = string
  default = "${env("AWS_ACCESS_KEY_ID")}"
}

variable "aws_secret_access_key" {
  type    = string
  default = "${env("AWS_SECRET_ACCESS_KEY")}"
}

variable "aws_session_token" {
  type    = string
  default = "${env("AWS_SESSION_TOKEN")}"
}

variable "aws_ami_users" {
  default = <redacted>
}

variable "aws_ami_regions" {
  default = <redacted>
}

data "amazon-ami" "source" {
  access_key  = "${var.aws_access_key_id}"
  secret_key  = "${var.aws_secret_access_key}"
  token       = "${var.aws_session_token}"
  region      = <redacted>
  owners      = <redacted>
  filters = {
    architecture        = <redacted>
    name                = <redacted>
    root-device-type    = <redacted>
    virtualization-type = <redacted>
  }
  most_recent = true
}

source "amazon-ebs" "proxy" {
  access_key                  = "${var.aws_access_key_id}"
  secret_key                  = "${var.aws_secret_access_key}"
  token                       = "${var.aws_session_token}"
  region                      = <redacted>
  vpc_id                      = <redacted>
  subnet_id                   = <redacted>
  security_group_id           = <redacted>
  ami_name                    = "daphne-${var.type}-proxy ubuntu ${legacy_isotime("20060102T150405.999999999")}"
  ami_regions                 = "${var.aws_ami_regions}"
  ami_users                   = "${var.aws_ami_users}"
  source_ami                  = "${data.amazon-ami.source.id}"
  ssh_username                = <redacted>
  instance_type               = <redacted>
  associate_public_ip_address = true

  tags = {
    OS_Version      = <redacted>
    OperatingSystem = <redacted>
    Project         = <redacted>
    Usage           = <redacted>
    SourceAMI       = "{{ .SourceAMI }}"
  }

  run_tags = {
    OS_Version      = <redacted>
    OperatingSystem = <redacted>
    Project         = <redacted>
    Usage           = <redacted>
    SourceAMI       = "{{ .SourceAMI }}"
  }
}

build {
  sources = ["source.amazon-ebs.proxy"]

  post-processor "ami-copy" {
    ami_users    = "${var.aws_ami_users}"
  }
}

amazon-import not supported?

I have a build that creates an AMI using the amazon-import post-processor, which does not support encryption.
For this, I'd like to use the ami-copy post-processor to encrypt the imported ami, however the module does not accept that artifact as input:

Build 'file' errored: 1 error(s) occurred:                                                                                                                     
                                                                              
* Post-processor failed: Unexpected artifact type: packer.file
Can only export from Amazon builders       

I'm using the file builder to import an existing ova file, like this:

{
  "builders":
  [{
    "type": "file",
    "source": "files/appliance.ova",
    "target": "output-file/appliance.ova"
  }],
  "post-processors":
  [{
    "type": "amazon-import",
    "s3_bucket_name": "vmimportbucket",
    "ami_name": "appliance-import",
    "license_type": "BYOL"
  },
  {
    "type": "ami-copy",
    "ami_users": "{{ account_id }}",
    "kms_key_id": "{{ kms_key_id }}",
    "encrypt_boot": true,
    "ensure_available": true
  }]
}

Plugin cannot be used because the name has packer in it

as on packer version 1.7.0

Plugin source "github.com/martinbaillie/packer-post-processor-ami-copy" has a type with the prefix "packer-", which isn't allowed because it would be redundant to name a Packer plugin with that prefix. If you are the author of this plugin, rename it to not include the prefix.

This happens while running packer-init

I think this will mean renaming the repository?

It would be nice to give an organization unit (OU) instead of a list of `ami_users`

If an organization unit can be given i.e. ami_ou instead of a list of ami_users, then the account ids can be listed within the organizational unit and those accounts can be used to share the AMI.

The benefit is that we can expand and contract the list of accounts that we'd want to share our AMIs with without having to update the packer configuration.

Cross region copy

Hey Team,

We're happy with your plugin but we have a slightly complex architecture. We have different accounts with different regions enabled. Our "build" account has all regions enabled at the same time production account A only region A, production account B only region B, and so on. The problem is we have to enable all required regions in every account.

Is it possible to copy AMIs using a custom map? Where we can set for each account required region.
Or enable some kind of "error ignoring". To ignore "access denied" errors for part of the regions in accounts.

Which AWS permissions do I need to give to make the copy and tagging work.

Hi @martinbaillie,

• I'm loved your plugin and am trying to use it now. My problem is that I need to restrict the permissions for the role this plugin will use and I'm unable to find which permissions it needs.
• Can you help with this?