shantanugadgil/cob

Yet Another Yum S3 Plugin (AWS SigV4)

Cob

Cob, yet another yum S3 plugin, provides the way to accessing yum repository hosted on AWS S3.

What's the difference between Cob and original yum s3 plugin?

• Support more secure AWS Signature Version 4 while original one still in version 2, especially for the new region eu-central-1 only SigV4 allowed
• Hook on higher layer of Yum built-in library to avoid complicated low-level handlings
• Support static AWS credentials prior to IAM role
• Add retry mechanism to fetch IAM role credentials

Quick Start

• Installation

  • ./install.sh
  • plugin conf: cob.conf --> /etc/yum/pluginconf.d/cob.conf
  • plugin code: cob.py --> /usr/lib/yum-plugins/cob.py

• Setup minimal IAM Role Policy for Cob

  {
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "s3:GetObject"
        ],
        "Resource": "*"
      }
    ]
  }

  • For cross-account access, setup the policy of yum s3 bucket

    {
      "Version": "2008-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "AWS": [
              "arn:aws:iam::37ABC0340XYZ:root",
              "arn:aws:iam::24ABC3058XYZ:root"
            ]
          },
          "Action": "s3:GetObject",
          "Resource": "arn:aws:s3:::yum-s3-bucket-XYZ/*"
        }
      ]
    }

    • "37ABC0340XYZ", "24ABC3058XYZ": AWS account id with the permission to access
    • "yum-s3-bucket-XYZ": S3 bucket for yum access

• Configure your yum repo conf under /etc/yum.repos.d/, like the example below cob.repo

  [cob]
  name=cob
  baseurl=https://your-bucket-name-0.s3.amazonaws.com/repo-name/arch/
          https://your-bucket-name-1.s3-eu-west-1.amazonaws.com/repo-name/arch/
          https://your-bucket-name-2.s3-us-west-2.amazonaws.com/repo-name/arch/
  failovermethod=priority
  enabled=1
  gpgcheck=0

  • add the bucket region name in the baseurl as shown in the example, especially for us-east-1, like your-bucket-name-0, no region name needed there
  • s3 endpoint reference: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

• An example from cob.conf is taken to indicate its usages:

  [main]
  cachedir=/var/cache/yum/$basearch/$releasever
  keepcache=1
  debuglevel=4
  logfile=/var/log/yum.log
  exactarch=1
  obsoletes=0
  gpgcheck=0
  plugins=1
  distroverpkg=centos-release
  enabled=1
  
  [aws]
  # access_key = 
  # secret_key =
  timeout = 60
  retries = 5
  metadata_server = http://169.254.169.254

  • set main/enabled=1 to enable this yum plugin
  • for static AWS credentials, you could specify via aws/access_key, aws/secret_key
  • aws/timeout and aws/retries, used to indicate params in the way of fetching IAM role credentials
  • metadata_server used to help testing

• Enable verbose log to help troubleshoot the Cob issue:

  URLGRABBER_DEBUG=1 yum -v makecache