airshipit/images

Collection of supplementary Airship images

images's Issues

Create Customized Ironic Images for Airship 2.0

Incorporate required customization into local Airship ironic images.

Example provided by @sb464f
ironic customization.txt

Enhance helm-chart-collator to authenticate with private git repos

Today, helm-chart-collator can collate charts from open git source repos, but not from ones that require authentication. This issue is to add authentication capabilities.

The underlying ansible library used for git operations supports authentication already, so this should be a straightfoward change to the collator.

test

Describe the bug
A clear and concise description of what the bug is.

Steps To Reproduce
Minimal steps to reproduce the issue.

Expected behavior
A clear and concise description of what you expected to happen.

Environment

• image buillder Version:
• Operating System:
• Kernel version:
• Kubernetes Version:
• Go version:
• Hardware specs (e.g. 4 vCPUs, 16GB RAM, bare metal vs VM):

Add default NTP configuration to ISO image cloud-init

As a Image Builder user, I want to have explicit default NTP server configuration specified in the ISO cloud-init, so that I am able to override the NTP settings downstream when generating the ISO image.

Currently the NTP server is not explicitly set, so the Ubuntu default configuration is used. This prevents users from overriding the NTP server information with the own values when generating the ISO image.

Update networking data to add default NTP server entries. Update the ansible task to set the NTP server configuration based on the declared list of NTP servers.

Add docker image tagging to generate QCOW image bundles

There is a need to identify multiple sets of QCOW images that are generated by Image Builder. As an example, there may be a set of host QCOW images (control plane & worker) & a set to define the sub-clusters VMs.

This issue requests that introduce docker image tagging which will allow a user to specify what type of image pairs are being generated.

Release automation

Problem description (if applicable)
We currently have no release automation to:

• publish semver image tags to quay
• publish release notes

Proposed change
Add similar release automation to that of airshipctl/krm functions: airshipit/airshipctl#354

I believe the current direction is to use a single version (git tag) for all images in this repo, rather than independent versions for each image.

Images publishing is broken, Zuul jobs running out of space

Describe the bug

The airship/images publishing jobs have been failing due to running out of space on the build VMs:

https://zuul.openstack.org/builds?job_name=airship-images-publish-commit
https://zuul.openstack.org/builds?job_name=airship-images-publish-latest

Hopefully moving the image builder to its own repo should allow the builds to fit tempoararily: #22

If we add more images in the future though we may be back in the same situation. A more permanent solution may be to create individual Zuul jobs per image, and only run each job when the code for that image changes using:
https://zuul-ci.org/docs/zuul/reference/job_def.html?highlight=irrelevant#attr-job.files

We could probably then undo the change to split latest vs commit publish jobs as a per image split should be granular enough:
https://review.opendev.org/c/airship/images/+/784703
https://review.opendev.org/c/airship/images/+/786955

Expected behavior

Image publishing jobs do not run out of space, so that the images get automatically published after each change to their source code.

Create a build pipeline process for images with latest or master for version

Some images are not providing the version number in their tag, just latest or master. Those without specific version numbers are overwriting the existing image so that image is no longer available. This can cause issues if rollback required because the older image is no longer there.

This feature would create a build pipeline process for "versionless" images to extract & assign a version number. Once the version number has been determined, then need to update the artifacts with the correct version number vs. latest/master

Examples for Metal3 Bare Metal Operator (BMO), Ironic & Cluster API controller:
Image Version
gcr.io/k8s-staging-cluster-api/cluster-api-controller master
rdm9r008c001.rdm9a.cci.att.com:5000/quay.io/metal3-io/baremetal-operator latest

Images with latest tag are not being pushed to quay

Describe the bug
The publish job isn't using only images tagged with commit hash

Steps To Reproduce
Check latest updated tags of images in quay.io

Expected behavior
New images with tags latest-... appear with quay.io with each newly merged commit

Possible cause
It appears that the job name has to be unique in gate definition.
https://github.com/airshipit/images/blob/master/.zuul.yaml#L39-L42

Install VRRP keepalive supporting packages in base Image Builder image

As part of delivering VRRP keepalive support in airshipit/treasuremap#94, the following packages are being updated/installed: bridge-utils, keepalived, ipset, ipvsadm. These packages should be included in the base Image Builder docker image instead of being installed in Treasuremap.

The current treasuremap code can be found here:
https://review.opendev.org/c/airship/treasuremap/+/778953/35/manifests/function/k8scontrol-ha/vrrp_keepalived_patch.yaml

Update the Image Builder docker image to install/update the above & remove the install/update from Treasuremap.

This needs to be implemented in v2.0 as airshipit/treasuremap#121 will remove the package installation from Treasuremap with its update.

Mirror dockerhub images to quay.io

Docker hub ratelimiting continues to be a thorn in our side. Our reference manifests now only point to a few remaining images in dockerhub.

This issue is to add a job to the Jenkins in the STL lab to effectively mirror the images in our remaining dockerhub registries into quay.io/airshipit. This could be done periodically (e.g. nightly) or could be triggered by changes to the dockerhub registries, if that's doable.

The versions.yamls in airshipctl and treasuremap should be updated to to point to the quay.io mirrors as part of this issue.

Libvirt base image is using outdated Xenial base image

Describe the bug
Ubuntu Xenial support has ended on April, 2021. Libvirt base image is using outdated base image.

Expected behavior
Libvirt base image to be based on Ubuntu Bionic, which is supported until April, 2023. Latest available tag to date is ubuntu_bionic-20210514.

libvirt: HIGH security vulnerability

Describe the bug

CVE-2020-1971: [High] 
Found in: openssl [1.0.2g-1ubuntu4.15]
Fixed By: 1.0.2g-1ubuntu4.18
The X.509 GeneralName type is a generic type for representing different types of names. One of those name types is known as EDIPartyName. OpenSSL provides a function GENERAL_NAME_cmp which compares different instances of a GENERAL_NAME to see if they are equal or not. This function behaves incorrectly when both GENERAL_NAMEs contain an EDIPARTYNAME. A NULL pointer dereference and a crash may occur leading to a possible denial of service attack. OpenSSL itself uses the GENERAL_NAME_cmp function for two purposes: 1) Comparing CRL distribution point names between an available CRL and a CRL distribution point embedded in an X509 certificate 2) When verifying that a timestamp response token signer matches the timestamp authority name (exposed via the API functions TS_RESP_verify_response and TS_RESP_verify_token) If an attacker can control both items being compared then that attacker could trigger a crash. For example if the attacker can trick a client or server into checking a malicious certificate against a malicious CRL then this may occur. Note that some applications automatically download CRLs based on a URL embedded in a certificate. This checking happens prior to the signatures on the certificate and CRL being verified. OpenSSL's s_server, s_client and verify tools have support for the "-crl_download" option which implements automatic CRL downloading and this attack has been demonstrated to work against those tools. Note that an unrelated bug means that affected versions of OpenSSL cannot parse or construct correct encodings of EDIPARTYNAME. However it is possible to construct a malformed EDIPARTYNAME that OpenSSL's parser will accept and hence trigger this attack. All OpenSSL 1.1.1 and 1.0.2 versions are affected by this issue. Other OpenSSL releases are out of support and have not been checked. Fixed in OpenSSL 1.1.1i (Affected 1.1.1-1.1.1h). Fixed in OpenSSL 1.0.2x (Affected 1.0.2-1.0.2w).
http://people.ubuntu.com/~ubuntu-security/cve/CVE-2020-1971

Image Builder ability to create QCOW from declarative

This will be a pivotal component of how we introduce NC customizations and evolution to Airship 2.0.

How will this work? How will it be declarative?

How can it separate rebuilding the root (thus all new .debs/etc) from introducing a targeted layer change (e.g. a NC release wants to target updating i40e, NOTHING ELSE.)

Some considerations:

• Should be same dockerfile for ephmeral iso & target image
• Should split concerns between building rootfs (during docker img build) and packaging (ie iso/qcow, cloud init injection, runtime config - eg partitioning etc) at runtime: both to reduce time, but more importantly to ensure a consistent 'promoteable' artifact
• Should bake in k8s reqs (ie no konfigadm/package install required during instantiation)
• Should allow additional scripts to be injected during runtime - ie i40e driver install
• Currently bash, but ansible would be better (re declarative comment above)
  -- How would this declarative interface look?
  -- What do we want to declare?
  --- Ansible VARs yaml file?
  --- Two of them:
  ---- Something that constructs root files system
  ----- defines packages
  ----- common config
  ---- Something that defines what to construct (e.g. iso, qcow2, etc)
  ----- would also need to be able to inject custom role for i40e etc.

Complete Move of Image Builder to New image-builder Repository

Proposed change
Upon completion of airshipit/image-builder#1 which sets up the Image Builder code and associated gating in the new image-builder repository, the latest image builder updates need to be pulled into the new repository and the image builder code removed from this images repository so that all future Image Builder work is done in the new repository.

Enhanced Images CI/Validation

Create a Zuul pipeline to provide enhanced CI gating/validation for the Images repo. This would leverage the approach being taken for Open Stack Helm but be enhanced for Airship Images. This may start as a proof of concept and expand from there.

Create a separate qcow image bundle for sub-cluster VMs

We need a separate container to house the sub-cluster qcow bundle to separate it from the control plane qcow bundle.

Push commit tags to quay

Problem description (if applicable)
Currently for images in this repo we are only pushing "latest-*" (moving) tags to quay for each commit. Thus if we need to pin to a specific commit of any of the images in e.g. airshipctl/treasuremap manifests, this is not possible.

Proposed change
As we do with airshipctl/krm functions, we should push commit tags to quay after each commit is merged to this repo.

Create reference implementation to build and push a Helm-Chart Collator image

As an Airship user, I want to have a working example of the tooling needed to build & push a Helm-Chart Collator image for the Airship-Core type in Treasuremap, so that I have a solid foundation upon which to build HCC images for my own implementation needs.

Could leverage https://github.com/airshipit/images/blob/master/helm-chart-collator/examples/charts.yaml as the basis of the image definition.

Github automation to add triage label to new issues

Mirror the functionality in airshipctl repo to add triage label to new issues when they're created. By automating this, it will prevent new issues being missed during Flight Plan.

Multi Distro support for Image Builder

Problem description
Image builder currently only supports Ubuntu

Proposed change
Build on issues airshipit/airshipctl#248 and airshipit/airshipctl#249 to provide support for additional image distributions (e.g. Red Hat). Identify a new distro and path to configure image builder for it.