vfarcic/crossplane-tutorial

Crossplane tutorial

Video tutorial series

Full playlist: https://youtube.com/playlist?list=PLyicRj904Z99i8U5JaNW5X3AyBvfQz-16&si=f_mVV9GH6j5uhzaf

Part 1

• Video: https://youtu.be/bBpE0rfE-JM?si=o-1r9U_60mbPlOp5
• Gist: https://gist.github.com/vfarcic/732bf76feb51489add89567433019460

Part 2

• Video: https://youtu.be/o53_7vuWjw4?si=D7adBi7b-SFLoNmu
• Gist: https://gist.github.com/vfarcic/aa5ecfa315608d1257ba56df18088f2f

Part 3

• Video: https://youtu.be/X7E6YfXWgvE?si=04-MKY0pqCQIrBJl
• Gist: https://gist.github.com/vfarcic/08162d1f3f4954c1f420fae59704b629

Part 4

• Video: https://youtu.be/ompdPvaHn0U?si=A_7GAZTRdFNN01A8
• Gist: https://gist.github.com/vfarcic/3f4f9bf05c937b9f12e6bcb43f3c0bc7

Part 5

• Video: https://www.youtube.com/watch?v=XSzKs97Ls4g
• Gist: https://gist.github.com/vfarcic/9ae7abe89ebab199aa6e52a364e01edc

Resources

• Crossplane: https://crossplane.io
• Say Goodbye to Containers - Ephemeral Environments with Nix Shell: https://youtu.be/0ulldVwZiKA?si=lCQM-bwrWcKfSvqv

The Book

• Amazon: https://www.amazon.com/dp/B0CWCYP5CJ
• LeanPub: https://leanpub.com/crossplane

crossplane-tutorial's Issues

ControllerConfig outdated

replace with current implementation

Issue with azure-v1-sql example

the file azure-sql-v1.yaml has the wrong name for the secret I think my-db-20240114235140-password.
Considering the composition in sql-v1/azure.yaml is referencing my-db-password name.

Files published unintentionally

Hi Master

Maybe I should say 'Hi Main', bad joke sorry.

Even though they are listed in your gitignore, you have published the kubeconfig.yaml and gke_gcloud_auth_plugin_cache.

Is that on purpose ?

Looking forward for the matching video.

AWS - Compositions - Cannot find version 15.3 for postgres

@vfarcic Great work with the tutorials!
I'm currently working through them and adapting where required for specific enterprise constraints.

We encounter an issue under Crossplane Compositions | Tutorial (Part 3) when creating the AWS RDS database ( See [ 1 ] )
It appears Postgres version 15.3 has now dropped off.

crossplane-tutorial/compositions/sql-v1/aws.yaml needs updating to 15.4 or higher.
After this version bump, RDS instance comes up fine 👍

  - name: rdsinstance
    base:
      apiVersion: rds.aws.upbound.io/v1beta1
      kind: Instance
      spec:
        forProvider:
          region: us-east-1
          dbSubnetGroupNameSelector:
            matchLabels:
              crossplane.io/composite: my-db
          vpcSecurityGroupIdSelector:
            matchLabels:
              crossplane.io/composite: my-db
          instanceClass: db.m5.large
          username: masteruser
          engine: postgres
          engineVersion: "15.4" # Bumping version. InvalidParameterCombination: Cannot find version 15.3 for postgres
          skipFinalSnapshot: true
          publiclyAccessible: true
          allocatedStorage: 200
          passwordSecretRef:
            namespace: crossplane-system
            key: password
            name: my-db-password

You can quicky get the supported engine versions

aws rds describe-db-engine-versions --engine postgres |  jq -r '.DBEngineVersions[].EngineVersion'
11.22
12.15
12.16
12.17
12.18
13.11
13.12
13.13
13.14
14.9
14.10
14.11
**15.4**
15.5
15.6
16.1
16.2

[ 1 ]

crossplane beta trace sql my-db -o wide
NAME                                       RESOURCE                    SYNCED   READY   STATUS
SQL/my-db                                                              True     False   Creating: Unready resources: rdsinstance
├─ VPC/my-db-dxzld                         vpc                         True     True    Available
├─ Subnet/my-db-z6zlj                      subnet-a                    True     True    Available
├─ Subnet/my-db-vplhb                      subnet-b                    True     True    Available
├─ Subnet/my-db-lrpx5                      subnet-c                    True     True    Available
├─ SubnetGroup/my-db-gxwnd                 subnetgroup                 True     True    Available
├─ InternetGateway/my-db-cxrmx             gateway                     True     True    Available
├─ RouteTable/my-db-bm28k                  routeTable                  True     True    Available
├─ Route/my-db-nzjrc                       route                       True     True    Available
├─ MainRouteTableAssociation/my-db-8sc84   mainRouteTableAssociation   True     True    Available
├─ RouteTableAssociation/my-db-rr46p       routeTableAssociation1a     True     True    Available
├─ RouteTableAssociation/my-db-lzmmk       routeTableAssociation1b     True     True    Available
├─ RouteTableAssociation/my-db-wbhnr       routeTableAssociation1c     True     True    Available
├─ SecurityGroup/my-db-gc4sl               securityGroup               True     True    Available
├─ SecurityGroupRule/my-db-vcqff           securityGroupRule           True     True    Available
└─ Instance/my-db-gv8xl                    rdsinstance                 True     False   Creating

Status:
  At Provider:
  Conditions:
    Last Transition Time:  2024-05-01T10:58:22Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced
    Last Transition Time:  2024-05-01T10:58:22Z
    Reason:                Creating
    Status:                False
    Type:                  Ready
    Last Transition Time:  2024-05-01T11:18:56Z
    Message:               async create failed: failed to create the resource: [{0 creating RDS DB Instance (terraform-20240501111853069300000052): InvalidParameterCombination: Cannot find version 15.3 for postgres
                           status code: 400, request id: 6ebff90e-f75e-4f57-859d-3d0bd9837ef6  []}]
    Reason:                AsyncCreateFailure
    Status:                False
    Type:                  LastAsyncOperation
Events:
  Type     Reason                           Age                 From                                               Message
  ----     ------                           ----                ----                                               -------
  Warning  CannotResolveResourceReferences  28m (x5 over 29m)   managed/rds.aws.upbound.io/v1beta2, kind=instance  cannot resolve references: mg.Spec.ForProvider.VPCSecurityGroupIds: referenced field was empty (referenced resource may not yet be ready)
  Normal   PendingExternalResource          23m (x23 over 27m)  managed/rds.aws.upbound.io/v1beta2, kind=instance  Waiting for external resource existence to be confirmed
  Normal   CreatedExternalResource          2m (x33 over 28m)   managed/rds.aws.upbound.io/v1beta2, kind=instance  Successfully requested creation of external resource

GH CLI using git/ssh

ArgoCD got a SSH_AUTH_SOCCK error after running your setup/00-intro.sh script. By manually changing to https (from git@) instead, everything started to work. I narrowed this down to me using GH CLI with git/ssh protocol. And when I do, this line REPO_URL=$(git config --get remote.origin.url) gives me [email protected].....

I made a workaround for myself changing:
REPO_URL=$(git config --get remote.origin.url)

to

REPO_URL_RAW=$(git config --get remote.origin.url)
REPO_URL=$(echo $REPO_URL_RAW | sed 's/git@/https:\/\//') # replace git@ to https://

A tiny bit tacky, but happy to make a PR with the change. I started with an issue, in case there are better ways of doing this with argocd and ssh/git protocol?

ReconcileError in Intro chapter

Thank Viktor for this amazing tutorial.

I just encountered a ReconcileError in Intro chapter. I followed all steps, but when I want to confirm if claims are resolved and provisioned, the errors came out.

$ kubectl get managed 
NAME                                                    KIND         PROVIDERCONFIG   SYNCED   READY   AGE
object.kubernetes.crossplane.io/silly-demo-deployment   Deployment   cluster-01       False            16m
object.kubernetes.crossplane.io/silly-demo-ingress      Ingress      cluster-01       False            16m
object.kubernetes.crossplane.io/silly-demo-secret       Secret       cluster-01       False            16m
object.kubernetes.crossplane.io/silly-demo-service      Service      cluster-01       False            16m

$ crossplane beta trace clusterclaim cluster-01 --namespace a-team
NAME                                   SYNCED   READY   STATUS
ClusterClaim/cluster-01 (a-team)       True     False   Waiting: Claim is waiting for composite resource to become Ready
└─ CompositeCluster/cluster-01-kt9h7   False    -       ReconcileError: cannot select Composition: no compatible Compositions found

$ crossplane beta trace sqlclaim silly-demo-db --namespace a-team
NAME                              SYNCED   READY   STATUS
SQLClaim/silly-demo-db (a-team)   True     False   Waiting: Claim is waiting for composite resource to become Ready
└─ SQL/silly-demo-db-gqr2q        False    -       ReconcileError: cannot select Composition: no compatible Compositions found

Gist Part 3 + Video continuity issue

@vfarcic

Found small continuity issue within Gist Part 3 related video

I believe we should use sql-v2 rather than sql-v1 with the change from matchLabels to use matchControllerRef confirmation, otherwise we are just applying the same composition as before...

#####################################
# Resource References and Selectors #
#####################################

cat examples/$HYPERSCALER-vm.yaml

cat compositions/sql-v1/$HYPERSCALER.yaml

cat compositions/sql-v2/$HYPERSCALER.yaml

# kubectl apply --filename compositions/sql-v1 # <-- Should be sql-v2
kubectl apply --filename compositions/sql-v2

kubectl delete --filename examples/$HYPERSCALER-sql-v1.yaml # <-- This is correct

kubectl get managed

script typo

./setup/01-managed-resources.sh: line 122: f: command not found

typo in

crossplane-tutorial/setup/01-managed-resources.sh

Line 122 in 258840a

f

.

00-intro.sh open: command not found - running from nix shell

./setup/00-intro.sh: line 94: open: command not found

Replacing open with xdg-open fixes the problem. Maybe there is an alias missing?

Argocd helm chart setting global.domain needs overriding

https://github.com/argoproj/argo-helm/blob/main/charts/argo-cd/values.yaml#L42

global.domain is set to argocd.example.com, this needs overriding to argocd.127.0.0.1.nip.io for the ingress to work

awscli2 currently not working in nix-shell (unstable)

There is an error when trying to use awscli2 from nix-shell and the default unstable channel. Switching to the latest release channel did not solve the problem either (3 or 4 of the other packages does not exist in 23.10). Pinning version would probably solve this, but my knowledge in Nix is a bit too limited as of now.

Should be solved very soon, I hope from this PR: NixOS/nixpkgs#296007 (which is already merged)

Status here: https://nixpk.gs/pr-tracker.html?pr=296007

As soon as the status for nixpkgs-unstable goes green, this should not be a problem anymore.

Workaround:

• Comment out/remove awscli2 from shell.nix
• Install AWS CLI directly from Amazon (if you use AWS): https://aws.amazon.com/cli/