crossplane-contrib/function-cue-archived

Cuelang Integration for Crossplane's Composition Function

Notice

This repo is Deprecated and will be merging with https://github.com/elastic/crossplane-function-cue. It is recommended that if you are using this repo to move to the elastic repository until it is migrated into this repo

function-cue

A cuelang integration for Crossplane's Composition Functions.

This function is based on /play codebase and cmd

Purpose

This function is intended to give a crossplane composition creator the ability to compile and run cue templates within a composition function. The user can target the compilation output to various objects determined by the target passed into the CUEInput.Export.Target

Targeting Patches

See Targeting Objects for info on utilizing the types of targets

Resources (default)
PatchDesired
PatchResources
XR

Expected Function Input

See the kubebuilder generated CRD or the go definition

Example Compositions

See examples folder

Expected Output

The compilation output of the CUEInput.Export.Value must be in YAML or JSON documents, or it will fail parsing.

• Each document produced should be a valid crossplane resource xr or mr
• Each document must have an apiVersion, kind

Ex cueCompile() output schema

#outputSchema: {
	// Name of your resource
	name: string
	// Optional connection details
	// see docs/CONNECTION_DETAILS.md for more details
	connectionDetails?: [...#connectionDetail]
	// Optional readiness checks
	// see docs/READINESS_CHECKS.md for more details
	readinessChecks?: [...#readinessCheck]
	// Resource spec
	resource: {
		// Example
		// apiVersion: "nobu.dev/v1"
		// kind: "Example"
		//
		// etc...
	}
}

json

single

{
  "name": "my-cluster",
  "resource": {
    "apiVersion": "nobu.dev/v1",
    "kind": "XCluster"
  }
}

multiple - single json document per line, -e json.MarshalStream(field) will produce this format

{"name": "my-cluster", "resource": {"apiVersion": "nobu.dev/v1", "kind": "XCluster"}}
{"name": "my-network", "resource": {"apiVersion": "nobu.dev/v1", "kind": "XNetwork"}}

yaml

single

name: "my-cluster"
resource:
  apiVersion: "nobu.dev/v1"
  kind: "XCluster"

multiple documents, separated by ---, -e yaml.MarshalStream(field) will produce this format

name: "my-cluster"
resource:
  apiVersion: "nobu.dev/v1"
  kind: "XCluster"
---
name: "my-network"
resource:
  apiVersion: "nobu.dev/v1"
  kind: "XNetwork"
...

CUE Version supported

v0.7.0

Command Support

export

Runs cue export based off of a cue template provided by the CUEInput.Export.Value field

See the currently supported options

Installing

apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-cue
spec:
  package: mitsuwa/function-cue:v0.1.2

Importable Packages

See libraries

Debugging

Logs are emitted to the Function's pod logs. Look for the Function pod in crossplane-system.

Levels

Info   # default
Debug  # run with --debug flag

function-cue-archived's Issues

Convert Injected EnvironmentConfigs into CUE objects injected into the function run

What problem are you facing?

This issue - https://github.com/crossplane-contrib/function-cue/issues/79 - around injecting multiple fields is quite cumbersome and problematic for users. Allowing the EnvironmentConfigs created on Compositions to be converted into CUE and made available would alleviate this problem.

There may still be a use case for also injecting multiple nested fields as is requested in issue #79 but this seems to require a more native crossplane solution

How could this Function help solve your problem?

multiple nested fields could be better alleviated with the conversion of the Environment Configs into CUE it can be injected into the build through an #Env variable, thus allowing multiple injected variable configurations for environments

This should be fairly straightforward to do with cuecontext

Support #connectionDetail and #readinessCheck per document

What problem are you facing?

Currently the functionality for a user to express #connectionDetails and #readinessChecks is at the root of the cuelang template at just #connectionDetails and #readinessChecks. This is more cumbersome than necessary, because this requires that the user specify a gvk+name to match the readinessCheck/connectionDetail with a specific document resource.

If we allow for these configurations to be expressed per document, this would remove the whole excess of adding the gvk+name targetting twice

How could this Function help solve your problem?

Supporting this functionality

ie - current imlementation

#connectionDetails: [
    {
         Match: apiVersion: "example.org/v1"
         Match: kind: "xkind"
         metadata: name: "somename"
         
         someConnectionDetails...etc...
    }
    ...etc
]

#readinessChecks: [
    {
         Match: apiVersion: "example.org/v1"
         Match: kind: "xkind"
         metadata: name: "somename"
         
         readinessCheck...etc...
    }
    ...etc
]

output: [
    {
        apiVersion: "example.org/v1"
        kind: "xkind"
        metadata: name: "somename"
    }
    ...
]

Desired implementation

output: [
    {
        apiVersion: "example.org/v1"
        kind: "xkind"
        metadata: name: "somename"
        
        #connectionDetail: {
            someConnectionDetails...etc
        }
        
        #readinessCheck: {
            someReadinessCheck...etc
        }
    }
    ...
]

This can be achieved by

• If there is only 1 document - add additional expression for #readinessCheck/#connectionDetail, to check if there is a single configurations within the document
• If there is any amount of nonstream expressions, take those expressions and also add an another with the original $expr.#readinessCheck etc..
• If there is a stream expression, add the additional expression to get the connection details per stream yaml.MarshalStream(list.FlattenN([for e in $expr {e.#readinessCheck}], 2))

Support Readiness Checks and readiness propagation

What problem are you facing?

support readiness checks similar to function patch and transform

Allow a means for users to specify readiness checks for the XR and also propagate readiness from the composed resources

Support non stdlib import packages

What problem are you facing?

If you attempt to import a non stdlib package, for example

import (
	xpv1 "github.com/crossplane/crossplane-runtime/apis/common/v1"
	corev1 "k8s.io/api/core/v1"
)

These packages will not be able to be found because the current module root / has no packages.

This issue is to track adding this support

How could this Function help solve your problem?

Potentially there could be certain set of core packages that can be generated by a function-cue contributor and stored normally in a cue.mod folder via go get + cue get go, this folder could be COPY'd in the dockerfile and then referenced in the load.Config https://github.com/crossplane-contrib/function-cue/blob/main/cue.go#L70. However, it would be nice if there was a way to have the user specify this, I suppose one could build their own dockerfile that reference's FROM this dockerfile and adds its own cue.mod additions

It may also be the case that instead of loading all of the MRs provider schemas from configmaps, that instead this repo gets the MR schemas stored into the cue mod file and loaded that way

Migrate to cue.Context()

What problem are you facing?

this issue is for tracking migrating this function to use cue.Context() instead of the current implementation of cueCompile

https://pkg.go.dev/cuelang.org/go/[email protected]#Context

How could this Function help solve your problem?

This will allow the base Encode() and the Compile to function off of cue's core apis that are being built currently

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update module google.golang.org/protobuf to v1.33.0 [security]
• fix(deps): update k8s.io/utils digest to 4693a02
• fix(deps): update module github.com/crossplane/crossplane-runtime to v1.15.1
• chore(deps): update dependency go to v1.22.2
• fix(deps): update kubernetes packages to v0.30.0 (k8s.io/api, k8s.io/apimachinery)
• fix(deps): update module cuelang.org/go to v0.8.1
• fix(deps): update module github.com/alecthomas/kong to v0.9.0
• fix(deps): update module github.com/stretchr/testify to v1.9.0
• fix(deps): update module sigs.k8s.io/controller-tools to v0.15.0
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update golang docker tag to v1.22

Detected dependencies

dockerfile

Dockerfile

• docker/dockerfile 1
• golang 1

tools/cuemod-generator/Dockerfile

• golang 1.20

github-actions

.github/workflows/ci.yml

• actions/checkout v4
• actions/setup-go v5
• golangci/golangci-lint-action v4
• actions/checkout v4
• actions/setup-go v5
• docker/setup-qemu-action v3
• docker/setup-buildx-action v3
• actions/checkout v4
• docker/build-push-action v5
• actions/upload-artifact v4
• actions/checkout v4
• actions/download-artifact v4
• docker/login-action v3
• ubuntu 22.04
• ubuntu 22.04
• ubuntu 22.04
• ubuntu 22.04

gomod

go.mod

• go 1.21
• go 1.21.3
• cuelang.org/go v0.7.0
• github.com/alecthomas/kong v0.8.1
• github.com/crossplane/crossplane-runtime v1.15.0-rc.0.0.20231215091746-d23a82b3a2f5
• github.com/crossplane/function-sdk-go v0.2.0
• github.com/ghodss/yaml v1.0.0
• github.com/google/go-cmp v0.6.0
• github.com/stretchr/testify v1.8.4
• google.golang.org/protobuf v1.32.0
• k8s.io/api v0.29.2
• k8s.io/apimachinery v0.29.2
• k8s.io/utils v0.0.0-20240102154912-e7106e64919e@e7106e64919e
• sigs.k8s.io/controller-tools v0.14.0

---

• Check this box to trigger a request for Renovate to run again on this repository

How can I inject several fields at once from the observed resource into the managed resource?

How can I inject several fields at once from the observed resource into the managed resource?

In my example, I would like to transfer everything under spec.test to the managed resource.

apiVersion: nopexample.org/v1
kind: XSubnetwork
metadata:
  name: test-xrender
spec:
  test: 
    key1: "hello"
    key2: "dummy string"
    karray:
      - v1 
      - v2 
    key3: 5

---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: xnopresources.nop.example.org
spec:
  compositeTypeRef:
    apiVersion: nop.example.org/v1alpha1
    kind: XNopResource
  mode: Pipeline
  pipeline:
  - step: conditional
    functionRef:
      name: function-cue
    input:
      apiVersion: cue.fn.crossplane.io/v1beta1
      kind: CUEInput
      metadata:
        name: patch-xr
      export:
        options:
          inject:
          - name: test
            path: spec.test
        target: XR
        value: |
          #env: string @tag("test")

          name: "XRPatching"
          resource: {
            spec: test: #env
          }

crossplane beta render xr.yaml composition.yaml function.yaml

When executing - using the function-cue - I get the following resource.

---
apiVersion: nopexample.org/v1
kind: XSubnetwork
metadata:
  name: test-xrender
spec:
  test: map[karray:[v1 v2] key1:hello key2:dummy string key3:%!s(float64=5)]

How can I iterate over the string "test" in Cuelang so that I get the desired output?

---
apiVersion: nopexample.org/v1
kind: XSubnetwork
metadata:
  name: test-xrender
spec:
  test: 
    key1: "hello"
    key2: "dummy string"
    karray:
      - v1 
      - v2 
    key3: 5

Identifiers do not work

What happened?

See the current test that is commented out

#deployment: [ID=_]: {
	apiVersion: "apps/v1"
	kind:       "Deployment"
	metadata: name: ID
	spec: {
		replicas: *1 | int
		template: {
			metadata: labels: {
				app:       ID
				domain:    "prod"
				component: string
			}
			spec: containers: [{name: ID}]
		}
	}
}

#deployment: echoserver: spec: template: {
	metadata: annotations: {
		"prometheus.io.scrape": "true"
		"prometheus.io.port":   "7080"
	}
	metadata: labels: {
		"component": "core"
	}
}
#deployment.echoserver

should produce

{
    "apiVersion": "apps/v1",
    "kind": "Deployment",
    "metadata": {
        "name": "echoserver"
    },
    "spec": {
        "replicas": 1,
        "template": {
            "metadata": {
                "annotations": {
                    "prometheus.io.scrape": "true",
                    "prometheus.io.port": "7080"
                },
                "labels": {
                    "app": "echoserver",
                    "domain": "prod",
                    "component": "core"
                }
            },
            "spec": {
                "containers": [
                    {
                        "name": "echoserver"
                    }
                ]
            }
        }
    }
}

How can we reproduce it?

Run a test with this uncommented and you will see the failure

        + 				"failed compiling cue template: failed to encode: spec.template.m",
        + 				"etadata.labels.app: incomplete value string (and 2 more errors)",

there is probably some configuration that is not being set correctly in encConfig or the loadConfig. Start basing more of this implementations off of the cue/cmd/export.go intead of /play

Would `Export` make sense as the input kind?

What problem are you facing?

Looking at the function a little more, I wonder if this would make sense:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
# Removed for Brevity
spec:
  # Removed for Brevity
  mode: Pipeline
  pipeline:
  - step: cue-export-resources
    functionRef:
      name: function-cue
    input:
      apiVersion: cue.fn.crossplane.io/v1beta1
      kind: Export
      export:
        target: Resources
        value: |
          apiVersion: "s3.aws.upbound.io/v1beta1"
          kind: "Bucket"
          spec: forProvider: {
          	region: "us-east-2"
          }

The idea being that the "input" to the CUE function is the export configuration. It seems like the primary field folks are working with is export. This would match function-patch-and-transform where you mostly work with the resources input field, and the kind of the input is Resources in the pt.fn.crossplane.io API group.

This is really just a minor optimization so I could be convinced it's not worth the breaking change.

Push to xpkg.upbound.io?

What problem are you facing?

We now have a nascent CI setup for functions. See:

• https://github.com/crossplane/function-template-go/blob/main/.github/workflows/ci.yml
• https://github.com/crossplane/function-template-go/blob/main/Dockerfile

If you're interested, I can open a PR to add these and create xpkg.upbound.io/crossplane-contrib/function-cue as a repo to push this to.

Use the `cue.fn.crossplane.io` API group?

What problem are you facing?

      apiVersion: template.fn.crossplane.io/v1beta1
      kind: CUEInput

I notice the input uses the template.fn.crossplane.io API group. My intention was that folks would s/template/something-more-specific-to-their-function/.

How could this Function help solve your problem?

What do you think about using cue.fn.crossplane.io instead? Not sure what kind would be appropriate then though. We could just use Input I suppose.