crossplane-contrib/function-kcl

Crossplane Composition Functions using KCL Programming Language

function-kcl's Issues

How to modify composite resource connection details from kcl function?

I am developing a composite using KCL function code to create resources needed for an Azure workload identity, namely a UserAssignedIdentity, a FederatedIdentityCredential and a K8 Object for a service account. The PrincipalId of the UserAssignedIdentity is needed by other composites to facilitate the creation of RoleAssignment resources for their own access needs. So I need to surface the PrincipalId property from the composite somehow. NOTE: RoleAssignment only accepts a PrincipalId and has no selector option.

The problem is UserAssignedIdentity does not surface any information in connection details so writeConnectionSecretToRef produces a secret with no data, and of course this then provides no data to propagate up to the composite itself.

The status.atProvider data surfaced by the UserAssignedIdentity in the observed ocds dictionary does contain all the data needed of course but I'm not sure how to surface this to the composite. Theoretically I could write my own secret with the required data but was wondering if there was a way of still using a composite's connectiondetails so the secret is written by crossplane. Any ideas?

Ability to decouple `metadata.name` from `crossplane.io/composition-resource-name` and `crossplane.io/external-name` annotations

What problem are you facing?

Currently, in this function the metadata.name is a single entry point and identifier that is getting automatically propagated to crossplane.io/composition-resource-name and crossplane.io/external-name annotations.

That is convenient but frequently undesired behavior.

The situations where it is required for them to have different values:

• I want to have the same metada.name across different resource kinds. Currently if I try to do it with function-kcl the resources with the same metada.name are getting overrides by the last one, because the same crossplane.io/composition-resource-name is getting set automatically.
  Example:

            providerConfigTypes = ["helm", "kubernetes"]
            providerConfigs = [{
              apiVersion = "{}.crossplane.io/v1alpha1".format(t)
              kind = "ProviderConfig"
              metadata.name = id
              metadata.annotations = {
                  "krm.kcl.dev/ready": "True"
              }
              spec.credentials = {
                  secretRef = {
                    name = "{}-ekscluster".format(uid)
                    namespace = connectionSecretNamespace
                    key = "kubeconfig"
                  }
                  source = "Secret"
                }
            } for t in providerConfigTypes]

Only last kubernetes ProviderConfig is getting rendered in this case

apiVersion: kubernetes.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
  annotations:
    crossplane.io/composition-resource-name: configuration-aws-eks

because the function is setting identical crossplane.io/composition-resource-name annotation

• The case when we need to set crossplane.io/external-name different from metadata.name due to MR identifier specifics. It is requirement when the external resource identifier is not compliant with RFC 1123.
  It frequently affects resources that has dots and other characters as part of their identifier like a domainname
  https://marketplace.upbound.io/providers/upbound/provider-aws-apigatewayv2/v1.3.1/resources/apigatewayv2.aws.upbound.io/DomainName/v1beta1

apiVersion: apigatewayv2.aws.upbound.io/v1beta1
kind: DomainName
metadata:
  annotations:
    crossplane.io/external-name: example-email.upbound-providers.io
  name: example
spec:
  forProvider:
...

In this case metada.name != crossplane.io/external-name and it is mandatory.
It is relatively frequent case when we need to override external-name.

How could this Function help solve your problem?

Respect when user sets custom values like

metadata.annotations = {
    "crossplane.io/composition-resource-name" = "custom-composition-name"
    "crossplane.io/external-name" = "custom-external-name"
}

Currently they are getting overridden by metadata.name value and it breaks the desired behavior

DatabaseInstance wrong value type

What happened?

While trying to create / update a DatabaseInstance.sql.gcp.upbound.io with database flags e.g.

items = [
            {
              apiVersion: "sql.gcp.upbound.io/v1beta1"
              kind: "DatabaseInstance"
              spec: {
                forProvider: {
                  project: "test-project"
                  settings: [
                    {
                      databaseFlags: [
                        {
                          name: "log_checkpoints"
                          value: "on"
                        }
                      ]
                    }
                  ]
                }
              }
            }
        ]

The update / create fails with:

defined/compositeresourcedefinition.apiextensions.crossplane.io  cannot compose resources: cannot apply composed resource "alir-test-crossplane-db-instance": failed to create typed patch object (/alir-test-crossplane-db-instance; sql.gcp.upbound.io/v1beta1, Kind=DatabaseInstance): errors:
  .spec.forProvider.settings[0].databaseFlags[0].value: expected string, got &value.valueUnstructured{Value:true}

This does not happen without function-kcl.
I saw a similar issue reported here maybe it is related: crossplane-contrib/provider-upjet-aws#1261

What environment did it happen in?

Function version: latest
Function revision: kcl-function-011644505e8f
Cloud: GCP
Crossplane version: 1.15.2

Add additional package metadata

What happened?

Now that this function-kcl has been published to the marketplace at https://marketplace.upbound.io/functions/crossplane-contrib/function-kcl, we should improve the metadata of this package to provide a more rich experience with more details.

How can we reproduce it?

Navigate to https://marketplace.upbound.io/functions/crossplane-contrib/function-kcl, and see:

• Overview says "No overview provided."
• Name is kcl-function, even though the repo is function-kcl.
• Source code field is missing
• maybe others too?
• note that an icon is missing, but the marketplace will be adding that on the backend, so don't worry about that

We can compare this to another function like https://marketplace.upbound.io/functions/crossplane-contrib/function-auto-ready, which has more of these details defined. As an example, the source of its metadata can be found in https://github.com/crossplane-contrib/function-auto-ready/blob/main/package/crossplane.yaml.

What environment did it happen in?

Function version: v0.2.0

Function unable to run intermittently when multiple claims applied to cluster

What happened?

When applying multiple claims to a cluster using a single manifest file (using "---" separator between claims) Crossplane initially reports the following error when describing most of the composites:

"Message: cannot compose resources: cannot run Composition pipeline step "normal": cannot run Function "kcl-function": rpc error: code = DeadlineExceeded desc = context deadline exceeded"

Eventually all composites are reconciled correctly but it would appear like the function cannot handle concurrent execution perhaps? This has the effect of degrading the performance of reconciling many resources at once.

NOTE: Applying each claim separately and waiting for a result works without the error occurring.

How can we reproduce it?

Create a composite and apply several claims in one go. Observe each of the composites to see the error message.

What environment did it happen in?

Function version: v0.4.0
Crossplane: v1.15.0
Client Version: v1.29.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.0

[Enhancement] Support annotation to set the plain http for the crossplane kcl function

What problem are you facing?

OCI Source is managed by function kcl and should be able to be set through the fields of KCLRun, in theory something like this perhaps.

    functionRef:
      name: kcl-function
    input:
      apiVersion: krm.kcl.dev/v1alpha1
      kind: KCLRun
      metadata:
        name: basic
        annotations:
          kcl-something/allow-insecure-source: True
      spec:
        target: Resources
        source: oci://localhost:7900/my-composition-kcl:0.0.1

How could this Function help solve your problem?

Support non-SSL OCI registry for composition local development

What problem are you facing?

Attempting to use crossplane beta render on windows to render out a kcl package from a local docker registry:2 that has no ssl enabled.

I've tried using a relative and absolute file path to my module's main.k file but I'm always informed of "no such file or directory" so I have no idea of the required path format (for windows) or what the file path is relative to. That would be a separate issue to this.

If I use oci:// scheme, the kcl function expects to pull the package using https even though I have explicitly set "DefaultOciPlainHttp":true in my local kpm.json file.

I can push my package to the registry using kcl mod push oci://localhost:7900/my-composition-kcl no problem.

If in the KCLRun I use a source oci://host.docker.internal:7900/my-composition-kcl:0.0.1 I get the following error reported back from the crossplane beta render:

crossplane: error: cannot render composite resource: pipeline step "normal" returned a fatal result: failed to run kcl function pipelines: failed to select latest version from 'host.docker.internal:7900/my-composition-kcl:0.0.1'

Get "https://host.docker.internal:7900/v2/my-composition-kcl/tags/list": http: server gave HTTP response to HTTPS client

Could we allow the KCLRun to use the local oci registry settings, specifically DefaultOciPlainHttp to use http instead of https for download?

How could this Function help solve your problem?

Allow me to continue to use a non-SSL local registry

Support 'Function pipeline context'

What problem are you facing?

It would be really helpful if the function supported function pipeline context.

This function-environment-configs writes the EnvironmentConfig in the context which is not usable in this function atm.

I would like to use EnvironmentConfigs to have a common config per 'environment'.

[Enhancement] setting composition resource status

What happened?

I want to provide values to custom composition resource status fields, similar to how patch and transform do it using ToCompositeFieldPath. I am defining extra properties on my CompositeResourceDefinition's status property and attempting to set the property value using the below code:

_dxr = option("params").dxr
_dxr.status.myExtraProp = "sample"

However, the function fails to compile complaining of an UndefinedType:

    Message:               cannot compose resources: pipeline step "normal" returned a fatal result: failed to run kcl function pipelines: failed to compile the kcl package
EvaluationError
failed to update the dict. An iterable of key-value pairs was expected, but got UndefinedType. Check if the syntax for updating the dictionary with the attribute 'myExtraProp' is correct

How can we reproduce it?

Define a custom status property called myprop in xrd and attempt to set its value on dxr.status.myprop.

What environment did it happen in?

Function version: xpkg.upbound.io/crossplane-contrib/function-kcl:v0.3.4

How do you patch the XR status?

I've got a pipeline that creates resources, but I need to bubble status back up to the XR.

Is there an example of how to accomplish basically an ToCompositeFieldPath patch back to the XR using this function or with some other function?

The patch and transform function needs to know the name of the resource and that's not really deterministic.

Example KCL to pass resource data between created cloud resources

I'm writing KCL code to create Azure resources using the KCL crossplane function as part of a composition. I need to supply some of one resource's "atProvider" data to another resource's "forProvider" data.

In the past this would be achieved by using a patch with the patch and transform function but I would prefer to keep this encapsulated within the KCL code if possible.

Should I use the option("params").ocds some how to access the named resource and obtain its atProvider data? If so, can you provide an example of accessing a a resource this way. Please also let me know if this is not the preferred way of doing this.

published version doesn't work

Hey 👋

Thanks for the interesting function!

What happened?

Looks like there are no published function, which I can try? Not sure if that's docker image problem or my local machine setup. And I'm not sure how to list versions available.

So I'm trying to run examples/resources/loop with functions.yaml changed to:

apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: kcl-function
spec:
  package: kcllang/crossplane-kcl

which IIUC should run function in docker image, pulled from somewhere? 🤔

but all I get is:

➜  loop git:(main) make run
crossplane beta render xr.yaml composition.yaml functions.yaml -r
crossplane: error: cannot render composite resource: cannot start Function "kcl-function": cannot start Docker container: Error response from daemon: unable to find user nonroot: no matching entries in passwd file

I'm running on arm64 macos with docker desktop

Thanks!

`"kcl" returned a fatal result: failed to run kcl function pipelines: Permission denied (os error 13)` with `v0.5.2` release

What happened?

Status:
  Conditions:
    Last Transition Time:  2024-04-16T16:50:32Z
    Message:               cannot compose resources: pipeline step "kcl" returned a fatal result: failed to run kcl function pipelines: Permission denied (os error 13)
    Reason:                ReconcileError
    Status:                False
    Type:                  Synced
  Warning  ComposeResources   7s (x5 over 17s)  defined/compositeresourcedefinition.apiextensions.crossplane.io  cannot compose resources: pipeline step "kcl" returned a fatal result: failed to run kcl function pipelines: Permission denied (os error 13)

k -n upbound-system logs -f deploy/crossplane-contrib-function-kcl-b6e7ced47cd4
2024/04/16 16:44:51 kclvm.go:41: [WARN] install kclvm failed: mkdir /go: permission denied
2024/04/16 16:44:51 kclvm.go:53: [WARN] install kclvm failed: open /go/init.lock: no such file or directory
2024/04/16 16:44:51 kclvm.go:58: [WARN] install kclvm failed: mkdir /go: permission denied
{"level":"info","ts":1713285921.3011003,"caller":"fn/fn.go:32","msg":"Running Function","tag":""}
{"level":"info","ts":1713285921.402543,"caller":"fn/fn.go:32","msg":"Running Function","tag":""}

How can we reproduce it?

Run basic kcl example e2e with XR and Composition (not crossplane beta render but the full XR instantiation )

What environment did it happen in?

Function version: xpkg.upbound.io/crossplane-contrib/function-kcl:v0.5.2

The function works as expected with the previous release of v0.5.1

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update github artifact actions to v4 (major) (actions/download-artifact, actions/upload-artifact)
• chore(deps): update module oras.land/oras-go to v2
• fix(deps): update module gopkg.in/yaml.v2 to v3

Detected dependencies

dockerfile

Dockerfile

• golang 1

github-actions

.github/workflows/ci.yaml

• actions/checkout v4
• actions/setup-go v5
• docker/setup-qemu-action v3
• docker/setup-buildx-action v3
• actions/checkout v4
• docker/build-push-action v5
• actions/upload-artifact v3
• actions/checkout v4
• actions/download-artifact v3
• docker/login-action v3
• ubuntu 22.04
• ubuntu 22.04
• ubuntu 22.04

gomod

go.mod

• go 1.22.2
• dario.cat/mergo v1.0.0
• github.com/alecthomas/kong v0.9.0
• github.com/crossplane/crossplane-runtime v1.15.1
• github.com/crossplane/function-sdk-go v0.2.0
• github.com/google/go-cmp v0.6.0
• github.com/pkg/errors v0.9.1
• google.golang.org/protobuf v1.33.0
• gopkg.in/yaml.v2 v2.4.0
• k8s.io/apimachinery v0.30.0
• kcl-lang.io/krm-kcl v0.8.5
• sigs.k8s.io/controller-tools v0.14.0
• sigs.k8s.io/yaml v1.4.0
• oras.land/oras-go v1.2.5
• oras.land/oras-go/v2 v2.5.0

---

• Check this box to trigger a request for Renovate to run again on this repository

[Enhancement] Add Webhook to verify the correctness of KCL code and pull code from OCI in advance for the composite resources