crossplane-contrib / provider-alibaba Goto Github PK
View Code? Open in Web Editor NEWCrossplane provider for Alibaba Cloud
License: Apache License 2.0
Crossplane provider for Alibaba Cloud
License: Apache License 2.0
We urgently need to use kafka in the project, and hope to support it as soon as possible.
The Alibaba Cloud provider is currently outdated and has not yet transitioned to Upjet. Could we consider migrating the Alibaba provider to Upjet?
This link to https://crossplane.io/docs/master/running-resources.html is missing in README.
@lowkeyrd , from Alibaba internal team, would like to extend crossplane/provider-alibaba
in his own codebase and make some customizations on api structures, controller setup, PublishConnection and so on.
Forking this community provider is not an option as it will take into so many troubles of merging future new releases.
Here are some examples on how the integration would take place.
For example, in OSS managed resource controller, SetupBucket
is as below. There is only one managed.ReconcilerOption
.
// SetupBucket adds a controller that reconciles Bucket.
func SetupBucket(mgr ctrl.Manager, l logging.Logger) error {
options := []managed.ReconcilerOption{managed.WithExternalConnecter(&Connector{
Client: mgr.GetClient(),
Usage: resource.NewProviderConfigUsageTracker(mgr.GetClient(), &aliv1alpha1.ProviderConfigUsage{}),
NewClientFn: ossclient.NewClient,
})}
...
}
In internal provider, there are two more managed.ReconcilerOption
, one of which is to setup account initialization related reconciler option, the other is for managed.WithConnectionPublishers
.
func SetupBucket(mgr ctrl.Manager, l logging.Logger) error {
options := []managed.ReconcilerOption{
managed.WithExternalConnecter(&connector{
baseConnector: crossplaneossctl.Connector{
...
},
xxxAccountClient: ...
}),
managed.WithConnectionPublishers(
...
}
OSS managed resource doesn't need PublishConnection
.
The internal custom provider needed PublishConnection
.
func (p *CloudAccountPublisher) PublishConnection(ctx context.Context, mg resource.Managed, c managed.ConnectionDetails) error {
...
}
func (p *CloudAccountPublisher) UnpublishConnection(ctx context.Context, mg resource.Managed, c managed.ConnectionDetails) error {
...
}
Observe
, Create
, Update
, Delete
logicsBoth in crossplane/provider-alibaba, and internal provier-alibaba, in a managed resource controller, functions Observe
, Create
, Update
, Delete
are the same.
func (e *external) Observe(ctx context.Context, mg resource.Managed) (managed.ExternalObservation, error) {
...
}
func (e *external) Create(ctx context.Context, mg resource.Managed) (managed.ExternalCreation, error) {
...
}
func (e *external) Update(ctx context.Context, mg resource.Managed) (managed.ExternalUpdate, error) {
...
}
func (e *external) Delete(ctx context.Context, mg resource.Managed) error {
...
}
Whether it's possible, in the aspect of Crossplane Runtime, to support more friendly extension.
For example, like supporting not copying Observe
, Create
, Update
and Delete
functions. If the runtime found they are missing, it will redirect to call the Basexxx
one in the same managed resource controller.
Please let me know if I didn't have this feature understood. Thanks.
OSS Bucket resources created via Crossplane can't be deleted via Crossplane if they contain data. We would like this to be possible for cleanup of our temporary test environments.
Add a force-destroy option to the OSS Bucket resource, similar to the Terraform provider argument: force_destroy
Adding some stricter CI linters to help improve the quality of this code base.
Crossplane's provider installation unit recently changed from a Stack
/StackInstall
/ClusterStackInstall
to Package
/PackageInstall
/ClusterPackageInstall
. provider-alibaba
should be updated to use this new terminology.
provider-gcp
: crossplane-contrib/provider-gcp#228I am using the crossplane provider to provision the alibaba resources. But the redis is not added now. Do you have a plan to support redis?
demo is not work.
$ make demo
kubectl apply -f ./config/crd
error: the path "./config/crd" does not exist
make: *** [demo] Error 1
run make demo
Crossplane version:
RDS account has been created and synced successful
status: atProvider: accountReady: true dbInstanceID: pgm-uf67w8778xxxx dbInstanceStatus: Running
While, it keeps sending the create account request
Created RDS or Redis resources which account
spec: forProvider: engine: postgresql engineVersion: "10.0" dbInstanceClass: pg.n2.small.2c dbInstanceStorageInGB: 20 dbInstanceNetType: "Intranet" payType: Postpaid securityIPList: 0.0.0.0/0 masterUsername: xxxxxxxxxxxx networkType: VPC zoneId: cn-shanghai-b vpcId: vpc-xxxxxxxxxxx vSwitchId: vsw-xxxxxxxx
Crossplane version:
ALIYUN ACK
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.8-aliyun.1", GitCommit:"27f24d2", GitTreeState:"", BuildDate:"2021-08-19T10:00:16Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
Alibaba Cloud Resource Orchestration Service (https://www.alibabacloud.com/product/ros) is a fully managed service for cloud computing resource orchestration and automated deployment.
It's based on Alibaba Cloud API.
After compiling a resource stack template, define what cloud resources you need, the dependency between the resources, and the resource configurations. With the orchestration engine, ROS can automatically create and configure all resources for automatic deployment, operation, and maintenance according to your template. An orchestration template is a text file in JSON format that you can read and edit at any time. You can compile your template in JSON directly, or you can use Visual Editor in the ROS console to compile the template more visually. You can control the template version with tools such as SVN or Git. You can also enable IAC (Infrastructure as Code) using API, SDK, and other methods to integrate the orchestration capabilities of ROS into your own applications.
The resource orchestration template also provides a standard delivery method for resources and applications. You can use the template to deliver integrated systems and solutions that include cloud resources and applications. Independent software vendors (ISV) can use such delivery method to easily integrate Alibaba Cloud resources with their own software systems for consistent delivery.
ROS manages cloud resources in groups. A group of resources is a resource stack. Therefore, cloud resources can be created, deleted, modified, and cloned in groups. In DevOps practices, you can easily clone, develop, and test the environments, simplifying the overall migration and scaling of applications.
In short, ROS is similar to Terraform/CloudFormation/Heat.
Here is a discussion on whether we should replace direct API call with ROS.
Here are some concerns from @negz
- There are multiple sources of truth. e.g. Crossplane -> ROS -> presumably APIs being called by ROS.
Indeed, ROS is based on Alibaba Cloud resources API.
- We don’t really want our users to need to know how the underlying system (e.g. ROS, CloudFormation etc) works.
Indeed.
- We want users to work with a strongly typed Kubernetes resource like RDSInstance not ROSDeployment or similar.
ROSDeployment or similar object won't exist as ROS is in the form of API called by managed resource Observe/Create/Update/Delete.
Applying the following RDSInstance yaml.
apiVersion: database.alibaba.crossplane.io/v1alpha1
kind: RDSInstance
metadata:
name: poc
labels:
example: "true"
engine: mysql
spec:
forProvider:
engine: mysql
engineVersion: "8.0"
dbInstanceClass: "rds.mysql.c1.large"
dbInstanceStorageInGB: 20
securityIPList: "0.0.0.0/0"
masterUsername: "oam-dev"
writeConnectionSecretToRef:
namespace: crossplane-system
name: example-rds-2
providerConfigRef:
name: default
deletionPolicy: Delete
Alibaba cloud resource mysql db is successfully created, but the reconciling failed during to the following issue.
➜ /Users/zhouzhengxi/Downloads k get rdsinstance
NAME READY SYNCED STATE ENGINE VERSION AGE
poc True False Running mysql 8.0 4m24s
2021-03-20T17:56:34.331+0800 DEBUG controller-runtime.manager.events Warning {"object": {"kind":"RDSInstance","name":"poc","uid":"6e74fad2-ce0f-4ee6-b394-2367e11e5c39","apiVersion":"database.alibaba.crossplane.io/v1alpha1","resourceVersion":"18463048"}, "reason": "CannotObserveExternalResource", "message": "cannot create RDS database account: SDK.ServerError\nErrorCode: InvalidAccountName.Malformed\nRecommend: https://error-center.aliyun.com/status/search?Keyword=InvalidAccountName.Malformed&source=PopGw\nRequestId: 04A612C0-5A37-4F3D-95C0-2EAD856F9E2A\nMessage: The specified parameter \"AccountName\" is not valid."}
The result between Crossplane provider and Alibaba cloud resource should be consistent.
Apply the template above.
Crossplane version:
v0.12.0
kubectl version
)uname -a
)When executing make reviewable
, an issue happended.
➜ /Users/zhouzhengxi/Programming/golang/src/github.com/crossplane/provider-alibaba git:(master) make reviewable
13:41:56 [ .. ] go generate darwin_amd64
13:42:03 [ OK ] go generate darwin_amd64
13:42:03 [ .. ] cleaning generated CRDs
13:42:04 [ OK ] cleaned generated CRDs
13:42:04 [ .. ] verify dependencies have expected content
all modules verified
13:42:08 [ OK ] go modules dependencies verified
13:42:08 [FAIL] unsupported go version. Please make install one of the following supported version: 1.7|1.8|1.9|1.10|1.11|1.12|1.13|1.14
make[1]: *** [go.init] Error 1
make: *** [lint] Error 2
➜ /Users/zhouzhengxi/Programming/golang/src/github.com/crossplane/provider-alibaba git:(master) go version
go version go1.14.9 darwin/amd64
The go version meets 1.14
.
make reviewable
Crossplane version:
NA
kubectl version
)uname -a
)Contributors have to manually test all supported cloud resources creation/update/deletion in almost every PR, especially, in a large one, like #64.
Add e2e tests to provision, update and delete cloud resources in Alibaba Cloud with real credentials.
I can find provider version 0.5 ,if I want to use the the newest crossplane/provider-alibaba
I can find provider version 0.5 ,if I want to use the the newest crossplane/provider-alibaba ,if I use the Terrajet Generate Crossplane Providers from https://github.com/aliyun/terraform-provider-alicloud ,Can the Generate Crossplane Providers work well?
With more and more cloud resources supported, I suggest we continue the release of new versions.
Followed provider-alibaba README to make demo
for the PoC, I hit an issue.
➜ /Users/zhouzhengxi/Programming/golang/src/github.com/zzxwill/provider-alibaba git:(oss) k describe applicationconfiguration service-tracker-managed
Name: service-tracker-managed
Namespace: default
Labels: <none>
Annotations: API Version: core.oam.dev/v1alpha2
Kind: ApplicationConfiguration
Metadata:
Creation Timestamp: 2020-05-27T08:30:52Z
Generation: 1
Managed Fields:
API Version: core.oam.dev/v1alpha2
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:conditions:
Manager: crossplane
Operation: Update
Time: 2020-05-27T08:30:52Z
API Version: core.oam.dev/v1alpha2
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:components:
Manager: kubectl
Operation: Update
Time: 2020-05-27T08:30:52Z
Resource Version: 8006
Self Link: /apis/core.oam.dev/v1alpha2/namespaces/default/applicationconfigurations/service-tracker-managed
UID: 3e275c18-0148-4a69-b674-db40bf321a82
Spec:
Components:
Component Name: tracker-managed-postgres-db
Parameter Values:
Name: dbconn
Value: trackerdbconn
Component Name: data-api
Parameter Values:
Name: dbsecret
Value: trackerdbconn
Name: dbname
Value: postgres
Name: dbport
Value: 3433
Name: dbdriver
Value: postgres
Name: dboptions
Value:
Component Name: flights-api
Parameter Values:
Name: dataUri
Value: http://data-api.default.svc.cluster.local:3009/
Traits:
Trait:
API Version: core.oam.dev/v1alpha2
Kind: ManualScalerTrait
Metadata:
Name: flights-api
Spec:
Replica Count: 2
Component Name: quakes-api
Parameter Values:
Name: dataUri
Value: http://data-api.default.svc.cluster.local:3009/
Traits:
Trait:
API Version: core.oam.dev/v1alpha2
Kind: ManualScalerTrait
Metadata:
Name: quakes-api
Spec:
Replica Count: 2
Component Name: weather-api
Parameter Values:
Name: dataUri
Value: http://data-api.default.svc.cluster.local:3009/
Traits:
Trait:
API Version: core.oam.dev/v1alpha2
Kind: ManualScalerTrait
Metadata:
Name: weather-api
Spec:
Replica Count: 2
Component Name: service-tracker-ui
Parameter Values:
Name: flightsUri
Value: http://flights-api.default.svc.cluster.local:3003/
Name: weatherUri
Value: http://weather-api.default.svc.cluster.local:3015/
Name: quakesUri
Value: http://quakes-api.default.svc.cluster.local:3012/
Status:
Conditions:
Last Transition Time: 2020-05-27T08:30:52Z
Message: cannot apply components: cannot apply workload "app-postgresql": cannot create object: PostgreSQLInstance.database.crossplane.io "app-postgresql" is invalid: spec.engineVersion: Unsupported value: "10.0": supported values: "9.6", "11"
Reason: Encountered an error during resource reconciliation
Status: False
Type: Synced
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal RenderedComponents 7s (x6 over 2m7s) oam/applicationconfiguration.core.oam.dev Successfully rendered components
Warning CannotApplyComponents 7s (x6 over 2m7s) oam/applicationconfiguration.core.oam.dev cannot apply workload "app-postgresql": cannot create object: PostgreSQLInstance.database.crossplane.io "app-postgresql" is invalid: spec.engineVersion: Unsupported value: "10.0": supported values: "9.6", "11"
Followed provider-alibaba README
Followed README.md
to make a PoC, I accidentally export Alibaba Cloud ACCESS_KEY_ID with wrong value, it hit the issue Managed resource is not yet bindable
, which doesn't sounds useful to me, and it took me a while to locate the root cause. It would be better if it could raise error message from Cloud provider provisioning Cloud resource, like Specified access key is not valid
➜ /Users/zhouzhengxi/Programming/golang/src/github.com/zzxwill/provider-alibaba git:(oss) ✗ k describe PostgreSQLInstance app-postgresql
Name: app-postgresql
Namespace: default
Labels: <none>
Annotations: <none>
API Version: database.crossplane.io/v1alpha1
Kind: PostgreSQLInstance
Metadata:
Creation Timestamp: 2020-06-16T09:34:05Z
Finalizers:
finalizer.resourceclaim.crossplane.io
Generation: 3
Owner References:
API Version: core.oam.dev/v1alpha2
Block Owner Deletion: true
Controller: true
Kind: ApplicationConfiguration
Name: service-tracker-managed
UID: 1332979b-1575-485c-8e2c-9f439691b5ee
Resource Version: 1141943400
Self Link: /apis/database.crossplane.io/v1alpha1/namespaces/default/postgresqlinstances/app-postgresql
UID: 8c7f0fc8-e138-44b9-a6ef-a9dcca56aabf
Spec:
Class Ref:
API Version: database.alibaba.crossplane.io/v1alpha1
Kind: RDSInstanceClass
Name: rdspostgresql-standard
UID: 914d6bed-aab3-4260-a108-a920a822ecda
Class Selector:
Match Labels:
Engine: postgresql
Example: true
Engine Version: 9.4
Resource Ref:
API Version: database.alibaba.crossplane.io/v1alpha1
Kind: RDSInstance
Name: default-app-postgresql-jqkvb
UID: 74c36f49-7cb5-4341-a7d1-deb6b8fe7bfa
Write Connection Secret To Ref:
Name: trackerdbconn
Status:
Conditions:
Last Transition Time: 2020-06-16T09:34:05Z
Reason: Managed claim is waiting for managed resource to become bindable
Status: False
Type: Ready
Last Transition Time: 2020-06-16T09:34:05Z
Reason: Successfully reconciled resource
Status: True
Type: Synced
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SelectedResourceClass 12m claimscheduling/postgresqlinstance.database.crossplane.io Selected matching resource class
Normal CreatedManagedResource 12m claimbinding/postgresqlinstance.database.crossplane.io Successfully created managed resource
Normal WaitingToBind 2m3s (x1972 over 12m) claimbinding/postgresqlinstance.database.crossplane.io Managed resource is not yet bindable
Followed provider-alibaba README and export wrong access-key information.
Crossplane version:
Main repo: https://github.com/crossplane/provider-alibaba
Owner: @hongchaodeng
This is the umbrella issue to track all progress on Alibaba Cloud Infra Stack:
In the demo the scrips ./hack/demo/prepare-alibaba-credentials.sh using the env ALICOUD_ACCESS_KEY_ID/ALICOUD_ACCESS_KEY_SECRET but the default name is ACCESS_KEY_ID/ACCESS_KEY_SECRET in README.md, also when create the secret the namespace crossplane may not be created yet, result in the failure of create the secret.
Crossplane version:
When hitting an issue of creating an external resource, the reconciler interval is so short.
}
Warning CannotCreateExternalResource 4m49s managed/project.sls.alibaba.crossplane.io FailedToCreateSLSProject: {
"httpCode": 400,
"errorCode": "ProjectAlreadyExist",
"errorMessage": "Project sls-project-test already exist",
"requestID": "606B2B449A3FCDEBF04A39AB"
}
Warning CannotCreateExternalResource 4m49s managed/project.sls.alibaba.crossplane.io FailedToCreateSLSProject: {
"httpCode": 400,
"errorCode": "ProjectAlreadyExist",
"errorMessage": "Project sls-project-test already exist",
"requestID": "606B2B44C65D859813230B62"
}
Warning CannotCreateExternalResource 4m48s managed/project.sls.alibaba.crossplane.io FailedToCreateSLSProject: {
"httpCode": 400,
"errorCode": "ProjectAlreadyExist",
"errorMessage": "Project sls-project-test already exist",
"requestID": "606B2B456B5BF43B1BE88971"
}
Warning CannotCreateExternalResource 4m42s (x16 over 4m48s) managed/project.sls.alibaba.crossplane.io (combined from similar events): FailedToCreateSLSProject: {
"httpCode": 400,
"errorCode": "ProjectAlreadyExist",
"errorMessage": "Project sls-project-test already exist",
"requestID": "606B2B4B191163A6A2BA0214"
}
Delete the requestID
field from the error message.
when combine use with oam/kubevela:
Status: running
Workflow:
App Revision: logic-application-sample-1-v1:6595e5303f2de867
Context Backend:
API Version: v1
Kind: ConfigMap
Name: workflow-logic-application-sample-1-context
UID: e1c8975f-bfb0-48f1-b7ce-6efbd920a666
Mode: DAG
Steps:
Id: fhawg14wc7
Name: rds-sample-1
Phase: succeeded
Type: apply-component
Id: oej7toqwvc
Name: slb-sample-1
Phase: succeeded
Type: apply-component
Id: ug9jngn1eg
Name: logic-sample-1
Phase: succeeded
Type: apply-component
Suspend: false
Terminated: false
Events: <none>
kubectl get clb
NAME READY SYNCED AGE
slb-sample-1 True True 104m
After delete kubevela's application, the clb component is not correctly delete:
chenyu.jiang:~$ kubectl get clb
NAME READY SYNCED AGE
slb-sample-1 False False 122m
Then when after delete the finilizer in the clb runtime spec, it may mismatch and delete wrong cloud resource.
Crossplane version:
apiVersion: core.oam.dev/v1beta1
kind: ComponentDefinition
metadata:
name: alibaba-slb
annotations:
definition.oam.dev/description: "slb is the instance for create and watch alicloud slb resource"
spec:
workload:
type: clbs.slb.alibaba.crossplane.io
definition:
apiVersion: slb.alibaba.crossplane.io/v1alpha1
kind: CLB
schematic:
cue:
template: |
output: {
apiVersion: "slb.alibaba.crossplane.io/v1alpha1"
kind: "CLB"
metadata: {
name: context.name
namespace: "default"
}
spec: {
forProvider: {
region: "cn-shanghai"
addressType: "internet"
loadBalancerSpec: "slb.s1.small"
}
writeConnectionSecretToRef: {
namespace: "crossplane-system"
name: parameter.writeConnectionSecretToRefName
}
}
}
parameter: {
appid: string
writeConnectionSecretToRefName: string
}
---
apiVersion: core.oam.dev/v1beta1
kind: ComponentDefinition
metadata:
name: alibaba-rds
annotations:
definition.oam.dev/description: "rds is the instance for create and watch alicloud rds resource"
spec:
workload:
type: rdsinstances.database.alibaba.crossplane.io
definition:
apiVersion: database.alibaba.crossplane.io/v1alpha1
kind: RDSInstance
schematic:
cue:
template: |
output: {
apiVersion: "database.alibaba.crossplane.io/v1alpha1"
kind: "RDSInstance"
metadata: {
name: context.name
}
spec: {
forProvider: {
engine: "mysql"
engineVersion: "8.0"
dbInstanceClass: "rds.mysql.c1.large"
dbInstanceStorageInGB: 20
securityIPList: "0.0.0.0/0"
masterUsername: "test123"
writeConnectionSecretToRef: {
namespace: "crossplane-system"
name: parameter.writeConnectionSecretToRefName
}
providerConfigRef: name: "default"
deletionPolicy: "Delete"
}
}
}
parameter: {
appid: string
writeConnectionSecretToRefName: string
}
---
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: my-sample-1
spec:
components:
- name: rds-sample-1
type: alibaba-rds
properties:
writeConnectionSecretToRefName: example-rds-1
outputs:
- name: dbAddress
valueFrom: output.status.atProvider.dbInstanceID + ".mysql.rds.aliyuncs.com"
- name: dbInstanceID
valueFrom: output.status.atProvider.dbInstanceID
- name: slb-sample-1
type: alibaba-slb
properties:
writeConnectionSecretToRefName: clb-1
outputs:
- name: slb-type
valueFrom: output.spec.forProvider.loadBalancerSpec
- name: slb-ID
valueFrom: output.status.atProvider.loadBalancerID
- name: myserver
type: deplotment
dependsOn:
- rds-sample-1
- slb-sample-1
inputs:
- from: dbAddress
parameterKey: properties.dbAddress
- from: dbInstanceID
parameterKey: properties.dbInstanceID
- from: slb-type
parameterKey: properties.loadbalancerSpec
- from: slb-ID
parameterKey: properties.loadbalancerID
the server can be any deployment. three component are gather by the kubevela application. when delete the application the status of clb CR will be false. and cannot be delete directly using kubectl delete clb xxx
, but the behavior that delete the wrong resource is not easy to find and hard to look up from the logs of kubevela's controller.
supply alibaba cloud database for redis
Make a helm chart for this provider to enable the easy deployment of this controller.
Due to aliyun/aliyun-log-go-sdk#131, currently the parameter shardCount
of an SLS store #77 could be updated.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.