googlecloudplatform/terraform-provider-cdap

Custom Terraform Provider for CDAP

Terraform CDAP Provider

This custom provider for Terraform can be used to manage a CDAP API (exposed for example by a GCP Cloud Data Fusion Instance) in an infra-as-code manner.

This is a community maintained provider and not an official Google or Hashicorp product.

GCP Data Fusion specific helpers and modules can be found in the corresponding Cloud Foundation Toolkit repo.

Documentation

• Website: https://registry.terraform.io/providers/GoogleCloudPlatform/cdap/
• Blog post: https://cloud.google.com/blog/products/data-analytics/open-source-etl-pipeline-tool

Contributing

See CONTRIBUTING.md

Development

To build a local version of the provider, run go build -o ${test_dir} where test_dir is the path to a directory hosting test Terraform configs.

Releasing

Automated releases are handled by Github Actions.

1. Choose a version. It should match the regex ^v[0-9]+\.[0-9]+\.[0-9]+$. That is, a leading "v", followed by three period-separated numbers.

   version="v0.1.0"

2. Create the Git tag.

   For binaries:

   git tag -a "${version}" -m "${version}"

3. Push the tag:

   git push origin --tags

terraform-provider-cdap's Issues

Support for Application Properties

If your application uses Runtime Arguments and Macros to make it reusable between environments, then when you go to Runtime Arguments in the Data Fusion webapp, these are actually stored as properties for the application.

I'm looking to implement this functionality, but I'm unsure which design is better. I see two big alternatives:

1. Because the properties of the application are fundamentally about the application, we add them as an attribute to the resource, and when we perform the create, we also do a second call to set the properties, if appropriate.
2. Because the properties are a separate web request, we could do them as a separate resource. This could be a resource specific to preferences of applications, or we could allow the user to specify path the properties are for much like

PUT /v3/preferences/
PUT /v3/namespaces/<namespace-id>/preferences
PUT /v3/namespaces/<namespace-id>/apps/<app-id>/preferences
PUT /v3/namespaces/<namespace-id>/apps/<app-id>/<program-type>/<program-id>/preferences

Rather than using paths, we could use these as optional arguments, and depending on which ones are set, we decide which path to use.

While 2 is more extensible over all, it's definitely more complex. I wouldn't personally benefit from the complexity, and I'm not sure anyone else would, so my preference would be to add properties as an attribute of the application.

gitignore ignores docs templates.

Right now the gitignore file has a generate_docs
I believe the intention is to ignore the generate_docs executable.
However there is a side effect of ignoring the directory tools/generate_docs
We should add:

!tools/**

Investigate deploying plugins directly from the hub

https://docs.cdap.io/cdap/current/en/integrations/hub.html

Potentially a new resource "hub_artifact"

The main problem here is an artifact is multiple resources in the API (a JAR resource + properties). If one of the resources fails how do we recover from that and do we consider the entire resource broken?

Terraform has concept of "partial state". We could leverage that here.

Make token optional

When running a local cdap instance you don't need a token.

Support updates for profiles

Error 409 leads to a blocked state

Hey there,
Mike Turk here. I was running an apply to install 6 plugins. I got a couple of 502 errors. I did the apply again but then got "Error 409 (Conflict)!!1". At this point, trying the apply only returns the 409 error. I looked in the UI but did not see the plugin installed. So i seemed to find myself in a case where it seems like something is already installed but doesn't appear to be and i can't find a way to recover. I end up deleting the CDF instance.

Improve error handling

Right now we're mainly using return err blocks everywhere. We should instead add additional context to the errors to make them easier to understand.

Add support for compute profiles

https://docs.cdap.io/cdap/current/en/reference-manual/http-restful-api/profile.html

Error on terraform init when added module terraform-google-modules/data-fusion/google//modules/dataproc_profile

Code used.

  required_version = ">= 0.13"  
  required_providers {
    cdap = {
      source = "GoogleCloudPlatform/cdap"
      version = "0.9.0"
    }
    google = {
      source = "hashicorp/google"
      version = "3.82.0"
    }
  }
}


data "google_client_config" "current" {}

provider "cdap" {
  host  = "${module.project_dev_dcipoc_01_data_fusion.service_endpoint}/api"
  token = data.google_client_config.current.access_token
}

module "custom_dataproc" {
  source  = "terraform-google-modules/data-fusion/google//modules/dataproc_profile"
  version = "0.1"

  name   = "custom-dataproc"
  label  = "Custom Dataproc Profile"

Error:

Initializing modules...
Downloading terraform-google-modules/data-fusion/google 0.1.0 for custom_dataproc...
- custom_dataproc in .terraform/modules/custom_dataproc/modules/dataproc_profile

Initializing the backend...

Initializing provider plugins...
- Using previously-installed googlecloudplatform/cdap v0.9.0
- Using previously-installed hashicorp/google v3.82.0
- Using previously-installed hashicorp/random v3.1.0
- Using previously-installed hashicorp/google-beta v3.81.0
- Finding latest version of hashicorp/cdap...

Error: Failed to install provider

Error while installing hashicorp/cdap: provider registry registry.terraform.io
does not have a provider named registry.terraform.io/hashicorp/cdap

404s when using cdap provider right after CDF instance deployment

We should add polling for the system artifact rather than assuming the cdap REST API is immediately available.

provider publishing on registry.terraform.io

Hey guys,
We are trying to use the provider with terraform 0.13 and the workaround to make it work is quite ugly. Could you please post it to the registry?
Thanks!

"host" misnomer in provider

the provider takes an input "host"
however, it expects scheme, host and path
e.g.
"https://example-instance-example-project-dot-usc1.datafusion.googleusercontent.com/api/"

In my mind this should be named "service_endpoint" (consistent w/ data fusion rest api) or "rest_endpoint" (more general for for OSS / non data fusion CDAP users)

Unfortunately this is an interface breaking change.
Fortunately I think we are the only users of this provider so far.

Migrate from legacy plugin sdk

This provider uses a legacy plugin sdk yielding warnings like:

[WARN] Provider "registry.terraform.io/-/cdap" produced an invalid plan for cdap_streaming_program_run.test, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .name: planned value cty.StringVal("DataStreamsSparkStreaming") does not match config value cty.NullVal(cty.String)
      - .allow_multiple_runs: planned value cty.False does not match config value cty.NullVal(cty.Bool)
2020/03/04 15:38:20 [INFO] backend/local: apply calling Plan

We should consider migrating to a new SDK with the automated tool described at: https://www.terraform.io/docs/extend/plugin-sdk.html

Users seeing 504s

I personally got sporadic 502s / 504s today from CDF today.
Worked on running terraform apply again.
I think we should treat CDF as a flaky server and retry on all 5xx codes and add a few retries to protect users of this provider from this flakiness.
Mike Turk mentioned 502s and 504s on his instance yesterday.

Dataproc profile

Error: 404: <title>Error 404 (Not Found)!!1</title> <style> {margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px} > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px} </style>

404. That’s an error.

That’s all we know.
with cdap_profile.test
on main.tf line 24, in resource "cdap_profile" "test":resource "cdap_profile" "test" {

CONTRIBUTING should explain how to run generate docs tool.

Help for dataproc_profile installation

I follow the provided example, but I'm still getting the error. Could you help me to solve this one? Thanks in advanced.
Terraform v0.13.5

data "google_client_config" "current" {}

provider "cdap" {
  host  = "${module.project_dev_dcipoc_01_data_fusion.service_endpoint}/api"
  token = data.google_client_config.current.access_token
}

module "custom_dataproc" {
  source  = "terraform-google-modules/data-fusion/google//modules/dataproc_profile"
  version = "~> 0.1"

  name   = "custom-dataproc"
  label  = "Custom Dataproc Profile"

  network         = "example-network"
  subnet          = "example-subnetwork"
  service_account = "example-sa"
  gcs_bucket      = "example-bucket"
}

The error is:

Initializing modules...

Initializing the backend...


Initializing provider plugins...
- Using previously-installed hashicorp/google-beta v3.70.0
- Using previously-installed googlecloudplatform/cdap v0.9.0
- Using previously-installed hashicorp/google v3.70.0
- Using previously-installed hashicorp/random v3.1.0
- Finding latest version of hashicorp/cdap...

Error: Failed to install provider

Error while installing hashicorp/cdap: provider registry registry.terraform.io
does not have a provider named registry.terraform.io/hashicorp/cdap

Exist check returns error when namespace no longer exists

In the case of the CDF instance being recreated all CDAP resources within that instance should signal that they no longer exists.

Problem is all resources within the namespace start returning 404 errors since the namespace no longer exists.

We need to check for namespace existence first.

Error Message Misleading

When using jsonencode(file()) in the main.tf file of a module, an error saying "Unable to marshal type map(string)" is thrown. However, this issue here is that jsonencode() isn't needed as file() can parse through a JSON file. Is it possible to change the error message to point to the JSON, as the "unable to marshal type.." indicates that is an issue with the Go code?

Profile destroy disables profile instead of deleting it

The current profile destroy call seems to disable the profile rather than delete it.

Get "/v3/namespaces": unsupported protocol scheme ""

Hello,

I've deployed a data fusion instance (with a private network, namespace and so) but now every time I launch a plan or an apply I get the following error :

Error: Get "/v3/namespaces": unsupported protocol scheme ""

I can provide logs and/or everything that can help.

Thanks,

Simplify installing with terraform init

Let's explore exposing this provider through the terraform registry so it can be more seemlessly installed for users with terraform init.

Return clearer message when CDF instance is unreachable

The current error is not very helpful:

Error: Get /v3/namespaces: unsupported protocol scheme ""

We should call a general health check endpoint in the provider init.

Deleting cdap_namespace fails

Problem

Executing terraform destroy on a cdap_namespace resource fails with the following error:

Error: 405:

<title>Error 405 (Method Not Allowed)!!1</title> <style> *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px} </style>

405. That’s an error.

That’s all we know.

Version

Terraform v0.12.24

• provider.cdap v0.6.0
• provider.google-beta v3.22.0

Minimum Code To Reproduce

Apply the following terraform script:

provider "google-beta" {
  credentials = "<credentials>"
  project     = "<project-id>"
  region      = "us-west1"
}

resource "google_data_fusion_instance" "instance" {
  provider = google-beta
  name     = "example"
  region   = "us-west1"
  type     = "BASIC"
}

data "google_client_config" "current" {
  provider = google-beta
}

provider "cdap" {
  host  = "${google_data_fusion_instance.instance.service_endpoint}/api/"
  token = data.google_client_config.current.access_token
}

resource "cdap_namespace" "dev" {
  provider = cdap
  name     = "dev"
}

Try to destroy everything or just the individual resource:

terraform destroy -target cdap_namespace.dev

Allow the entire exported JSON config to be used as-is

Support updates for the primary artifact version

The only fields supported for updates in applications are artifact versions. We should support updating these fields in the provider.

CDAP endpoint no longer available if Data Fusion instance updated.

So consider the scenario where I have a Data Fusion in version 6.4.1 and I wish to re-deploy it as 6.5.0 version via Terraform. In Terraform, this implies simply changing the attribute version attribute. After making this change, when performing a terraform plan the following error is obtained:

| ERROR Get "/v3/namespaces" : unsupported protocol scheme""

To solve this, I am forced to remove all cdap ressources which produce the error using the terraform state rm command. Only after this can Terraform update the instance and re-create all the CDAP resources that have to be re-created due to the fact that the Data Fusion instance needs to be destroyed and recreated.

I believe that this error comes from the fact that there is a dependency between a Data Fusion instance and all its CDAP resources that is not considered by the CDAP provider. One should never have to resort to editing the tfstate file to solve these types of issues. Is there a way to fix this?

Error dialing Google

Problem

Executing terraform plan or terraform apply results in the following error:

Error: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
on main.tf line 18, in provider "cdap":
18: provider "cdap" {

In the example code below, removing the cdap resource prevents the script from failing and builds the data fusion instance so there is no issue with the google provider configuration.

Version

Terraform v0.12.24

• provider.cdap v0.5.0
• provider.google-beta v3.22.0

Minimum Code To Reproduce

provider "google-beta" {
  credentials = "<credentials>"
  project     = "<project-id>"
  region      = "us-west1"
}

resource "google_data_fusion_instance" "instance" {
  provider = google-beta
  name     = "example"
  region   = "us-west1"
  type     = "BASIC"
}

data "google_client_config" "current" {
  provider = google-beta
}

provider "cdap" {
  host  = "${google_data_fusion_instance.instance.service_endpoint}/api/"
  token = data.google_client_config.current.access_token
}

resource "cdap_namespace" "dev3" {
  provider = cdap
  name     = "dev"
}

Add CDAP streaming program resrouce

This resource will be used in tandem with the application resource as a means of starting a streaming pipeline.

this resource should:

• Use the program endpoint of resource of the Lifecycle API datafusion public docs example
• On apply / destroy behave similarly to the google_dataflow_job resource
• Have a runtime_args input that will POST-ed as the payload and should support arbitrary number of keys.

Add CI to check for gofmt, lint and docs

Automate checking for common things to ensure PRs are conformant.

CONTRIBUTING should explain who to build / test locally

It would be good to detail good practices for building the provider binary and (manually) integration testing your new features w/ terraform locally. Similarly for go tests once added.

It might be good to consider a Makefile to standardize some of these things e.g. specify build target into ~/terraform.d/plugins/terraform-provider-cdap.

Add support for specifying compute profile for a pipeline

The cdap application resource should support a "profile" field to specify the name of the compute profile to use when starting this pipeline.

https://docs.cask.co/cdap/5.1.2/en/developer-manual/cloud-runtimes/profiles/assigning-profiles.html#overriding-profile-configuration

Add Support for dataproc cluster property overrides

We should expose an interface for users to set dataproc cluster property overrides

CDAP Dataproc Provisioner seems to have undocumented support for (only in REST API not UI):
https://github.com/cdapio/cdap/blob/1d62163faaecb5b888f4bccd0fcf4a8d27bbd549/cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocConf.java#L382-L386

Changing instance's version triggers forces replacement

Hi all,

Summary

I'm using google_data_fusion_instance resource and created a data fusion instance with version 6.7.2.

When I change the version to 6.8.0 in Terraform and apply the changes, the Data Fusion instance is destroyed and replaced

What I expect

Terraform to update in-place the instance without destroying it, because updating the version from Google UI updates the instance

What is actually happening

Terraform destroy the instance, then create an instance with the new version

What is your setup

• Terraform 1.2.7
• googlecloudplatform/cdap v0.10.0
• hashicorp/google v4.50.0

Code used

Child module

resource "google_data_fusion_instance" "instance" {

  name                          = var.name
  project                       = var.project
  description                   = var.description
  region                        = var.region
  type                          = var.type
  enable_stackdriver_logging    = var.enable_stackdriver_logging
  enable_stackdriver_monitoring = var.enable_stackdriver_monitoring
  labels                        = var.labels
  version                       = var.datafusion_version
  options                       = var.options
  private_instance              = var.private_instance

  dynamic "network_config" {
    for_each = var.network_config == null ? [] : [var.network_config]
    content {
      network       = var.network_config.network
      ip_allocation = var.network_config.ip_allocation
    }
  }
} 

Root module

module "data_fusion" {
  source = "../modules/instance/"

  name                     = "instance-test"
  project                  = local.project_id
  region                   = local.region
  dataproc_service_account = data.google_app_engine_default_service_account.default.email
  datafusion_version = "6.8.0"
  network_config = {
    network       = local.network
    ip_allocation = local.ip_allocation
  }
}

Terraform plan output

  # module.data_fusion.google_data_fusion_instance.instance must be replaced
-/+ resource "google_data_fusion_instance" "instance" {
      ~ api_endpoint                  = "https://instance-test-xxx-xxx-dot-xxx.datafusion.googleusercontent.com/api" -> (known after apply)
      ~ create_time                   = "2023-01-25T03:17:04.119017648Z" -> (known after apply)
      - enable_rbac                   = false -> null
      ~ gcs_bucket                    = "gs://df-1xxx25169-zxn6xx4l4xxxbbqaaaa" -> (known after apply)
      ~ id                            = "projects/xxxx/locations/europe-west1/instances/instance-test" -> (known after apply)
      - labels                        = {} -> null
        name                          = "instance-test"
      - options                       = {} -> null
      ~ p4_service_account            = "[email protected]" -> (known after apply)
      ~ service_endpoint              = "https://instance-test-xxxx.datafusion.googleusercontent.com" -> (known after apply)
      ~ state                         = "ACTIVE" -> (known after apply)
      + state_message                 = (known after apply)
      ~ tenant_project_id             = "zbb26cd6b97964399-tp" -> (known after apply)
      ~ update_time                   = "2023-01-25T03:31:11.307380027Z" -> (known after apply)
      ~ version                       = "6.7.2" -> "6.8.0" # forces replacement
      + zone                          = (known after apply)
        # (6 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Support System Profiles

Right now, the cdap_profile resource requires a namespace, and omitting it uses the "default" namespace, so it is currently impossible to configure System Compute Profiles.

Set run_id once as a computed field in create for streaming program run

Right now the resource is fetching all runs and looking for the faux id field. Instead, we can just do this once when the create func is called to get the run and and store that. After that the faux id is not needed anymore.

This will also make it easy to migrate when the API starts returning the run id.

Release prebuilt binaries

It will be great if we could release pre-built binaries, building a binary locally requires setting up the Go toolchain which is not trivial for some users.

Improve docs

User should be able to read docs and understand what resources are implemented, what fields are exposed (or NOT exposed) and special behavior (when an application resource is added does it start the pipeline?)

We might be able to autogenerate some of this? https://github.com/segmentio/terraform-docs

Installing docs missing step to make binary executable

https://googlecloudplatform.github.io/terraform-provider-cdap/
should prompt user to
chmod +x ~/.terraform.d/plugins/terraform-provider-cdap
or similar

Add tests, check for gofmt, lint and docs

We should add unit tests as well as integration tests for the provider.

Print endpoint that failed when get non 200 code

When the provider fails due to 4xx / 5xx code, we don't know which API call failed, as just the error code / message is printed.
We should add the url that returned this response code to the error message.

Add support for Namespaces / Preferences

This will be helpful in helping set preferences within a namespace
(e.g. In this namespace which HL7v2 store should pipelines read from).
Naturally this helps ease the promotion of a pipeline from dev to prod.
In the pipeline JSON the source URI for a plugin should point to a preference (rather than a hard coded reference).
https://docs.cdap.io/cdap/current/en/reference-manual/http-restful-api/namespace.html
https://docs.cdap.io/cdap/current/en/reference-manual/http-restful-api/preferences.html

cdap_streaming_program_run error 404

We are facing an error while running the streaming pipeline in datafusion instance.
Terraform is not able to recognise the app name.

Code :

data "google_client_config" "current" {}

terraform {
required_providers {
cdap = {
source = "GoogleCloudPlatform/cdap"
version = "0.9.0"
}
}
}

provider "cdap" {
#.
host = "https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.googleusercontent.com/api/"
token = data.google_client_config.current.access_token
}

resource "cdap_streaming_program_run" "test" {
app = "testpoc" ( the app name is referred from deployed pipeline name)
type = "spark"

runtime_arguments = {
"system.profile.name" = "SYSTEM:dataproc"
}
}

-------error------------

Error: 404:
│
│
│
│
│ <title>Error 404 (Not Found)!!1</title>
│ <style>
│ {margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px} > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
│ </style>
│
│

404. That’s an error.
│

That’s all we know.
│
│
│ with cdap_streaming_program_run.test,
│ on main.tf line 18, in resource "cdap_streaming_program_run" "test":
│ 18: resource "cdap_streaming_program_run" "test" {

---

Configuring network tags in `cdap_profile` for CDAP application instance

We are trying to configure the Network Tags used by the ephemeral Dataproc clusters created by Cloud Data Fusion, in order to set up correct firewall configurations.

Manually, this can be configured via System Admin -> Configuration -> System Compute Profiles -> [create new] -> Cluster Metadata -> Network Tags or in a deployed pipeline, from Configure -> Compute Config -> (select profile) -> Customize -> Cluster Metadata -> Network Tags.

I understand the CDF provider for Terraform does have the Dataproc Profile module which also accepts arbitrary Dataproc Cluster Properties. However, the network tags are not configurable via cluster properties, but rather the --tags argument.

Is there a workaround for achieving this? Note that the configuration pertains to ephemeral clusters created by Cloud Data Fusion during its lifetime, so the name/ID of the target GCE VMs is unknown beforehand and it has to be configured in the Dataproc Profile.

Artifact upload timeout too short

When uploading an artifact with a maximum size of 160 MB, I am getting a timeout in my terraform job with the following error:

Error: Post "https://<CDAP_endpoint>/api/v3/namespaces/<namespace>/artifacts/<artifact_anem>": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

This errors is being obtained for all namespaces and for all artifacts. Can this be solved by increasing the default timeout of the Go HTTP client in the resource_local_artifact.go file?

destroy fails on profile if any pipelines still have that profile assigned.

Error: 409 Conflict: Profile 'adp-aintegration-hl7-stream-ingest-us-central1-n' is still assigned to application 'HL7v2_to_fhir' in namespace 'adp_staging'. Please delete all assignments before deleting the profile.

v0.4.0 provider name

Hey there. did the name of the 0.4.0 binary get accidentally changed? The words cdap and provider are switched.
terraform-cdap-provider_v0.4.0_linux-amd64
terraform-provider-cdap_v0.3.0_linux-amd64
Noticed it because i got an error when i first tried to use it... but renaming it fixes it.
Just wanted to mention it.

Support updates for the primary application artifact version

Currently the resource just takes an entire application config as a string. We should instead implement the entire spec in the provider which follows Terraform standards more closely.