hashicorp/terraform-provider-scaffolding-framework

Quick start repository for creating a Terraform provider using terraform-plugin-framework

Terraform Provider Scaffolding (Terraform Plugin Framework)

This template repository is built on the Terraform Plugin Framework. The template repository built on the Terraform Plugin SDK can be found at terraform-provider-scaffolding. See Which SDK Should I Use? in the Terraform documentation for additional information.

This repository is a template for a Terraform provider. It is intended as a starting point for creating Terraform providers, containing:

• A resource and a data source (internal/provider/),
• Examples (examples/) and generated documentation (docs/),
• Miscellaneous meta files.

These files contain boilerplate code that you will need to edit to create your own Terraform provider. Tutorials for creating Terraform providers can be found on the HashiCorp Developer platform. Terraform Plugin Framework specific guides are titled accordingly.

Please see the GitHub template repository documentation for how to create a new repository from this template on GitHub.

Once you've written your provider, you'll want to publish it on the Terraform Registry so that others can use it.

Requirements

• Terraform >= 1.0
• Go >= 1.21

Building The Provider

1. Clone the repository
2. Enter the repository directory
3. Build the provider using the Go install command:

go install

Adding Dependencies

This provider uses Go modules. Please see the Go documentation for the most up to date information about using Go modules.

To add a new dependency github.com/author/dependency to your Terraform provider:

go get github.com/author/dependency
go mod tidy

Then commit the changes to go.mod and go.sum.

Using the provider

Fill this in for each provider

Developing the Provider

If you wish to work on the provider, you'll first need Go installed on your machine (see Requirements above).

To compile the provider, run go install. This will build the provider and put the provider binary in the $GOPATH/bin directory.

To generate or update documentation, run go generate.

In order to run the full suite of Acceptance tests, run make testacc.

Note: Acceptance tests create real resources, and often cost money to run.

make testacc

terraform-provider-scaffolding-framework's Issues

Update to a `.goreleaser.yml` valid for > v0

I am having a bit of a headache recently updating the .goreleaser.yml config for Hashistack plugins to be compliant with v2, or even v1, as this is being enforced by the recent versions of the goreleaser Github Action and goreleaser module. I am working on trying to validate it with v1.25.1 as my digging around revealed this was the last version supporting Go 1.21, but it would be super helpful for the repository to be updated with an example that validates for > v0 goreleaser (other than locking to versions from a long time ago). I also assume that of all the examples this would be the best place to inquire as e.g. another Hashi repo still contains --rm-dist in the GHA config file for the goreleaser args.

Thanks for any updates.

Update: My personally modified version of the example is working for v1.25.1 goreleaser and 1.21 go for a packer plugin, and so that explains why the conflicts were recent, and helps inform that this is probably more of a >= 1.26 and 1.22 update.

no required module provides package github.com/hashicorp/terraform-exec/tfinstall

go generate
../../go/pkg/mod/github.com/hashicorp/[email protected]/internal/provider/generate.go:14:2: no required module provides package github.com/hashicorp/terraform-exec/tfinstall; to add it:
        go get github.com/hashicorp/terraform-exec/tfinstall
main.go:19: running "go": exit status 1

Create link/CTA in framework documentation that points to this repository

Seems like it could live right next to the Learn tutorial on the overview page: https://www.terraform.io/plugin/framework

Documentation: Plugin module addressing

I had substantial difficulty figuring out how terraform locates plugin and understanding "Address" in ServeOpts works in plugin discovery and initialization.

The relationship between the top-level project (e.g. github project name), the ServeOpts address, and the go modules were unclear in the documentation and tutorial. For example when building the project, if the last plugin name is "abc", then terraform seems to look for: terraform-provider-abc in the go bin directory (apparently requiring the GO project to be called terraform-provider-abc) and the last segment of the Address needs to be "abc".

The server opts address value in main.go, .terraformrc, and project name need to "align" (but not equal).

This was difficult because the current hashicups example tutorial does not use the current plugin framework scaffolding and uses a different plugin start procedure avoiding the ServeOpts Address value.

Hashicups:

func main() {
	tfsdk.Serve(context.Background(), hashicups.New, tfsdk.ServeOpts{
		Name: "hashicups",
	})
}

Plugin Scaffolding Framework:

func main() {
	var debug bool

	flag.BoolVar(&debug, "debug", false, "set to true to run the provider with support for debuggers like delve")
	flag.Parse()

	opts := providerserver.ServeOpts{
		// TODO: Update this string with the published name of your provider.
		Address: "registry.terraform.io/hashicorp/scaffolding",
		Debug:   debug,
	}

	err := providerserver.Serve(context.Background(), provider.New(version), opts)

	if err != nil {
		log.Fatal(err.Error())
	}
}

convertProviderType: incorrect message

Hello

the convertProviderType function is intended to check that the provider in argument satisfies the correct interface is identical to the type *scaffoldingProvider:

p, ok := in.(*scaffoldingProvider)

and then displays an error if it isn't the case:

fmt.Sprintf("While creating the data source or resource, an unexpected provider type (%T) was received. 
This is always a bug in the provider code and should be reported to the provider developers.", p),

however if the check fails the value of p is the zero value for the type *scaffoldingProvider, see: https://go.dev/ref/spec#Type_assertions

As a result fmt.Sprintf should print the type of in, not p

mismatch between struct and object: Object defines fields not found in struct

I follow the example_rsource.go and get mismatch between struct and object: Object defines fields not found in struct: cluster_id when I try to convert config with Computed=true to resourcedata. I don't know how to avoid this. can anyone help?

the code likes

package provider

import (
	"context"
	"fmt"
	"github.com/hashicorp/terraform-plugin-framework/diag"
	"github.com/hashicorp/terraform-plugin-framework/path"
	"github.com/hashicorp/terraform-plugin-framework/tfsdk"
	"github.com/hashicorp/terraform-plugin-framework/types"
	"github.com/hashicorp/terraform-plugin-log/tflog"
	"github.com/hashicorp/terraform-provider-scaffolding-framework/tidbcloud"
)

// Ensure provider defined types fully satisfy framework interfaces
var _ tfsdk.ResourceType = clusterResourceType{}
var _ tfsdk.Resource = clusterResource{}
var _ tfsdk.ResourceWithImportState = clusterResource{}

type clusterResourceType struct{}

func (t clusterResourceType) GetSchema(ctx context.Context) (tfsdk.Schema, diag.Diagnostics) {
	return tfsdk.Schema{
		// This description is used by the documentation generator and the language server.
		MarkdownDescription: "Cluster resource",

		Attributes: map[string]tfsdk.Attribute{
			"project_id": {
				MarkdownDescription: "The ID of the project",
				Required:            true,
				Type:                types.StringType,
			},
			"name": {
				MarkdownDescription: "The name of the cluster",
				Required:            true,
				Type:                types.StringType,
			},
			"cluster_id": {
				MarkdownDescription: "The ID of the cluster",
				Computed:            true,
				Type:                types.StringType,
				PlanModifiers: tfsdk.AttributePlanModifiers{
					tfsdk.UseStateForUnknown(),
				},
			},
			"cluster_type": {
				MarkdownDescription: "DEVELOPER: create a Developer Tier cluster \n DEDICATED:create a Dedicated Tier cluster",
				Required:            true,
				Type:                types.StringType,
			},
			"cloud_provider": {
				MarkdownDescription: "AWS: the Amazon Web Services cloud provider \n GCP:the Google Cloud Platform cloud provider",
				Required:            true,
				Type:                types.StringType,
			},
			"region": {
				MarkdownDescription: "the region value should match the cloud provider's region code",
				Required:            true,
				Type:                types.StringType,
			},
			"config": {
				MarkdownDescription: "The configuration of the cluster",
				Required:            true,
				Attributes: tfsdk.SingleNestedAttributes(map[string]tfsdk.Attribute{
					"root_password": {
						Required: true,
						Type:     types.StringType,
					},
					"port": {
						Optional: true,
						Type:     types.Int64Type,
					},
					"components": {
						Optional: true,
						Attributes: tfsdk.SingleNestedAttributes(map[string]tfsdk.Attribute{
							"tidb": {
								Required: true,
								Attributes: tfsdk.SingleNestedAttributes(map[string]tfsdk.Attribute{
									"node_size": {
										Required: true,
										Type:     types.StringType,
									},
									"node_quantity": {
										Required: true,
										Type:     types.Int64Type,
									},
								}),
							},
							"tikv": {
								Required: true,
								Attributes: tfsdk.SingleNestedAttributes(map[string]tfsdk.Attribute{
									"node_size": {
										Required: true,
										Type:     types.StringType,
									},
									"storage_size_gib": {
										Required: true,
										Type:     types.Int64Type,
									},
									"node_quantity": {
										Required: true,
										Type:     types.Int64Type,
									},
								}),
							},
							"tiflash": {
								Optional: true,
								Attributes: tfsdk.SingleNestedAttributes(map[string]tfsdk.Attribute{
									"node_size": {
										Required: true,
										Type:     types.StringType,
									},
									"storage_size_gib": {
										Required: true,
										Type:     types.Int64Type,
									},
									"node_quantity": {
										Required: true,
										Type:     types.Int64Type,
									},
								}),
							},
						}),
					},
					"ip_access_list": {
						Optional: true,
						Attributes: tfsdk.ListNestedAttributes(map[string]tfsdk.Attribute{
							"cidr": {
								Required: true,
								Type:     types.StringType,
							},
							"description": {
								Optional: true,
								Type:     types.StringType,
							},
						}),
					},
				}),
			},
		},
	}, nil
}

func (t clusterResourceType) NewResource(ctx context.Context, in tfsdk.Provider) (tfsdk.Resource, diag.Diagnostics) {
	provider, diags := convertProviderType(in)

	return clusterResource{
		provider: provider,
	}, diags
}

type clusterResourceData struct {
	clusterId     types.String  `tfsdk:"cluster_id"`
	ProjectId     types.String  `tfsdk:"project_id"`
	Name          types.String  `tfsdk:"name"`
	ClusterType   types.String  `tfsdk:"cluster_type"`
	CloudProvider types.String  `tfsdk:"cloud_provider"`
	Region        types.String  `tfsdk:"region"`
	Config        ClusterConfig `tfsdk:"config"`
}

type ClusterConfig struct {
	RootPassword string     `tfsdk:"root_password"`
	Port         int32      `tfsdk:"port"`
	Components   Components `tfsdk:"components"`
	IPAccessList []IPAccess `tfsdk:"ip_access_list"`
}

type Components struct {
	TiDB    ComponentTiDB    `tfsdk:"tidb"`
	TiKV    ComponentTiKV    `tfsdk:"tikv"`
	TiFlash ComponentTiFlash `tfsdk:"tiflash"`
}

type ComponentTiDB struct {
	NodeSize     string `tfsdk:"node_size"`
	NodeQuantity int    `tfsdk:"node_quantity"`
}

type ComponentTiKV struct {
	NodeSize       string `tfsdk:"node_size"`
	StorageSizeGib int    `tfsdk:"storage_size_gib"`
	NodeQuantity   int    `tfsdk:"node_quantity"`
}

type ComponentTiFlash struct {
	NodeSize       string `tfsdk:"node_size"`
	StorageSizeGib int32  `tfsdk:"storage_size_gib"`
	NodeQuantity   int32  `tfsdk:"node_quantity"`
}

type IPAccess struct {
	CIDR        string `tfsdk:"cidr"`
	Description string `tfsdk:"description"`
}

type clusterResource struct {
	provider provider
}

func (r clusterResource) Create(ctx context.Context, req tfsdk.CreateResourceRequest, resp *tfsdk.CreateResourceResponse) {
	if !r.provider.configured {
		resp.Diagnostics.AddError(
			"Provider not configured",
			"The provider hasn't been configured before apply, likely because it depends on an unknown value from another resource. This leads to weird stuff happening, so we'd prefer if you didn't do that. Thanks!",
		)
		return
	}

	var data clusterResourceData

	diags := req.Config.Get(ctx, &data)
	resp.Diagnostics.Append(diags...)

	if resp.Diagnostics.HasError() {
		return
	}

	// write logs using the tflog package
	// see https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log/tflog
	// for more information
	tflog.Trace(ctx, "created a resource")

	// If applicable, this is a great opportunity to initialize any necessary
	// provider client data and make a call using it.

	createClusterResp, err := r.provider.client.CreateCluster(data.ProjectId.Value, convertToCreateClusterReq(data))
	if err != nil {
		resp.Diagnostics.AddError("Client Error", fmt.Sprintf("Unable to create cluster, got error: %s", err))
		return
	}

	// save into the Terraform state.
	data.clusterId = types.String{Value: fmt.Sprintf("%s", createClusterResp.ClusterID)}
	diags = resp.State.Set(ctx, &data)
	resp.Diagnostics.Append(diags...)
}

the main.tf likes

resource "tidbcloud_cluster" "cluster1" {
  project_id     = "1369847559691457392"
  name           = "cluster1"
  cluster_type   = "DEVELOPER"
  cloud_provider = "AWS"
  region         = "us-east-1"
  config = {
    root_password = "Shiyuhang1."
    port          = 4000
    components = {
      tidb = {
        node_size : "8C16G"
        node_quantity : 2
      }
      tikv = {
        node_size : "8C16G"
        storage_size_gib : 1024,
        node_quantity : 3
      }
      tiflash = {
        node_size : "8C16G"
        storage_size_gib : 1024,
        node_quantity : 4
      }
    }
    ip_access_list = [{
      cidr        = "0.0.0.0/0"
      description = "all"
      }
    ]
  }
}

Resource "Create" should read from plan, not config?

I could be mistaken on this, but the documentation and hashicups all recommend reading from the plan for create.

However, this scaffolding is configured to read from config:

terraform-provider-scaffolding-framework/internal/provider/example_resource.go

Line 58 in 194f73f

diags := req.Config.Get(ctx, &data)

edit: As an aside, I'm not super clear on when you would actually read directly from config outside of a data source.

Bump Go Version to 1.18

Description

Following the Go support policy and given the ecosystem availability of the latest Go minor version, it's time to upgrade. This will ensure that this project can use recent improvements to the Go runtime, standard library functionality, and continue to receive security updates.

Proposal

• Run the following commands to upgrade the Go module files and automatically fix outdated Go code:

go mod edit -go=1.18
go mod tidy
go fix ./...

• Ensure any GitHub Actions workflows (.github/workflows/*.yml) use 1.19 in place of any 1.18 and 1.18 in place of any 1.17 or earlier
• Ensure the README or any Contributing documentation notes the Go 1.18 expected minimum
• (Not applicable to all projects) Ensure the .go-version is at least 1.18 or later

References

• https://github.com/hashicorp/terraform-plugin-framework/#go-compatibility
• https://formulae.brew.sh/formula/go
• microsoft/winget-pkgs#68523
• https://community.chocolatey.org/packages/golang
• https://packages.ubuntu.com/kinetic/golang-1.19

Add debug flag

Reference: hashicorp/terraform-plugin-framework#239

When terraform-plugin-framework supports tfsdk.Debug() and the dependency is updated to that release, update the main function of this scaffolding provider to implement a -debug flag, similar to terraform-provider-scaffolding. For example:

func main() {
	var debug bool
	var err error

	flag.BoolVar(&debug, "debug", false, "set to true to run the provider with support for debuggers like delve")
	flag.Parse()

	ctx := context.Background()
	opts := tfsdk.ServeOpts{
		// TODO: Update this string with the published name of your provider.
		Name: "registry.terraform.io/hashicorp/scaffolding",
	}

	if debug {
		err = tfsdk.Debug(ctx, provider.New(version), opts)
	} else {
		err = tfsdk.Serve(ctx, provider.New(version), opts)
	}

	if err != nil {
		log.Fatal(err.Error())
	}
}

Bump Expected Minimum Go Version to 1.19

Use cases

Following the Go support policy and given the ecosystem availability of the latest Go minor version, it's time to upgrade. This will ensure that this project can use recent improvements to the Go runtime, standard library functionality, and continue to receive security updates

Proposal

• Run the following commands to upgrade the Go module files and automatically fix outdated Go code:

go mod edit -go=1.19
go mod tidy
go fix ./...

• Ensure any GitHub Actions workflows (.github/workflows/*.yml) use 1.20 in place of any 1.19 and 1.19 in place of any 1.18 or earlier
• Ensure the README or any Contributing documentation notes the Go 1.19 expected minimum
• (Not applicable to all projects) Ensure the .go-version is at least 1.19 or later

References

• https://github.com/hashicorp/terraform-plugin-framework/#go-compatibility
• https://formulae.brew.sh/formula/go
• microsoft/winget-pkgs#95835
• https://community.chocolatey.org/packages/golang
• https://packages.ubuntu.com/lunar/golang-1.20

Missing Provider Configuration Test Examples

If the provider requires configuration, like providing an API token or URL, an example on how to do it is missing from this repository.

This also includes optional values, and default values (are they even a thing with the new terraform-plugin-framework?).

This includes how to configure the provider for unit and acceptance testing.

Incorrect receiver in code comments

👋

I just noticed that your resource code makes reference to d.client.Do and I believe this should be r.client.Do.

I think you must have built the data source code first, then copy/pasted the snippet (which is otherwise identical) and forgot to update the receiver.

Docs generation is not working

$ go generate
rendering website for provider "terraform-provider-scaffolding-framework" (as "terraform-provider-scaffolding-framework"
)
exporting schema from Terraform
compiling provider "scaffolding-framework"
using Terraform CLI binary from PATH if available, otherwise downloading latest Terraform CLI binary
running terraform init
Error executing command: unable to generate website: exit status 1

Error: Incompatible provider version

Provider registry.terraform.io/hashicorp/scaffolding-framework v0.0.1 does
not have a package available for your current platform, windows_386.

Provider releases are separate from Terraform CLI releases, so not all
providers are available for all platforms. Other versions of this provider
may have different platforms supported.



exit status 1
main.go:23: running "go": exit status 1

Add Example Attributes Of All Types

The examples in this repo primarily use types.String and schema.StringAttribute.

I would be a big "nice to have" if this repo included examples for every supported attribute type from https://developer.hashicorp.com/terraform/plugin/framework/handling-data/attributes, as well as examples of block usage from https://developer.hashicorp.com/terraform/plugin/framework/handling-data/blocks.

This would be a big complement to the documentation as the articles mainly focus on an individual type in isolation, instead of a full working example of many of them.

Enhancements to the Dev Experience

Hi there! :) Recently my team was working on a project to build out a custom TF provider. We used a setup which included a dev container, Taskfile, and other toolings to help streamline the development experience and keep things consistent between contributors.

We would love to contribute the (generalized) template Taskfile.yml and/or devcontainer.json configuration, etc. to help others get started quickly on developing their own TF provider. For example, a typical workflow with this setup would look like this:

1. Open the repo in VS Code inside the dev container, which will automatically install all the necessary tools and dependencies.
2. Make changes to provider code.
3. Run task build to build the provider binary using GoReleaser.
4. Run task test to run unit/acceptance tests.
5. When testing an example Terraform configuration, run task tfp to run terraform plan, and task tfa to run terraform apply with auto-approval.
6. If all of the testing results look good, run task lint to check for linting errors or security concerns using the variety of linters set up through the dev container.
7. Run task docs to automatically generate documentation for the provider using tfplugindocs.
8. Commit to feature branch, and open PR.

I wanted to submit an issue here first to see if it was of interest before opening a PR or anything. Thanks!

Model declarations in `resource_example.go` are pointer to structs

At e.g. https://github.com/hashicorp/terraform-provider-scaffolding-framework/blob/main/internal/provider/example_resource.go#L93 the model declarations are pointers to structs. I believe these should just be the structs based on my understanding of .Plan.Get() and .State.Get() functionality, and code elsewhere makes the same declaration (including in the data source example).

Failed to load provider schema after upgraded from v0.7.0 to v0.8.0

with current v0.8.0:
opts := providerserver.ServeOpts{ // TODO: Update this string with the published name of your provider. Address: "<host/ns/name>", Debug: debug, }
How to file the Address when develop the provider on local side ?

On previous verison v0.7.0, my local provider works fine with:
tfsdk.ServeOpts{ Name: "tca", }

But after upgraded to v.0.8.0, it always failed to load the provider schema as:
Error while loading schemas for plugin components: Failed to obtain provider schema: Could not load the schema for provider terraform/provider/tca: failed to instantiate provider "terraform/provider/tca" to obtain schema: Unrecognized
│ remote plugin message:

another way to reproduce the same error is modify the Name to Address as below under v0.7.0:
tfsdk.ServeOpts{ Address: "tca", }