kubernetes / cloud-provider-gcp Goto Github PK

cloud-provider-gcp contains several projects used to run Kubernetes in Google Cloud

License: Apache License 2.0

Go 63.21% Shell 28.26% Starlark 2.64% Dockerfile 0.06% Makefile 0.20% sed 0.05% PowerShell 5.59%

cloud-provider-gcp's Introduction

Kubernetes (K8s)

Kubernetes, also known as K8s, is an open source system for managing containerized applications across multiple hosts. It provides basic mechanisms for the deployment, maintenance, and scaling of applications.

Kubernetes builds upon a decade and a half of experience at Google running production workloads at scale using a system called Borg, combined with best-of-breed ideas and practices from the community.

Kubernetes is hosted by the Cloud Native Computing Foundation (CNCF). If your company wants to help shape the evolution of technologies that are container-packaged, dynamically scheduled, and microservices-oriented, consider joining the CNCF. For details about who's involved and how Kubernetes plays a role, read the CNCF announcement.

To start using K8s

See our documentation on kubernetes.io.

Take a free course on Scalable Microservices with Kubernetes.

To use Kubernetes code as a library in other applications, see the list of published components. Use of the k8s.io/kubernetes module or k8s.io/kubernetes/... packages as libraries is not supported.

To start developing K8s

The community repository hosts all information about building Kubernetes from source, how to contribute code and documentation, who to contact about what, etc.

If you want to build Kubernetes right away there are two options:

You have a working Go environment.

git clone https://github.com/kubernetes/kubernetes
cd kubernetes
make

You have a working Docker environment.

git clone https://github.com/kubernetes/kubernetes
cd kubernetes
make quick-release

For the full story, head over to the developer's documentation.

Support

If you need support, start with the troubleshooting guide, and work your way through the process that we've outlined.

That said, if you have questions, reach out to us one way or another.

Community Meetings

The Calendar has the list of all the meetings in the Kubernetes community in a single location.

Adopters

The User Case Studies website has real-world use cases of organizations across industries that are deploying/migrating to Kubernetes.

Governance

Kubernetes project is governed by a framework of principles, values, policies and processes to help our community and constituents towards our shared goals.

The Kubernetes Community is the launching point for learning about how we organize ourselves.

The Kubernetes Steering community repo is used by the Kubernetes Steering Committee, which oversees governance of the Kubernetes project.

Roadmap

The Kubernetes Enhancements repo provides information about Kubernetes releases, as well as feature tracking and backlogs.

cloud-provider-gcp's People

Contributors

Stargazers

Watchers

Forkers

awly mikedanese mborsz mirake enterstudio nikhita poothia ra489 vishh mooncak anlunas xichengliudui liggitt mrhohn bowei cheftako bhanditz rlenferink sambdavidson krzyzacy injeti-manohar joelsmith nashim220 hoskeri ahmedtd jerminliu prameshj pjh grayluck danielywong xueweiz taipanbox gardener rainbowmango cjcullen happinesstaker ialidzhikov amwat linshanyu dims vitalii-zazmic michaelmdresser cici37 mm4tt naidu-kjml dangerontheranger vinayakankugoyal unamem visierl jpbetz mattcary cindy52 cpanato mrbobbytables jiahuif sfowl tosi3k basantsa1989 isgasho jachou sergeykanzhelev adisky sdmodi kiranopensource princerachit darialarkin ykakarap terrorizer1980 wyuan1704 yjuns spiffxp jeffnais leiyiz openshift aravindakidambi vpnachev isp1r0 sugangli on2itsecurity bigdarkclown achandrasekar aledbf panslava malt3 acumino renatolaq jprzychodzen frankfanslc dhanrajsr geotho devopsleague jindijamie mzkhan isabella232 nathanburnett128 bobbypage notorioustobp blackzlq codersherlock omertuc

cloud-provider-gcp's Issues

Migrate seccomp annotation to API field in manifests

/kind deprecation

Seccomp (secure computing mode) support for Kubernetes has graduated to General Availability (GA). This feature can be used to increase the workload security by restricting the system calls for a Pod (applies to all containers) or individual containers.

seccomp.security.alpha.kubernetes.io/pod and container.seccomp.security.alpha.kubernetes.io/… annotations are now deprecated, and will be removed in Kubernetes v1.22.0. We should identify and transition workloads using the annotations to use the API fields before v1.21.

More resources:

The deprecated annotation is found in these files:

deploy/cloud-controller-manager.manifest
cluster/gce/manifests/glbc.manifest
cluster/gce/manifests/cluster-autoscaler.manifest
cluster/gce/manifests/kube-apiserver.manifest
cluster/gce/manifests/kube-controller-manager.manifest
cluster/gce/manifests/etcd-empty-dir-cleanup.yaml
cluster/gce/manifests/cloud-controller-manager.manifest
cluster/gce/manifests/kube-addon-manager.yaml
cluster/gce/manifests/kube-scheduler.manifest
cluster/gce/manifests/etcd.manifest
cluster/gce/manifests/konnectivity-server.yaml
cluster/addons/metrics-server/metrics-server-deployment.yaml
cluster/addons/metadata-agent/stackdriver/metadata-agent.yaml
cluster/addons/fluentd-elasticsearch/fluentd-es-ds.yaml
cluster/addons/fluentd-elasticsearch/kibana-deployment.yaml
cluster/addons/dns/kube-dns/kube-dns.yaml.in
cluster/addons/dns/kube-dns/kube-dns.yaml.base
cluster/addons/dns/kube-dns/kube-dns.yaml.sed
cluster/addons/dns/coredns/coredns.yaml.in
cluster/addons/dns/coredns/coredns.yaml.base
cluster/addons/dns/coredns/coredns.yaml.sed
cluster/addons/dashboard/dashboard-deployment.yaml
cluster/addons/dns-horizontal-autoscaler/dns-horizontal-autoscaler.yaml
cluster/addons/cluster-loadbalancing/glbc/default-svc-controller.yaml

verify prow job should print go version used

While debugging cloud-provider-gcp-verify-all failures for #294 (e.g. https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/cloud-provider-gcp/294/cloud-provider-gcp-verify-all/1469112391093456896). I couldn't find the go version that was used in the prow job run. This made it difficult to debug. We should print it out.

cc @sugangli

Call utilflag.InitFlags() in CCM main.go after switch over to Cobra commands

Ref:

cloud-provider-gcp/cmd/cloud-controller-manager/main.go

Line 53 in 4c23353

    
           // TODO: once we switch everything over to Cobra commands, we can go back to calling

It was been added by kubernetes/kubernetes#58407. And together within some other files(like https://github.com/kubernetes/kubernetes/blob/271476446b32bfc69a95ea20d8b02d5303128f5e/cmd/kube-proxy/proxy.go#L39).

Add "how to get bazel" instructions to the README

We should document how to get the build working from the readme. Right now, it seems to skip that step.

gcp-controller-manager should support enabling, disabling specific controllers

cc @awly

Handle build tag `providerless` when extractions compeleted

providerless is a build directive specific to K/K. (See kubernetes/kubernetes#80353) It may make sense to keep it for now, as it may still be useful when we are vendored back into K/K but we should have an issue to clean this up once extraction completes. Also its worth being aware of as if we somehow inherit those build rules into this repo it will cause us problems.

Originally posted by @cheftako in #207 (comment)

Ref: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cloud-provider/1179-building-without-in-tree-providers/README.md

Incorrect URL format for load balancers

When creating a load balancer, I get the next error from google API:

Error syncing load balancer: failed to ensure load balancer: failed to create forwarding rule for load balancer (a8811629b918a4c4d8da5705f41f4277(openshift-ingress/router-default)): googleapi: Error 400: Invalid value for field 'resource.target': 'https://compute.googleapis.com/compute/v1/**********************/regions/us-central1/targetPools/a8811629b918a4c4d8da5705f41f4277'. The URL is malformed. Must be a valid In-Project Target Proxy URL or a supported Google API bundle., invalid

This happens because of the change in Google API SDK that was added here with the version bump in #262:

❯ git diff de861a67f32828f69175d064c9847316b1731184~1..HEAD vendor/google.golang.org/api/compute/v1/compute-gen.go
diff --git a/vendor/google.golang.org/api/compute/v1/compute-gen.go b/vendor/google.golang.org/api/compute/v1/compute-gen.go
index a0213951..18c179dd 100644
--- a/vendor/google.golang.org/api/compute/v1/compute-gen.go
+++ b/vendor/google.golang.org/api/compute/v1/compute-gen.go
@@ -1,4 +1,4 @@
-// Copyright 2020 Google LLC.
+// Copyright 2021 Google LLC.
 // Use of this source code is governed by a BSD-style
 // license that can be found in the LICENSE file.
 
@@ -78,11 +78,12 @@ var _ = internaloption.WithDefaultEndpoint
 const apiId = "compute:v1"
 const apiName = "compute"
 const apiVersion = "v1"
-const basePath = "https://compute.googleapis.com/compute/v1/projects/"
+const basePath = "https://compute.googleapis.com/compute/v1/"
+const mtlsBasePath = "https://compute.mtls.googleapis.com/compute/v1/"
 
 // OAuth2 scopes used by this API.

basePath has been changed, so targetPoolURL returns incorrect result: https://github.com/kubernetes/cloud-provider-gcp/blob/master/providers/gce/gce_loadbalancer_external.go#L635

run test jobs / gce-pd tests that're skipped in k/k due to CSIMigrationGCE

in K/K there're jobs that are skipped, either explicitly by adding Driver:.gcepd to the test skip, or implicitly by adding [[Feature:StorageProvider] or skipped if there is no default storage class installed.

In order to recover the lost testing coverage, we need to:

run them with clusters brought up by cloud-provider-gcp since they would have pdcsi drivers installed by default.
make sure that there's a default storage class installed for those clusters.

Here is a list of PRs that have test skipping:
kubernetes/kubernetes@2298dc6
kubernetes/test-infra#24330
kubernetes/test-infra#24425
kubernetes/kubernetes#106503
kubernetes/test-infra#23866
kubernetes/kubernetes#105052
kubernetes/test-infra#24438
kubernetes/test-infra#24490
kubernetes/test-infra#24695

kubernetes/test-infra#24625

"Forbidden" error code when using GCP CCM

A recent change #268 introduced IPv6 support, but it also began to require Google Cloud Compute Aplha API access. Unfortunately our CI doesn't have this access and therefore, when I start the CCM, it shows this error:

1 node_controller.go:241] Error getting instance metadata for node addresses: error while querying for providerID "gce://openshift-gce-devel-ci/us-central1-c/ci-op-l5l1d85s-8c513-m8pk7-master-2": googleapi: Error 403: Required 'Alpha Access' permission for 'Compute API', forbidden

More logs.

It would be nice to make the IPv6 support configurable and use GA Compute API when it's not enabled.

Add example/docs for using cloud-controller-manager

We are using k8s.gcr.io/cloud-controller-manager and looking to upgrade it with latest cloud-provider-gcp binary, but simply replacing the image does not work it required extra arguments but they are not documented anywhere.
I checked out the manifest template mentioned in bug #232 (it also mentioned adding docs) but here CRD type is pod whereas earlier it was deployed as a daemonset which raises more questions.

Can we please add ready to use manifest with examples and documentation around it to help people get started.

No Release binary available for any GCP plugins

I was trying to use gcp auth exec plugin for kubelet credential provider
https://github.com/kubernetes/cloud-provider-gcp/tree/master/cmd/auth-provider-gcp
There are no binaries available, I have to build locally and use.

Migrate `cloud-provider-gcp` to use Lease API

/sig scalability
/sig api-machinery

Support for endpoints and configmaps is removed in K8s 1.24 with kubernetes/kubernetes#106852.
More details about the motivation to switch to leases - ref kubernetes/kubernetes#80289.

To preserve the backwards-compatibility, resource lock for migration purposes (endpointsleases, configmapsleases) should be used when switching from the legacy resource locks (endpoints, configmaps)

cloud-provider-gcp is currently using endpoints. We want to migrate this to endpointsleases and finally to leases.

CC - @wojtek-t

Create a SECURITY_CONTACTS file.

As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.

The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".

Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)

Thanks so much, let me know if you have any questions.

(This issue was generated from a tool, apologies for any weirdness.)

[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md

Unable to install GCP credential provider with 'go get' or 'go install'

Using go get getting following error


$ go get k8s.io/cloud-provider-gcp/cmd/auth-provider-gcp@latest
go get: k8s.io/cloud-provider-gcp@none updating to
	k8s.io/[email protected] requires
	k8s.io/cloud-provider-gcp/[email protected]: invalid version: unknown revision 000000000000

$ go get k8s.io/cloud-provider-gcp/cmd/auth-provider-gcp
go get: k8s.io/cloud-provider-gcp@none updating to
	k8s.io/[email protected] requires
	k8s.io/cloud-provider-gcp/[email protected]: invalid version: unknown revision 000000000000

$ go get k8s.io/cloud-provider-gcp/cmd/[email protected]
go get: k8s.io/cloud-provider-gcp@none updating to
	k8s.io/[email protected] requires
	k8s.io/cloud-provider-gcp/[email protected]: invalid version: unknown revision 000000000000

Using go install getting following error

$ go install  k8s.io/cloud-provider-gcp/cmd/auth-provider-gcp@latest
go install k8s.io/cloud-provider-gcp/cmd/auth-provider-gcp@latest: k8s.io/[email protected]
	The go.mod file for the module providing named packages contains one or
	more replace directives. It must not contain directives that would cause
	it to be interpreted differently than if it were the main module.

Documentation: missing instruction on how to generate cloud-controller-manager manifests ready to use

Seems that if you are not a bazel expert (like me) it's not straight forward to build the cloud-controller-manager manifets to try it.

README for updating dependencies should be a single command

https://github.com/kubernetes/cloud-provider-gcp#update-all-dependencies recommends calling go mod tidy && ./tools/update_vendor.sh to clean up unused dependencies. We should have update-vendor.sh do this automatically (and verify-vendor.sh should ensure it was done right, if it doesn't already).

Migrate cluster changes between v1.18 & v1.19 from K/K to K/cloud-provider-gcp

K/cloud-provider-gcp cluster is a forklift of the version in K/K. It then has changes applied to it to support things like the KCM, CSI migration, ANP etc. It is necessary to keep applyingmerging the changes from K/K to K/cloud-provider-gcp.

The list of these changes can be computed using
git rev-list origin/release-1.18..origin/release-1.19 cluster/gce cluster/kube-up.sh cluster/common.sh cluster/kube-util.sh cluster/addons

Dependency checklist:

Update go version

Since k/k has been updated to go1.15 in v1.19, go1.16 in v1.21, better to keep cloud-provider-gcp synced.

Are all failures merely temporary bumps in the road?

The logic for temporaryError is suspicious:

cloud-provider-gcp/cmd/gcp-controller-manager/node_csr_approver.go

Lines 560 to 562 in 3266c50

    
           // temporaryError is used within validators to decide between hard-deny and 
        
           // temporary inability to validate. 
        
           type temporaryError error

cloud-provider-gcp/cmd/gcp-controller-manager/node_csr_approver.go

Lines 451 to 455 in 3266c50

    
           if _, ok := err.(temporaryError); ok { 
        
           	return false, fmt.Errorf("fetching EK public key from API: %v", err) 
        
           } 
        
           klog.Infof("deny CSR %q: fetching EK public key from API: %v", csr.Name, err) 
        
           return false, nil

Because these are interfaces (not structs), I believe all errors will implement temporaryError:
https://play.golang.org/p/ZLq4FoS6vc8

I think we need temporaryError to be a struct.

cloud-provider-gcp-e2e-create is failing

Failing run

Most recent passing run

The only difference I see is a change in the kubekins-e2e image: v20200802-ff98b58-master vs v20200726-f8d6253-master

The error:

+ make install
/home/prow/go/src/k8s.io/kubetest2/hack/go_container.sh go build -v -o /out/kubetest2 .
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

I don't see the docker socket anywhere in the volume mounts for either the passing or failing run, so I'm really not sure how this worked to begin with.

Nodes network-unavailable taint not removed after installing ccm

After installing ccm the node.kubernetes.io/network-unavailable taint is added if --configure-cloud-routes=false is set to false. Here's the args used to initialize the ccm

args:
    - --bind-address=127.0.0.1
    - --cloud-provider=gce
    - --use-service-account-credentials
    - --configure-cloud-routes=false
    - --allocate-node-cidrs=true
    - --cluster-cidr=10.244.0.0/16

RBAC lacks cluster-id ConfigMap permissions

and I can't determine whether it is intentional.

Looking at the implementation, it seems like there is a clear need for a CREATE permission on configmap resources. Unfortunately, Role manifest lacks this permissions. This leads to:

E1028 13:43:43.000801       1 gce_clusterid.go:201] GCE cloud provider failed to create kube-system/ingress-uid config map to store cluster id: configmaps is forbidden: User "system:serviceaccount:d8-cloud-provider-gcp:cloud-controller-manager" cannot create resource "configmaps" in API group "" in the namespace "kube-system"

Cut a release for K8s 1.23

What would you like to be added:
Please cut a release for use with Kubernetes 1.23.

Why is this needed:
Kubernetes 1.23.0 has been released. Users should have a fixed tag for pulling a corresponding GCP CCM.

/kind feature
/sig cloud-provider
/priority important-soon

Provide steps how to push to docker.io

Currently as far I see only publishing image to gcr.io is supported

cloud-provider-gcp/defs/container.bzl

Lines 28 to 36 in ef10a68

    
           container_push( 
        
               name = "publish", 
        
               format = "Docker", 
        
               image = ":image", 
        
               registry = "gcr.io", 
        
               repository = repository, 
        
               stamp = True, 
        
               tag = "{STABLE_IMAGE_TAG}", 
        
           )

I would be nice if there are steps how to achieve the same for docker.io.

Support `--account` option for `gke-gcloud-auth-plugin`

The gke-gcloud-auth-plugin plugin doesn't allow to inject custom gcloud params, like --account option:

cloud-provider-gcp/cmd/gke-gcloud-auth-plugin/cred/cred.go

Line 345 in 66064c6

return executeCommand("gcloud", "config", "config-helper", "--format=json")

Account name option was so far - at lest for me - a very useful way to work with multiple user identities handled by one .kube/config file. Otherwise, every time before running kubectl against some some cluster, I'd need to run gcloud config set account ... (to make sure the the token will be generated properly).

Is there a way to expose it as a separate param? Or maybe a set of additional params to be passed to the underlying gcloud call?

Fix support on case-insensitive filesystems

cloud-provider-gcp has a BUILD file in its project root alongside a build directory. This causes issues on case-insensitive filesystems like on MacOS.

For example, a fresh git clone:

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	deleted:    BUILD

no changes added to commit (use "git add" and/or "git commit -a")

and a dep init in a project that includes k8s.io/cloud-provider-gcp:

$ dep init
init failed: unable to solve the dependency graph: Solving failure:
	(1) failed to clean up git repository at /gopath/pkg/dep/sources/https---github.com-kubernetes-cloud--provider--gcp - dirty? corrupted? status output:
 D build/BUILD

Would it be possible to fix this? perhaps renaming the build directory.

Update go version support and add go version check

It would be great to keep go version up to date with Kubernetes/kubernetes(we are currently behind).
Add go version check into presubmits checking would help for go version maintenance.

Break up gcp-controller-managed into logical packages

Right now it's:

minimal package main
package app that has ~everything

Proposed structure:

minimal main
pkg/controller
pkg/controller/csrapprover
pkg/controller/csrsigner
pkg/controller/nodeannotator
pkg/cacache

Migrate gcp-controller-manager leader election client to lease lock.

Missing CONTRIBUTING.md file

All K8s subrepositories should have a CONTRIBUTING.md file, which at the minimum should point to https://github.com/kubernetes/community/blob/master/contributors/guide/README.md. Care should be taken that all information is in sync with the contributor guide.

Subrepositories may also have contributing guidelines specific to that repository. They should be explicitly documented and explained in the CONTRIBUTING.md

Ref: kubernetes/community#1832

There are no official published images for CCM

I see that k8s.gcr.io/cloud-controller-manager is published up to version 1.15.9. Are the images not published anymore or has the images registry changed?

go mod tidy returns an error about a broken link for https://knative.dev/eventing-contrib

After a recent update of API, when I try to execute go mod tidy on a freshly cloned repo, I see this error:
❯ go mod tidy
go: sigs.k8s.io/[email protected] requires
sigs.k8s.io/[email protected] requires
k8s.io/[email protected] requires
github.com/tektoncd/[email protected] requires
knative.dev/[email protected]: unrecognized import path "knative.dev/eventing-contrib": reading https://knative.dev/eventing-contrib?go-get=1: 404 Not Found

tests in provider/gce can fail due to missing test dependencies

Example test failure for #204 : https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/cloud-provider-gcp/204/cloud-provider-gcp-tests/1387458716499972096/build-log.txt

The problem here seems to be related to the fact that there are now two modules in cloud-provider-gcp:

root module
module under providers/gcp

The failing test command is:

bazel test --test_output=errors -- //... -//vendor/...

This command runs the unit tests of the provider/gcp module as well as the unit tests of the root module. It fails when it can't find the k8s.io/cloud-provider/credentialconfig package that the providers/gce/gcpcredential package depends on.

k8s.io/cloud-provider/credentialconfig is missing because when go mod vendor is run on the root module, it decides not to include credentialconfig into the vendor directory. go mod vendor believes it can safely omit the credentialconfig directory because there are no non-test build dependencies on it . There is a test dependency on it, but only from the provider/gcp module which go mod vendor assumes it doesn't need to satisfy, because it sees it as just another vendored in dependency (golang/go#34435 (comment)). This is no unreasonable because go mod vendor has no way to know that we set up a symlink from vendor/k8s.io/cloud-provider-gcp/providers to provider/gcp and treat what it considers to be a vendored in dependency as source code that we need to test.

k8s/k8s presumably has a similar problem with the staging repos. I know that in k8s/k8s go test doesn't work on staging repos when run from the root. I'm less familiar with how bazel test works for staging (if anyone knows, please feel free to explain).

Possible solution: Replace bazel test --test_output=errors -- //... -//vendor/... with one test command that only tests the root module and another that tests the provider/gcp in a way that works--maybe just don't use the vendor directory for the provider/gcp module testing?

Documentation

Hey folks,

Apologies if this is not the place to ask this, but I couldn't find any documentation regarding using this controller on gce.

The k8s documentation is very broad and cloud agnostic. Any help/suggestions?

GCP Credential provider not returning valid cache key type

While trying to use the GCP image credential provider plugin getting errors in the kubelet logs
credential provider plugin did not return a valid cacheKeyType: ""

Not sure but on digging some code, it is not returning any value for cacheKeyType
https://github.com/kubernetes/cloud-provider-gcp/blob/master/cmd/auth-provider-gcp/provider/provider.go#L84

cacheKeyType is required to be specified by plugin as per the API https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/kubelet/pkg/apis/credentialprovider/types.go#L64

Cannot build on darwin_amd64

Operating system: macOS Catalina 10.15.1

Steps to reproduce:

Close the repository

$ mkdir -p ~/go/src/github.com/kubernetes
$ cd ~/go/src/github.com/kubernetes
$ git clone [email protected]:kubernetes/cloud-provider-gcp.git
$ cd cloud-provider-gcp

Install bazel

$ bazel --version
bazel 1.2.1

Ensure that the publish fails

$ IMAGE_REPO=foo/gcp-controller-manager IMAGE_TAG=v1 bazel run //cmd/gcp-controller-manager:publish
Starting local Bazel server and connecting to it...
INFO: Call stack for the definition of repository 'go_sdk' which is a _go_download_sdk (rule definition at /private/var/tmp/_bazel_me/63495b22c22bd0073891a0528125c5f7/external/io_bazel_rules_go/go/private/sdk.bzl:80:20):
 - /private/var/tmp/_bazel_me/63495b22c22bd0073891a0528125c5f7/external/io_bazel_rules_go/go/private/sdk.bzl:93:5
 - /Users/me/go/src/github.com/kubernetes/cloud-provider-gcp/WORKSPACE:45:1
ERROR: An error occurred during the fetch of repository 'go_sdk':
   unsupported platform darwin_amd64
ERROR: While resolving toolchains for target //cmd/gcp-controller-manager:publish: invalid registered toolchain '@go_sdk//:go_android_amd64': no such package '@go_sdk//': unsupported platform darwin_amd64
ERROR: Analysis of target '//cmd/gcp-controller-manager:publish' failed; build aborted: invalid registered toolchain '@go_sdk//:go_android_amd64': no such package '@go_sdk//': unsupported platform darwin_amd64
INFO: Elapsed time: 7.203s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (6 packages loaded, 6 targets configured)
FAILED: Build did NOT complete successfully (6 packages loaded, 6 targets configured)
    currently loading: @bazel_tools//tools/jdk
    Fetching @local_config_cc_toolchains; Restarting.
    Fetching @rules_java; fetching
    Fetching @rules_cc; fetching

the logic checking whether instance belongs to cluster needs to handle errors better

We hit situation when a single nodepool issue (IGM was deleted and got stuck in this state) caused this logic to fail:

cloud-provider-gcp/cmd/gcp-controller-manager/csr_approver.go

Lines 575 to 614 in 6a0760c

    
           func clusterHasInstance(ctx *controllerContext, instanceZone string, instanceID uint64) (bool, error) { 
        
           	// instanceZone looks like 
        
           	// "https://www.googleapis.com/compute/v1/projects/my-project/zones/us-central1-c" 
        
           	// Convert it to just "us-central1-c". 
        
           	instanceZone = path.Base(instanceZone) 
        
           	clusterName := fmt.Sprintf("projects/%s/locations/%s/clusters/%s", ctx.gcpCfg.ProjectID, ctx.gcpCfg.Location, ctx.gcpCfg.ClusterName) 
        
           	recordMetric := csrmetrics.OutboundRPCStartRecorder("container.ProjectsLocationsClustersService.Get") 
        
           	cluster, err := container.NewProjectsLocationsClustersService(ctx.gcpCfg.Container).Get(clusterName).Do() 
        
           	if err != nil { 
        
           		recordMetric(csrmetrics.OutboundRPCStatusError) 
        
           		return false, fmt.Errorf("fetching cluster info: %v", err) 
        
           	} 
        
           	recordMetric(csrmetrics.OutboundRPCStatusOK) 
        
           	for _, np := range cluster.NodePools { 
        
           		for _, ig := range np.InstanceGroupUrls { 
        
           			igName, igLocation, err := parseInstanceGroupURL(ig) 
        
           			if err != nil { 
        
           				return false, err 
        
           			} 
        
           			// InstanceGroups can be regional, igLocation can be either region 
        
           			// or a zone. Match them to instanceZone by prefix to cover both. 
        
           			if !strings.HasPrefix(instanceZone, igLocation) { 
        
           				klog.V(2).Infof("instance group %q is in zone/region %q, node sending the CSR is in %q; skipping instance group", ig, igLocation, instanceZone) 
        
           				continue 
        
           			} 
        
           			// Note: use igLocation here instead of instanceZone. 
        
           			// InstanceGroups can be regional, instances are always zonal. 
        
           			ok, err := groupHasInstance(ctx, igLocation, igName, instanceID) 
        
           			if err != nil { 
        
           				return false, fmt.Errorf("checking that group %q contains instance %v: %v", igName, instanceID, err) 
        
           			} 
        
           			if ok { 
        
           				return true, nil 
        
           			} 
        
           		} 
        
           	} 
        
           	return false, nil 
        
           }

Sync node-csr-<redacted> failed with : validating CSR "node-csr-<redacted>": checking VM membership in cluster: checking that group "<redacted>" contains instance <redacted>: googleapi: Error 404: The resource 'projects/<redacted>/zones/us-west1-b/instanceGroupManagers/<redacted>' was not found, notFound

Ideally the error handling logic in the function should skip errors and only fail with error if instance is not found in any of node pools.

Get prow to compile/run tests on PRs

@calebamiles do you know how to make this work?

Add soft link on //providers

Add soft link to make sure the development under //provider could be reflected immediately.

CCM/GCP feature gate filter should unconditionally log which feature gates were filtered out

#283 merged with a log line that would be printed if all feature gates were filtered from the CCM, but it would be better to unconditionally log which feature gates were filtered, regardless of whether or not the CCM ends up with any feature gates sent to it.

cloud-provider-gcp prow jobs should use same version as in go.mod

prow uses go 1.17
go.mod has 1.16

Having the go.mod files use 1.16 while prow uses 1.17 resulted in gofmt expecting files to be formatted differently (go 1.17 introduced "go:build" tags).

There are two parts to this:

we should probably bump to 1.17 now
I'd like prow and go.mod to stay in sync if at all possible

Update to use CSR certificates.k8s.io/v1 API before v1.22

CSR APIs earlier than certificates.k8s.io/v1 are deprecated and scheduled to no longer be served in Kubernetes v1.22

The CSR approver/signer controllers in this repo should be updated to use the v1 API prior to that.

The certificates.k8s.io/v1 APIs are available starting in v1.19

/cc @mikedanese @ahmedtd

Fix golint failure in cloud-provider-gcp/cmd/auth-provider-gcp/app/getcredentials.go

Golint check shows some suggestions in getcredentials.go

Ref: #198

Dual-stack support for LoadBalancer k8s services

Are there any plans to support dual-stack for LoadBalancer k8s services?

I have not found any docs clearly stating dual-stack support for GCP network load-balancers, so not sure whether it is even possible ATM. Just wondering if there are any plans on this?

gke-certificates-controller: approve kubelet serving certificates

Validate VM info in CSR against GCE API and approve/deny.

Migrate from dep to modules

Remove copied credential provider code once relevant PRs merge

In order for #168 to not depend on any k/k-private code, some packages had to be copied out from other PRs. Those packages are:

pkg/apis/credentialprovider - from kubernetes/kubernetes#94196, can be removed once cloud-provider-gcp supports k8s v1.20
pkg/gcpcredential - from kubernetes/kubernetes#95775, can be removed once the PR merges into a release cloud-provider-gcp supports
pkg/credentialconfig - also from kubernetes/kubernetes#95775, can be removed once the PR merges into a release cloud-provider-gcp supports

E2E testing for credential provider

The provider has a unit test suite but could use some work on the E2E side. It should be integrated with the new E2E test that cloud-provider-gcp has (cloud-provider-gcp-e2e-full), so either integration with the rest of the kubetest2 suite or modifying the presubmit itself will likely be necessary, I believe. The actual test itself should at least involve bringing up a cluster with the provider enabled, but testing error handling (attempting to fetch credentials on an image that doesn't exist, etc.) would be good, as well.

Cluster kube-up references sha1

https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/cloud-provider-gcp/144/cloud-provider-gcp-e2e-create/1252861902158041089#1:build-log.txt%3A96

I think those are no longer generated?

Cleaning the bazel cache exposes the problem.

bazel clean --expunge
bazel build //release:release-tars
cluster/kube-up.sh

/cc @cheftako @BenTheElder

Enable Functionality to Support a Shared-VPC Model

Hi All,

We are running a K8s Cluster in GCP deployed using Pivotal Container Service 1.6.1.

The current architecture is defined using a Shared-VPC Model inside of GCP. The vpc-network is configured in the Host project and the cluster is configured in the Service project.

After provisioning the cluster and deploying a Service of type LoadBalancer, it fails to actually provision a resource inside of GCP with the following:

Error syncing load balancer: failed to ensure load balancer: googleapi: Error 404: The resource 'projects/<my-project-id>/global/networks/<my-network-id>' was not found

Where <my-project-id> is the Service Project (this is where we want our Resources to be provisioned) and <my-network-id> is the Host Project (this is where the VPC lives and is shared to <my-project-id>)

We've tried to define the full Project ID of both the Host and Service projects, however it seems that it is unable to differentiate between the projects.

Given that there are many users who use this Shared-VPC model, I assumed that this functionality already exists inside of the provider. Is this a true statement?

If so, can someone guide us in the right direction on how to properly specify the project/vpc-network IDs? If not, could this feature be implemented?

Thanks

	// temporaryError is used within validators to decide between hard-deny and
	// temporary inability to validate.
	type temporaryError error

	if _, ok := err.(temporaryError); ok {
	return false, fmt.Errorf("fetching EK public key from API: %v", err)
	}
	klog.Infof("deny CSR %q: fetching EK public key from API: %v", csr.Name, err)
	return false, nil

	container_push(
	name = "publish",
	format = "Docker",
	image = ":image",
	registry = "gcr.io",
	repository = repository,
	stamp = True,
	tag = "{STABLE_IMAGE_TAG}",
	)

	func clusterHasInstance(ctx *controllerContext, instanceZone string, instanceID uint64) (bool, error) {
	// instanceZone looks like
	// "https://www.googleapis.com/compute/v1/projects/my-project/zones/us-central1-c"
	// Convert it to just "us-central1-c".
	instanceZone = path.Base(instanceZone)

	clusterName := fmt.Sprintf("projects/%s/locations/%s/clusters/%s", ctx.gcpCfg.ProjectID, ctx.gcpCfg.Location, ctx.gcpCfg.ClusterName)
	recordMetric := csrmetrics.OutboundRPCStartRecorder("container.ProjectsLocationsClustersService.Get")
	cluster, err := container.NewProjectsLocationsClustersService(ctx.gcpCfg.Container).Get(clusterName).Do()
	if err != nil {
	recordMetric(csrmetrics.OutboundRPCStatusError)
	return false, fmt.Errorf("fetching cluster info: %v", err)
	}
	recordMetric(csrmetrics.OutboundRPCStatusOK)
	for _, np := range cluster.NodePools {
	for _, ig := range np.InstanceGroupUrls {
	igName, igLocation, err := parseInstanceGroupURL(ig)
	if err != nil {
	return false, err
	}
	// InstanceGroups can be regional, igLocation can be either region
	// or a zone. Match them to instanceZone by prefix to cover both.
	if !strings.HasPrefix(instanceZone, igLocation) {
	klog.V(2).Infof("instance group %q is in zone/region %q, node sending the CSR is in %q; skipping instance group", ig, igLocation, instanceZone)
	continue
	}

	// Note: use igLocation here instead of instanceZone.
	// InstanceGroups can be regional, instances are always zonal.
	ok, err := groupHasInstance(ctx, igLocation, igName, instanceID)
	if err != nil {
	return false, fmt.Errorf("checking that group %q contains instance %v: %v", igName, instanceID, err)
	}
	if ok {
	return true, nil
	}
	}
	}
	return false, nil
	}