Comments (13)
In GKE, owner does not grant full permissions inside the kubernetes cluster. My guess is that you need to grant your user a clusterrolebinding with higher privileges.
This stack overflow does a decent job of describing the process:
https://stackoverflow.com/questions/46307325/gke-clusterrolebinding-for-cluster-admin-fails-with-permission-error
from rbac-lookup.
Thank you @sudermanjr, but I get the same error even with the cluster-admin clusterrolebinding assigned to my user:
% kubectl describe clusterrolebindings cluster-admin-binding
Name: cluster-admin-binding
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: cluster-admin
Subjects:
Kind Name Namespace
---- ---- ---------
User <MY_EMAIL>
from rbac-lookup.
Ah, the issue is not the account used to start the cluster, it's the account you are using when you run rbac-lookup and the permissions that it has in your GCP account. Basically, you need to be able to run some form of gcloud iam
command in your project. I can dig further to find out exactly what's needed, but that's the direction to look
from rbac-lookup.
Thank you, but I am using the same account to create the cluster and also to run rbac-lookup (my user is the owner of the project and I am creating a cluster-admin role for it):
gcloud container clusters create kube
kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user $(gcloud config get-value account)
rbac-lookup --gke
Error loading RBAC Bindings: googleapi: Error 403: The caller does not have permission, forbidden
from rbac-lookup.
Hey @ryepesg, thanks for reporting this problem! I think the confusion here is that you actually need GCP IAM access here, not just GCP cluster access. Even a GKE cluster admin might not actually have project IAM access.
There's 3 calls this could be happening on, to help me rule out one of them, can you try to run this command?
gcloud projects get-iam-policy YOUR_PROJECT_NAME_HERE
If that works for you, that rules out a problem with https://github.com/reactiveops/rbac-lookup/blob/master/lookup/lister.go#L198. On the other hand, if it doesn't you'll need more IAM access for your account.
Either way, there are a couple places above that in that method that could also trigger an error like that. I'll work on adding some better error handling so we can tell which of those actually triggered the error, as well as changing the initial scope request to be read only.
Thanks again for all the help debugging this!
from rbac-lookup.
Thank you @robscott, I am the owner of the project in IAM and I am also a cluster-admin of the cluster. Here is the output of the command:
% gcloud projects get-iam-policy $(gcloud config get-value project)
bindings:
- members:
- group:<A_GROUP_EMAIL>
role: projects/oauthz/roles/CustomRole
- members:
- serviceAccount:[email protected]
role: roles/compute.admin
- members:
- serviceAccount:[email protected]
role: roles/compute.serviceAgent
- members:
- user:<MY_EMAIL>
role: roles/container.admin
- members:
- serviceAccount:service-140183570641@container-engine-robot.iam.gserviceaccount.com
role: roles/container.serviceAgent
- members:
- serviceAccount:[email protected]
- serviceAccount:[email protected]
- serviceAccount:[email protected]
role: roles/editor
- members:
- serviceAccount:[email protected]
role: roles/logging.logWriter
- members:
- serviceAccount:[email protected]
role: roles/monitoring.metricWriter
- members:
- serviceAccount:[email protected]
role: roles/monitoring.viewer
- members:
- user:<MY_EMAIL>
role: roles/owner
- members:
- serviceAccount:[email protected]
role: roles/storage.objectViewer
Thank you,
from rbac-lookup.
@ryepesg I've released a new version of rbac-lookup with one potential fix, along with some more helpful error messages. Let me know if that helps narrow down the problem. Thanks again for all the help!
from rbac-lookup.
Thank you. Here is my new attempt using a service account:
% kubectl describe clusterrolebinding cluster-admin-binding
Name: cluster-admin-binding
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: cluster-admin
Subjects:
Kind Name Namespace
---- ---- ---------
User [email protected]
% gcloud config get-value project
oauthz
% gcloud config get-value account
[email protected]
% rbac-lookup --gke
Error loading Google Cloud IAM Policy for project: oauthz
Error loading RBAC Bindings: googleapi: Error 403: The caller does not have permission, forbidden
from rbac-lookup.
Hey @ryepesg, thanks for all that output! My best guess here is that the logic for parsing the GCP project ID here was not functioning here. When you use the gcloud CLI to get GKE credentials, it generally initializes a cluster with a name following this pattern:
gke_reactiveopsio_us-central1-a_rbac-lookup-testing
I split that by underscores, and the 2nd element in the resulting array generally represents the GCP project name. In your case that computed to oauthz
which seems unlikely to be accurate. In the latest release I added a couple fallback methods:
- It will get the GCP application default credentials as configured in your environment and attempt to parse a project ID from those
- It will get the value of the
CLOUDSDK_CORE_PROJECT
environment variable used by GCP to reference the project ID
In both cases, if there is a value found, it will attempt to use it to load an IAM policy, and if that fails, it will move on to the next one. Hopefully this helps, thanks for all the detailed feedback!
from rbac-lookup.
Thanks for the answer. This my new run:
% rbac-lookup --gke
Could not load IAM policy for oauthz project from parsed kubeconfig
No project ID found in default GCP credentials
Error loading RBAC Bindings: Error loading IAM policies for GKE, try setting CLOUDSDK_CORE_PROJECT environment variable
% CLOUDSDK_CORE_PROJECT=oauthz rbac-lookup --gke :(
Could not load IAM policy for oauthz project from parsed kubeconfig
No project ID found in default GCP credentials
Could not load IAM policy for oauthz project from CLOUDSDK_CORE_PROJECT environment variable
Error loading RBAC Bindings: googleapi: Error 403: The caller does not have permission, forbidden
% grep oauthz ~/.kube/config :(
name: gke_oauthz_australia-southeast1-a_kube
cluster: gke_oauthz_australia-southeast1-a_kube
user: gke_oauthz_australia-southeast1-a_kube
name: gke_oauthz_australia-southeast1-a_kube
current-context: gke_oauthz_australia-southeast1-a_kube
- name: gke_oauthz_australia-southeast1-a_kube
Using a different project, with a different user:
% set gcloud config set account <OTHER_EMAIL>
% gcloud container clusters create kube --project platform-test
% rbac-lookup --gke
Could not load IAM policy for platform-test project from parsed kubeconfig
No project ID found in default GCP credentials
Error loading RBAC Bindings: Error loading IAM policies for GKE, try setting CLOUDSDK_CORE_PROJECT environment variable
% grep platform-test ~/.kube/config
name: gke_platform-test_australia-southeast1-a_kube
cluster: gke_platform-test_australia-southeast1-a_kube
user: gke_platform-test_australia-southeast1-a_kube
name: gke_platform-test_australia-southeast1-a_kube
current-context: gke_platform-test_australia-southeast1-a_kube
- name: gke_platform-test_australia-southeast1-a_kube
% CLOUDSDK_CORE_PROJECT=platform-test rbac-lookup --gke
Could not load IAM policy for platform-test project from parsed kubeconfig
No project ID found in default GCP credentials
Could not load IAM policy for platform-test project from CLOUDSDK_CORE_PROJECT environment variable
Error loading RBAC Bindings: googleapi: Error 403: The caller does not have permission, forbidden
However, the rbac-lookup --gke
does work from my office with an existing cluster. Maybe this is a bug just working with new clusters. Will close the ticket, thanks a lot for your help.
from rbac-lookup.
@ryepesg Somehow I just realized I was likely going down the wrong, much more complicated path here. What happens if you run gcloud auth application-default login
before running rbac-lookup? The Google auth I'm using apparently searches for "application default credentials" which can be different than the credentials used by the gcloud CLI command: https://github.com/golang/oauth2/blob/HEAD/google/default.go#L61
from rbac-lookup.
Thank you @robscott! That was the problem indeed. I even tested changing the account with gcloud config set account
to a different one, but rbac-lookup
keep using the application default credentials (created by gcloud auth application-default login
in .config/gcloud/application_default_credentials.json
).
So, the correct way to change the account is:
GOOGLE_APPLICATION_CREDENTIALS=serviceaccount.json rbac-lookup --gke
That works even after using gcloud auth application-default revoke
.
Thanks for all the help, cheers,
from rbac-lookup.
Nice! Glad that solved it. I'll work on updating the docs to be more helpful here. Thanks for all your help working through this!
from rbac-lookup.
Related Issues (20)
- Dependabot can't parse your go.mod
- CLA Link Broken? HOT 1
- Dependabot can't resolve your Go dependency files
- Increase Test Coverage to 50% HOT 1
- Compile multi-arch HOT 3
- Dependabot can't resolve your Go dependency files HOT 1
- Service Account Namespace HOT 4
- CLOUDSDK_CORE_PROJECT env var HOT 8
- Dependabot can't resolve your Go dependency files HOT 1
- Dependabot can't resolve your Go dependency files
- permissions are too harsh IMO HOT 2
- DNS Resolution does not work when using VPN HOT 4
- Warning message on Brew MacOS HOT 2
- v0.8.1 does not run under bare ubi8/ubi HOT 1
- [FEAT]: make `lookup` directory a public pkg to be imported in golang projects HOT 1
- Having issues with kubectl 1.23 client version HOT 3
- Dependabot couldn't find a Gopkg.toml for this project
- Dependabot couldn't find a Gopkg.toml for this project HOT 1
- Use krew-release-bot for publishing plugin releases HOT 3
- Add basic e2e testing HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rbac-lookup.