Comments (13)
Hi @gonzaatcroud,
Apologies for the very late response.
I've started some initial work on this by beginning the mapping, however this may take some time before it's sufficient enough to be included in the tooling - but it is possible in the future.
from iamlive.
Preview support out today, but lots of mappings missing so use with caution.
Feedback welcomed!
from iamlive.
Hey @joemiller,
Thanks for the detailed response, that's super insightful.
According to the datasource, only the following methods should produce the orgpolicy.policy.set
permission:
cloudresourcemanager.folders.clearOrgPolicy
cloudresourcemanager.folders.setOrgPolicy
cloudresourcemanager.organizations.clearOrgPolicy
cloudresourcemanager.organizations.setOrgPolicy
cloudresourcemanager.projects.clearOrgPolicy
cloudresourcemanager.projects.setOrgPolicy
So I would also find it weird that the Terraform provider would attempt that.
There's an outstanding issue with intercepting responses, however I've cut a new version (1.1.3) which includes a --debug
flag which will dump the full request and disable terminal clearing so you can hopefully narrow down the call.
Let me know how this goes.
from iamlive.
Hey @joemiller,
Thanks for that detail. I've determined the presence of the :
character in the matching path is conflicting with the URL matching library I'm currently using.
Will work on a fix shortly.
from iamlive.
Hey @joemiller,
I've updated the pattern matching logic to now uses Regex to avoid the issue with the :
character as previously mentioned. An unfortunate but necessary performance hit - shouldn't be too bad for now though. This is released in v1.1.4.
Could you try the same now and let me know how you go?
from iamlive.
Awesome feedback, thanks!
Again, not every single permission is mapped out yet - but I'm working through it. Appreciate your time testing things out.
from iamlive.
Hi @gonzaatcroud,
Apologies for the very late response.
I've started some initial work on this by beginning the mapping, however this may take some time before it's sufficient enough to be included in the tooling - but it is possible in the future.
This would be amazing, thanks for sharing an update on it!
from iamlive.
This is fantastic! thanks for adding this.
It works fine with the gcloud
cli after setting the proxy config with gcloud config set
as described in the README, however, terraform google provider seems to ignore these settings. Are there environment vars that can be set instead? I'm assuming the tf google provider is using the standard go sdk for gcp
from iamlive.
Hey @joemiller,
I'm surprised that hasn't worked out of the box for you if it's using the SDK under the hood. I do notice an unresolved issue relating to this here: hashicorp/terraform#523
Do none of the configs work?
gcloud config set proxy/type http
gcloud config set proxy/address 127.0.0.1
gcloud config set proxy/port 10080
gcloud config set core/custom_ca_certs_file ~/.iamlive/ca.pem
http_proxy=http://127.0.0.1:10080 HTTP_PROXY=http://127.0.0.1:10080 HTTPS_PROXY=http://127.0.0.1:10080 https_proxy=http://127.0.0.1:10080 terraform plan
If not, I may go hunting in Terraform.
from iamlive.
@iann0036 The gcloud config set
pattern did not work. Running tf just ran as normal, no output from iamlive
proxy.
I did have a little more luck with the _PROXY env vars. The problem is that it also sends the aws provider thru the proxy, if I recall. I was using it with a rather large stack that had multiple providers (eg. aws, gcp, k8s)
I'm also running tf via terragrunt
. I'm not sure yet if that also complicating things. I hope to try again in the next day or two.
EDIT: spent a few minutes on another attempt today. Setting with gcloud config set
has no effect. Using *_PROXY env vars does seem like it could work but in my case only if there was a way to ensure those env vars were only set by TF when exec'ing the google provider. I can't find a way w/ terragrunt or terraform to only set env vars for some providers.
EDIT 2: Possible path... gcloud
cli respects these vars: CLOUDSDK_PROXY_TYPE=http CLOUDSDK_PROXY_ADDRESS=127.0.0.1 CLOUDSDK_PROXY_PORT=10080 CLOUDSDK_CORE_CUSTOM_CA_CERTS_FILE=/home/joe/.iamlive/ca.pem gcloud projects list
but not sure if the google tf provider does. I've tried a few different ways to get them into the process with no luck thus far.
According to this googleapis/google-api-go-client#221 (comment) the gcp go client would respect the standard HTTP_PROXY env vars and that's about it, which is consistent w/ my experiments so far.
My next step is to try a small fork of the provider to add some method to inject specific proxy settings, either via provider config or special env vars like the CLOUDSDK_*
vars that gcloud
cli recognizes
from iamlive.
SUCCESS! Here's what I had to do:
The key: NO_PROXY
env var.
I was able to get a successful terragrunt plan
run with output from iamlive
using:
NO_PROXY
to exclude everything except googleapis. This takes some trial and error. See mine below for a starting point.HTTPS_PROXY=http://localhost:10080
- Add the
iamlive.ca.pem
to the system trust store since there is no other way that I am aware of to induce the go http stdlib to use alternative CA certs. (on ubuntu/debian:sudo cp ~/.iamlive/ca.pem /usr/local/share/ca-certificates/iamlive.ca.crt && sudo update-ca-certificates
. The file must be suffixed.crt
for update-ca-certs to pick it up)
Trickiest part here was getting NO_PROXY
right and this will be different for everyone, most likely. Here is my final NO_PROXY
:
NO_PROXY=tailscale.com,okta.com,githubusercontent.com,github.com,terraform.io,hashicorp.com,amazonaws.com
Most of these were easy to figure out because tf would fail early with an error about not being able to fetch a plugin from *.terraform.io
or *.hashicorp.com
. A very tricky one was okta.com
however. This one was not obvious at first. Terraform was simply hanging forever on a specific resource data.google_project
in this case. It was not until I started watching the proxy with tcpdump -i any -n -s0 -A port 10080
that I noticed okta was being requested thru the proxy. This is specific to my setup, most likely, and others might not run into the exact case. But if you see terraform hanging (retrying a resource until timeout) I would reach to tcpdump
to figure out what is trying and failing to run thru the proxy.
One thing I found surprising is some of the permissions being reported by iamlive
. For example orgpolicy.policy.{get,set}
. I'm pretty sure these are not being used as I have not given the GSA used by TF any roles that contain those. And there are actually not that many roles that have those perms, eg:
$ gcp-iam-lookup orgpolicy.policy.set
==> Searching for roles containing permissions: [orgpolicy.policy.set] ...
roles/assuredworkloads.admin (21)
roles/assuredworkloads.editor (21)
roles/orgpolicy.policyAdmin (16)
I also saw resourcemanager.projects.undelete
in the list which seems strange on a terraform plan
run. These might be perms that TF is trying to use (probe) and expecting/ignoring failures. Is there a way to get more data on the permissions used, such as the result of the call (succes/failure)?
from iamlive.
@iann0036 I am trying out the latest release with --debug
. I can see a series of POST /v1/projects/<REDACTED>:getIamPolicy?alt=json&prettyPrint=false HTTP/1.1
early in the TF run and one of them results in the orgpolicy.policy.set
perm being added to the list. I can't tell anything different between the calls as the headers and path are all the same.
EDIT: I also see {"options":{"requestedPolicyVersion":3}}
in the body of the getIamPolicy
req but not sure how or why that would matter (probably doesn't?)
I added a debug print in handleGCPRequest()
and this is what I get when the orgpolicy.policy.set
is added:
POST /v1/projects/<REDACTED>:getIamPolicy?alt=json&prettyPrint=false HTTP/1.1
Host: cloudresourcemanager.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/1.5.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/4.84.0
Content-Length: 41
Accept-Encoding: gzip
{"options":{"requestedPolicyVersion":3}}
2023/11/05 20:12:52 API ID:
2023/11/05 20:12:52 API ID:
2023/11/05 20:12:52 API ID:
2023/11/05 20:12:52 API ID:
2023/11/05 20:12:52 API ID:
2023/11/05 20:12:52 API ID:
2023/11/05 20:12:52 API ID:
2023/11/05 20:12:52 API ID:
2023/11/05 20:12:52 API ID: cloudresourcemanager.projects.clearOrgPolicy
[
"iam.serviceAccounts.getOpenIdToken",
"orgpolicy.policy.set",
"resourcemanager.projects.get"
]
from iamlive.
@iann0036 looks great! no more orgpolicy.*
perms listed. This is a fairly big module that I am testing with and there are a lot of resources, but a quick glance of the resulting perms from iamlive looks much more plausible based on what I know to be in the module. ๐
I also ran 1.1.4 against a much smaller module (basically just https://registry.terraform.io/modules/terraform-google-modules/cloud-storage/google/latest/examples/simple_bucket) and the resulting permission set was much better. I had run 1.1.3 against this module and it did not output any storage.*
perms, now it does.
from iamlive.
Related Issues (20)
- Add checksums and/or signature to verify trustworthyness HOT 3
- containerized iamlive proxy doesn't generate --output-file on SIGHUP nor on exit HOT 10
- UnrecognizedClientException: The security token included in the request is invalid HOT 1
- In proxy mode, delegate non-aws calls to system-configured proxy? HOT 2
- error configuring S3 Backend HOT 2
- iamlive command not found HOT 1
- OIDC โiamlive CAโ certificate is not trusted HOT 4
- Certificate generation HOT 2
- Cognito IDP not tracked HOT 2
- Brew support on Linux HOT 2
- Problem with Release 1.1.1 HOT 10
- AWS ECR actions not tracked HOT 3
- GitHub Actions (goreleaser) Builds Inconsistent Failures
- AWS provider tag actions not tracked HOT 1
- Actions are not getting sorted with `--sort-alphabetically` in `--mode proxy`
- GitHub Action dependency `mitchellh/gon` has been archived
- No output from terraform on MacOS HOT 6
- Contributing a guide for deploying iamlive on Kubernetes? HOT 2
- Did a 404 on composite terraform page
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from iamlive.