nccgroup/pmapper

A tool for quickly evaluating IAM permissions in AWS.

pmapper's Issues

Nodes: Track "Available" Groups

Scenario: Imagine someone has permissions to change their own group memberships? That affects the effective permissions of the nodes.

Solutions:

• Evaluate the iam:*Group* actions while doing queries. Probably slow as hell to do that every time.
• (Planning To Do) Cache which groups can be added/removed during the gathering process as an attribute of the node (list of str).

graph --create command hanging

graph --create churns along for about one hour, continuing to print progress output. Then hangs. The last line of the hanging output is

Found new edge: role/aws-service-role/robomaker.amazonaws.com/AWSServiceRoleForRoboMaker can use Lambda to create a new function with arbitrary code, then pass and access role/service-role/[one of our roles]

Also, when trying to run pmapper repl, Did not find file at: /Users/beacomni/Library/Application Support/com.nccgroup.principalmapper/947682355454

There are many lines of "Found new edge". Is there some upper bound on edges that pmapper can handle? Is there a way to get log output that might suggest why it is hanging?

Thanks.

SSM edge, false positive ?

First, thanks or this great tool, It really help me understand AWS roles dependancies.

However, I tried on my environment the privesc module, and obtained :

user/external-admin can escalate privileges by accessing the administrative principal role/admin-role
   user/external-admin can call ssm:SendCommand to access an EC2 instance with access to role/admin-role

According to Pmapper, as SSM is an edge, the following user can privesc :

Account arn:aws:iam::XXX544221596:user/external-admin Permission Policy :

{
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ds:DescribeDirectories",
                "ec2:DescribeInstanceStatus",
                "ssm:SendCommand",
                "ec2messages:*"
            ],
            "Resource": "*"
        }
    ]
}

Administrative principal role detected as obtainable : arn:aws:iam::XXX544221596:role/admin-role
role/admin-role Permission Policy (AdministratorAccess) :

{
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": "*"
        }
    ]

Trust policy

{
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "ssm.amazonaws.com",
          "ec2.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
}

However, I don't see how SSM can be an edge in this case, is this a false positive ?

As no EC2 instance got the role/admin-role, and as far as I know, an EC2 instance can't get a role from a Principal service (ec2.amazonaws.com), isn't it ?
I tried connecting to an EC2 instance and assume the role admin-role, without success.

Regards.

Optimize Policy Simulation Step

The current behavior of PMapper when evaluating a user's access is to:

1. Grab the username.
2. Grab the Action to check.
3. Grab the Resource to check.
4. Throw these things in a call to iam:SimulatePrincipalPolicy (grouped where possible)

If we inspect the policies attached/inline with the principal, we can skip these calls when we don't find any potentially-matching statements in their policies. This should save boat-loads of time during the graphing phase.

Requires completion of Milestone 1.

Independently Track Access Keys By Activation

See title: IAM Users can have up to two access keys, but the keys can be active or not. We should track the status of the keys (either add a "active key" field to the Node class or just track key objects with full IDs + status... probably the former to avoid breaking changes).

Would need a minor version bump (currently 1.0.X -> 1.1.0).

Feature request: Skip Graph of certain Roles

There are a few roles that we are not able to run iam:SimulatePrincipalPolicy on, due to Permissions boundaries. It would be great if we could skip graphing certain roles either by including them in an exceptions file or via CLI.

@ncc-erik-steringer

Argquery: handle condition (v1.0.0)

The argquery subcommand needs to be loaded up with condition handling.

Optimization: GetAccountAuthorizationDetails

From AWS blog:

With the GetAccountAuthorizationDetails API you can get a snapshot of your IAM entities with a single API call. Previously you had to use a combination of multiple API calls, some of which had to be called multiple times. With your IAM settings in one place you could use the output to monitor your intended IAM settings, store snapshots to understand differences in your IAM settings between points in time, and show IAM settings for auditing purposes.

Somehow I missed this for a long time, but it has made my code much easier. I don't have to paginate through getting roles and connected inline policies, just do a single 10 second API call that returns 3MB of JSON. I wonder if it would streamline things enough such that maybe enhancements like #25 would be less necessary.

It even includes instance profiles so a straight listing of ec2 describe-instances could be linked up without api call pivots. A similar situation would work for Lambda with list-functions.

Considering Availability of Credentials in Querying

Along the lines of #44, maybe we can add a --consider-creds arg to the query subcommands to ignore users with inactive creds (but only as the "root" of a chain of principals since other users would have to create active creds to gain access to them).

Definitely not the default setting for queries tho, since PMapper should focus more on interpreting IAM Policies.

Can't List Graphs Without Valid Creds

Describe the bug
When running pmapper graph --list, PMapper won't run if the current creds aren't valid. This is due to an optimization where we pull a session from parsed args no matter what we're doing in the graph subcommand.

To Reproduce
Steps to reproduce the behavior, please include information on suspected users/roles that are the source of the issue when possible:

1. Use PMapper as normal to obtain a graph.
2. Do something to disable the creds, ensure nothing is available in the default botocore profile, ensure no environment variables with creds are set.
3. Run pmapper graph --list.
4. Observe that the command doesn't execute

Expected behavior
A list of available accounts should print no matter what.

We should also throw in the version of PMapper that generated the graph in the list.

Reorganize Code for Pip (pre-v1)

Need to make this deployable with setup.py.

Offline storage/access to the data collected by PMapper

Is there a way to store the data in a query-able format offline. My goal is to run PMapper on multiple accounts and get access to the data collected so that I can create reports and visualizations.

I am trying to create interactive graphs for IAM roles and policies for which I need to get access to all the IAM data data offline collected for multiple accounts I was thinking of writing a script to get all the needed data when I stumbled upon this tool. Since this is much more advanced than what I wanted to do, is there a way to give access to the json files or even the objects created as part of the tool run?

Reorganize Code for Pip (v1.0.0)

Need to make this deployable with setup.py.

ThrottlingException is not defined

Getting this exception / traceback while running pmapper:

Traceback (most recent call last):
  File "pmapper.py", line 176, in <module>
    main()
  File "pmapper.py", line 50, in main
    handle_graph(parsed)
  File "pmapper.py", line 59, in handle_graph
    graph = pull_graph(parsed.profile)
  File "pmapper.py", line 136, in pull_graph
    enumerator.fillOutGraph()
  File "/home/eth/tools/PMapper/principalmap/enumerator.py", line 32, in fillOutGraph
    checkrunner.runChecks()
  File "/home/eth/tools/PMapper/principalmap/edgeconditions/checkrunner.py", line 50, in runChecks
    edgelist = checker.performChecks(self.session, self.graph.nodes)
  File "/home/eth/tools/PMapper/principalmap/edgeconditions/iamchecks.py", line 44, in performChecks
    resultlist = test_node_access(iamclient, nodeX, ['iam:CreateAccessKey', 'iam:UpdateLoginProfile', 'iam:CreateLoginProfile'], users)
  File "/home/eth/tools/PMapper/principalmap/edgeconditions/util.py", line 59, in test_node_access
    result.extend(_test_less(iamclient, node, action, rlist))
  File "/home/eth/tools/PMapper/principalmap/edgeconditions/util.py", line 80, in _test_less
    except ThrottlingException as ex:
NameError: global name 'ThrottlingException' is not defined
Exception KeyError: KeyError(<weakref at 0x7fa8cabba998; to 'tqdm' at 0x7fa8d3cba110>,) in <bound method tqdm.__del__ of Principals Checked:   2%|###2                                                                                                                                      | 158/6640 [9:08:19<326:52:34, 181.54s/it]> ignored

"Fixed it" with:

diff --git a/principalmap/edgeconditions/cloudformationchecks.py b/principalmap/edgeconditions/cloudformationchecks.py
index 8b3f8f8..437cf76 100644
--- a/principalmap/edgeconditions/cloudformationchecks.py
+++ b/principalmap/edgeconditions/cloudformationchecks.py
@@ -52,7 +52,7 @@ class CloudFormationChecker():
                     try:
                         fullstacks = cfclient.describe_stacks(StackName=item['StackId'])
                         stack_retrieved = True
-                    except ThrottlingException as ex:
+                    except Exception as ex:
                         print('ThrottlingException hit, pausing execution for one second.')
                         time.sleep(1)  # TODO: implement escalate and backoff behavior
 
diff --git a/principalmap/edgeconditions/util.py b/principalmap/edgeconditions/util.py
index 69b241a..033a419 100644
--- a/principalmap/edgeconditions/util.py
+++ b/principalmap/edgeconditions/util.py
@@ -77,7 +77,7 @@ def _test_less(iamclient, node, action, resourceList):
                 ResourceArns=resourceList
             )
             done = True
-        except ThrottlingException as ex:
+        except Exception as ex:
             print('ThrottlingException hit, pausing execution for one second.')
             time.sleep(1)
             # TODO: implement escalate and backoff behavior

Had to use the ugly except Exception catch because I was unable to find the definition for ThrottlingException in boto or botocore.

New Preset: "Self-Aware" of Permissions

An idea for a preset query is to see which users can read their own permissions.

Pull Graph From Scout Output

The ScoutSuite tool, when ran against AWS, pulls enough resource data from the account that it should be possible to construct a Graph in PMapper with it. This might save people time when using both tools.

Analysis (v1.0.0)

The analysis subcommand needs to be implemented. This will dig through the graph data for a given account, then provide a list of findings (Markdown-text and JSON outputs). Findings can include:

• Privilege escalation risks
• Overpermissioned EC2 instances
• Overpermissioned Lambda functions
• MFA evasion risks

Store Attached/Inline Policies

Towards v1.0.0, PMapper should stash the policies attached to principals in order to make evaluation go faster.

REPL (v1.0.0)

Implement the repl subcommand. This subcommand let's users send query after query (query and argquery formatting).

pylint errors

Run pylint on PMapper and got:

$ pylint -E principalmap
No config file found, using default configuration
************* Module principalmap.visualizing
E: 40, 4: Instance of 'Dot' has no 'write_svg' member (no-member)
************* Module principalmap.awsnode
E:106,30: Undefined variable 'trustdoctobj' (undefined-variable)
E:106,72: Undefined variable 'trustdoctobj' (undefined-variable)
E:107,19: Undefined variable 'trustdoctobj' (undefined-variable)
E:108,36: Undefined variable 'trustdoctobj' (undefined-variable)
E:109,27: Undefined variable 'trustdoctobj' (undefined-variable)
E:111,34: Undefined variable 'trustdoctobj' (undefined-variable)
E:112,27: Undefined variable 'trustdoctobj' (undefined-variable)
E:115,36: Undefined variable 'trustdoctobj' (undefined-variable)
E:116,27: Undefined variable 'trustdoctobj' (undefined-variable)
E:118,34: Undefined variable 'trustdoctobj' (undefined-variable)
E:119,27: Undefined variable 'trustdoctobj' (undefined-variable)

This was after fixing #9 , where the exception was not defined. Maybe those are all false positives, but still they need to be checked.

New Feature: Extra Graphing Options

One of the challenges of examining large-scale accounts is that some of the groups of checks take an extensive amount of time. One of the ways that we can save time is skipping specific services and regions during the gathering process at the user's discretion. We're adding the following params for graph create.

• Add --services param to include a comma-separated list of services (so something like --services ec2,lambda can be specified to only run EC2 + Lambda edge-checks).
• Add --regions param to include a comma-separated list of regions (so something like --regions us-east-1,us-west-2 can be specified to only run checks in those regions).

Graph Version Update Mechanism

For the 1.0.X to 1.1.0 transition, we should have a mechanism in place for updating a Graph object that was created with 1.0.X code to a 1.1.0-compatible graph. This code should designed in a neat way to allow us to determine if a graph can be updated in a chain for future versions (so that 1.0.X graphs can be updated to 1.2.0 through 1.1.0, 1.3.0 through 1.2.0 through 1.1.0, etc.).

Fix Logging

Squash all the dprint calls and debug params. Just needs a micro version bump when done.

Connection Refused error 40% through IAM Checks

Describe the bug
./pmapper.py --profile myprofile graph
40% through IAM Checks
ConnectionRefusedError: [Errno 111] Connection refused
Could not connect to the endpoint URL: "https://iam.amazonaws.com/"

I also received a similar error for
./pmapper.py --profile myprofile query "who can do ec2:*"

I am using an sts assume role with mfa. It is possible that the temp creds from the mfa challenge
are expiring. If so, definitely need some checkpointing to run graph in phases if possible. Perhaps a
pull-raw-data phase per resource type and a process data phase?

To Reproduce
Steps to reproduce the behavior, please include information on suspected users/roles that are the source of the issue when possible:

git clone latest PMapper
I have the latest:
commit d38a5de
Author: ncc-erik-steringer [email protected]
Date: Thu Oct 18 18:18:22 2018 -0700
Query Updates: Code cleanup, new functionality (#13)

$ ./pmapper.py --profile myprofile graph

Expected behavior
I expect this is some sort of throttling and there needs to be a more robust handling of errors. Also, eve though it ran for several hours before crashing, I will have to start from scratch as there seems to be no checkpointing mechanism. Perhaps writing results to disk in phases would help.

[+] Pulling info on IAM users and roles, finding admins.
Principals Checked: 100%|#########################################################################################################################################################| 360/360 [03:08<00:00, 1.40it/s]
[+] Finished finding admins.
[+] Started EC2 checks.
Principals Checked: 100%|#########################################################################################################################################################| 360/360 [04:43<00:00, 2.00it/s]
[+] Finished EC2 checks.
[+] Started IAM checks.
Principals Checked: 40%|############################################################7 | 143/360 [19:02<28:56, 8.00s/it]Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 258, in send
decode_content=False,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 343, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 301, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x7f5147135400>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./pmapper.py", line 177, in
main()
File "./pmapper.py", line 51, in main
handle_graph(parsed)
File "./pmapper.py", line 60, in handle_graph
graph = pull_graph(parsed.profile)
File "./pmapper.py", line 137, in pull_graph
enumerator.fillOutGraph()
File "/PMapper/principalmap/enumerator.py", line 32, in fillOutGraph
checkrunner.runChecks()
File "/PMapper/principalmap/edgeconditions/checkrunner.py", line 50, in runChecks
edgelist = checker.performChecks(self.session, self.graph.nodes)
File "/PMapper/principalmap/edgeconditions/iamchecks.py", line 55, in performChecks
resultlist = test_node_access(iamclient, nodeX, ['sts:AssumeRole'], roles)
File "/PMapper/principalmap/edgeconditions/util.py", line 59, in test_node_access
result.extend(_test_less(iamclient, node, action, rlist))
File "/PMapper/principalmap/edgeconditions/util.py", line 77, in _test_less
ResourceArns=resourceList
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 320, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 610, in _make_api_call
operation_model, request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 102, in make_request
return self._send_request(request_dict, operation_model)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 136, in _send_request
success_response, exception):
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 210, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 183, in call
if self._checker(attempts, response, caught_exception):
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 251, in call
caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 317, in call
caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 223, in call
attempt_number, caught_exception)
File "/usr/local/lib/python3.7/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 179, in _get_response
http_response = self._send(request)
File "/usr/local/lib/python3.7/site-packages/botocore/endpoint.py", line 223, in _send
return self.http_session.send(request)
File "/usr/local/lib/python3.7/site-packages/botocore/httpsession.py", line 278, in send
raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://iam.amazonaws.com/"
Exception ignored in: <function tqdm.del at 0x7f51493e3730>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/tqdm/_tqdm.py", line 931, in del
self.close()
File "/usr/local/lib/python3.7/site-packages/tqdm/_tqdm.py", line 1133, in close
self._decr_instances(self)
File "/usr/local/lib/python3.7/site-packages/tqdm/_tqdm.py", line 496, in _decr_instances
cls.monitor.exit()
File "/usr/local/lib/python3.7/site-packages/tqdm/_monitor.py", line 52, in exit
self.join()
File "/usr/local/lib/python3.7/threading.py", line 1029, in join
raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread

Visualization Features

Suggested items to improve visualizations:

• Flag to only visualize priv-escs
• GraphML generation for other visualization and graph tools (Gephi, Cytoscape, NetworkX, igraph, Boost, Graph-tool)

facing the error "Exception KeyError: KeyError(<weakref at 0x7f80ace22ba8; to 'tqdm' at 0x7f80acd64110>,)"

Hi @ncc-erik-steringer

I have installed PMapper and required packages.

When running the command "python pmapper.py --profile Pmapper graph" getting the below error -

Exception KeyError: KeyError(<weakref at 0x7f80ace22ba8; to 'tqdm' at 0x7f80acd64110>,) in <bound method tqdm.del of Princ
ipals Checked: 0%| | 0/115 [00:09<?, ?it/s]> ignored
Traceback (most recent call last):
File "pmapper.py", line 174, in
main()
File "pmapper.py", line 52, in main
handle_graph(parsed)
File "pmapper.py", line 60, in handle_graph
graph = pull_graph(parsed.profile)
File "pmapper.py", line 136, in pull_graph
enumerator.fillOutGraph()
File "/root/PMapper/principalmap/enumerator.py", line 33, in fillOutGraph
checkrunner.runChecks()
File "/root/PMapper/principalmap/edgeconditions/checkrunner.py", line 51, in runChecks
edgelist = checker.performChecks(self.session, self.graph.nodes)
File "/root/PMapper/principalmap/edgeconditions/lambdachecks.py", line 55, in performChecks
change_testresults = testMass(iamclient, nodeX, ['lambda:UpdateFunctionCode', 'lambda:UpdateFunctionConfiguration', 'lambda:InvokeFunction'], functionarns)
File "/root/PMapper/principalmap/edgeconditions/util.py", line 38, in testMass
Marker=response['Marker']
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 586, in _make_api_call
api_params, operation_model, context=request_context)
File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 621, in _convert_to_request_dict
api_params, operation_model)
File "/usr/local/lib/python2.7/dist-packages/botocore/validate.py", line 291, in serialize_to_request
raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Missing required parameter in input: "PolicySourceArn"
Missing required parameter in input: "ActionNames"
Exception in thread Thread-5 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
File "/usr/local/lib/python2.7/dist-packages/tqdm/_monitor.py", line 72, in run
File "/usr/local/lib/python2.7/dist-packages/tqdm/_tqdm.py", line 106, in enter
File "/usr/local/lib/python2.7/dist-packages/tqdm/_tqdm.py", line 99, in acquire
File "/usr/lib/python2.7/threading.py", line 168, in acquire
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

Any idea how this error can be fixed.

Wrong condition?

I might be wrong but I guess this one works better.

diff --git a/principalmap/querying.py b/principalmap/querying.py
index d5a9a20..2fcd74a 100644
--- a/principalmap/querying.py
+++ b/principalmap/querying.py
@@ -26,7 +26,7 @@ def perform_query(input_str, session, graph):
         action = tokens[3]
         if len(tokens) == 4:
             result = test_for_node(session, graph, node, action)
-            if result is None:
+            if result is not None:
                 print_search_result(result, action)
             else:
                 print('Did not find a way for ' + str(node) + ' to do ' + action)

Add Permissions Boundaries Support

https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html

From my reading of the docs, these boundary policies have to allow a request in conjunction with the principal's other policies. Need to add:

• Permission Boundary field to Node objects (breaking change of graph data, we'll need a minor version bump).
• Obtain Permission Boundary data during gathering for graph creation.
• Support for evaluation when the Node has a Permission Boundary set (is not None).
• Test cases for the evaluation work.

New Feature: Visualize Queries, Add Labels/Titles

Currently, visualization is focused on showing privilege escalation. However, we can extend it to show the results of queries. For example, if we have a user that can run an instance with a role that can access S3, we can draw a graph showing the connection from the user to the role. We can color the nodes to show who had access with their existing permissions, and who gains access through other principals.

Additionally, we should look into adding titles and labels for visualization outputs. Something like a legend to explain why nodes have one color or another, and a title showing what the purpose of the visualization is.

Query (v1.0.0)

Need to add the query subcommand to v1.0.0.

This is the more English sentence-oriented query format, such as "who can do with when " or "can do with when "

Access key timeouts / Enhancement request to refresh auth

Greetings,

While testing this against several accounts, I've run into issues with role session limits that I don't have control over. In the case that prompted this request we are forced to use SAML with STS and I'm limited to a maximum of a 3h session limit.

I'm currently using the v1.0.0-dev branch. Is there a way to enable caching of results so that a subsequent re-run would pick up after the last successful API pulls and mapping? Is there a way to defer graph creation until all required data is collected?

Without changes to caching and code flow, if a routine to check for updated credentials were added between API pulls that may do the trick.

Required IAM permission to run pmapper

Hello PMapper Team,
I search in the documentation to find which IAM permissions are required to run PMapper. As I'm security concerned I use a own role and want to grant least privilge permissions to this role

I started with this policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowRead",
            "Effect": "Allow",
            "Action": [
                "sts:Get*",
                "lambda:List*",
                "lambda:Get*",
                "iam:Simulate*",
                "iam:List*",
                "iam:Get*",
                "ec2:Get*",
                "ec2:Describe*",
                "cloudformation:List*",
                "cloudformation:Get*",
                "cloudformation:Describe*"
            ],
            "Resource": "*"
        }
    ]
}

and PMapper didn't show any errors. I checked Cloudtrail and based on the events of this role I created this role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowRead",
            "Effect": "Allow",
            "Action": [
                "sts:GetCallerIdentity",
                "lambda:ListFunctions20150331",
                "lambda:ListFunctions",
                "iam:ListUsers",
                "iam:ListUserPolicies",
                "iam:ListRoles",
                "iam:ListRolePolicies",
                "iam:ListInstanceProfiles",
                "iam:ListGroupsForUser",
                "iam:ListGroups",
                "iam:ListAttachedUserPolicies",
                "iam:ListAttachedRolePolicies",
                "iam:ListAccessKeys",
                "iam:GetRolePolicy",
                "iam:GetPolicyVersion",
                "iam:GetPolicy",
                "cloudformation:DescribeStacks"
            ],
            "Resource": "*"
        }
    ]
}

Still, I see no errors. I wonder if you could confirm this policy and maybe add it to the documentation?

Visualization (v1.0.0)

Need to add the visualize subcommand in the v1.0.0 release.

Crash when user / role is removed

Describe the bug
pmapper.py graph will crash when a user / role is removed during the pull process:

[andres:~/tools/PMapper-fork] [venv] master(+2/-2)* 255 ± python pmapper.py graph
Using profile: default
Pulling data for account 080387373610
Using principal with ARN arn:aws:iam::080387373610:user/infosec..vpcreadonly
[+] Pulling info on IAM users and roles, finding admins.
Principals Checked: 100%|################################################################################################################################################| 6641/6641 [57:31<00:00,  2.41it/s]
[+] Finished finding admins.
[+] Started EC2 checks.
Principals Checked:  30%|###########################################4                                                                                                    | 2006/6641 [32:26<45:42,  1.69it/s]Traceback (most recent call last):
  File "pmapper.py", line 176, in <module>
    main()
  File "pmapper.py", line 50, in main
    handle_graph(parsed)
  File "pmapper.py", line 59, in handle_graph
    graph = pull_graph(parsed.profile)
  File "pmapper.py", line 136, in pull_graph
    enumerator.fillOutGraph()
  File "/home/eth/tools/PMapper-fork/principalmap/enumerator.py", line 32, in fillOutGraph
    checkrunner.runChecks()
  File "/home/eth/tools/PMapper-fork/principalmap/edgeconditions/checkrunner.py", line 50, in runChecks
    edgelist = checker.performChecks(self.session, self.graph.nodes)
  File "/home/eth/tools/PMapper-fork/principalmap/edgeconditions/ec2checks.py", line 56, in performChecks
    ActionNames=['ec2:RunInstances', 'ec2:AssociateIamInstanceProfile', 'iam:CreateInstanceProfile'],
  File "/home/eth/tools/PMapper-fork/venv/local/lib/python2.7/site-packages/botocore/client.py", line 320, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/eth/tools/PMapper-fork/venv/local/lib/python2.7/site-packages/botocore/client.py", line 623, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchEntityException: An error occurred (NoSuchEntity) when calling the SimulatePrincipalPolicy operation: Unable to retrieve specified IAM entity.
Exception KeyError: KeyError(<weakref at 0x7f01e803e890; to 'tqdm' at 0x7f01e9f1ddd0>,) in <bound method tqdm.__del__ of Principals Checked:  30%|###########################################4                                                                                                    | 2006/6641 [32:26<45:42,  1.69it/s]> ignored
[andres:~/tools/PMapper-fork] [venv] master(+2/-2)* 1h31m4s 1 ± 

To Reproduce
Just run pmapper.py graph on an AWS organization where things change: user / role is removed.

Expected behavior
Ignore the user that was deleted and continue with the next.

Using PMapper to map cross-account roles

I'm trying to use search_authorization_for to search for / validate inter and cross-account role assumptions.

Inter-account role assumption

Having a user myuser being able to assume myrole, I can validate the behavior:

principal = graph.get_node_by_searchable_name('user/myuser')
result = search_authorization_for(graph, principal, 'iam:AssumeRole', 'arn:aws:iam::account-id:role/myrole', {'aws:SourceIp': 'x.x.x.x'})
print(result.allowed) # True

Cross-account role assumption

User account1/user can assume a role account2/role.

How would you go about this? I'm not sure it's possible using PMapper. Using graph.get_node_by_searchable_name to search for the source principal in a different account won't work if graph was generated in the context of the target account.

Thank you!

New Findings

The following list are items to add to the analysis component:

• Local instance OS priv-esc with misconfigured instance profile role that has ssm:* or the like.
• Admins without MFA
• Admin actions callable without MFA
• Circular trust in Roles

Graphs cannot be created from AWS accounts with groups created under '/some-path/group-name'

Describe the bug
If you try to create a graph for an account with a group with a path not equal to '/', the graph building process will fail.

To Reproduce
Steps to reproduce the behavior, please include information on suspected users/roles that are the source of the issue when possible:

1. Create a group in an AWS account with the path attribute equal to /test/
2. Execute pmapper graph --create
   Wait for an error.
   Expected behavior
   The graph should succeed to be built.
   ** Remedy**
   I'll attach a pull request to this bug report.

Security Boundaries: Tracking, Preset Query, and Finding(s)

With the ability to tag IAM Users (and Roles?) it would be nice to be able to be able to identify when principals belong to certain clusters and then identify when principals can access other principals outside their cluster. Good for use-cases where people use aws:PrincipalTag conditions.

• Add tags to Nodes
• Add preset (cluster?)

Store Data Per Account, Not Profile

PMapper should stash stuff under ~/.principalmap divided by account, rather than by botocore profile. This'll make it easier to put together x-account-related features.

priv_esc preset incorrect error message on missing principal

Describe the bug
The error message when a principal isn't found says "priv_esc" can't be found, instead of the provided principal name.

To Reproduce

1. $ python pmapper.py --profile securityauditor query "preset priv_esc user/doesntexist"
Could not find a principal matching: priv_esc

Expected behavior
Could not find a principal matching: user/doesntexist

New Feature: --not-admin Parameter for Queries

In this tool, we define admin users as those capable of altering their own permissions. The result of having such a trait is that the calling principal should be able to give themselves any permissions they lack, or create an admin principal and access that. It's effectively unlimited permissions.

When you're querying with an account with Principal Mapper, it's going to report that the admin can do whatever you're querying... every time. That gets a bit redundant and clutters the results.

Adding a --not-admin parameter would improve the usability of PMapper. When running a query with that parameter, it'll skip checking admin principals for any queries that iterate through all the principals in the account ("who can" or "preset").

Validate Installation and Functionality on Target Operating Systems, Python Versions

Gotta make sure this tool works across multiple operating systems and Python versions. The biggest concern is the graph file-storage, since that's OS-specific code and I could only test against Ubuntu 16.

Principal Mapper needs to be tested on the following operating systems and Python versions (but not all combinations of which):

Python

• Python 3.6
• Python 3.7

Operating Systems

• macOS
• Windows 10
• Ubuntu 18.04
• Ubuntu Server 18.04
• Amazon Linux 2

Existence check for 'RoleARN' in stack

Describe the bug
Ran into an issue where 'RoleARN' was not present within stack.

To Reproduce
Ran into this when creating a new graph. Unfortunately I believe this is because of the account that I am using for enumeration.

Expected behavior
It would be nice to test for the existence of 'RoleARN' within stack. I have added a fix to my local copy

PMapper/principalmapper/graphing/cloudformation_edges.py

Lines 115 to 120 in 2aaae80

	relevant_stacks = [] # we'll reuse this for *ChangeSet
	for stack in stack_list:
	if stack['RoleARN'] == node_destination.arn:
	relevant_stacks.append(stack)

FIX:

relevant_stacks = []  # we'll reuse this for *ChangeSet
                for stack in stack_list:
                    if 'RoleARN' in stack:
                        if stack['RoleARN'] == node_destination.arn:
                            relevant_stacks.append(stack)
                    else:                        
                        continue

Throttle Hit When Using Tool

Desired Behavior:

When running the graph subcommand, the tool runs and composes the graph.

Observed Behavior:

When running the graph subcommand, the tool will dump traceback information for one of the running threads. That traceback will include some sort of exception message about hitting a limit and getting throttled. Then, the tool will not finish execution and require smashing Ctrl+C.

Platform:

N/A

Suspected Root Cause:

An over-use of the Simulation API cause a throttle to kick in which leads to the exception being thrown.

Help Wanted

Please confirm if you've hit this issue with a comment. If you're able to disclose the number of principals (users + roles) in your account, that will be helpful too. Additionally, if you have suggestions or information about the planned changes below, please leave a comment.

Resource Policy Support

Resource Policies include an additional Principal element in their statements, which has to match the calling principal for the statement to match. There is rudimentary resource policy handling already, we just need to flesh it out and add tests. We should also add options to the query subcommands to evaluate against a resource policy (string/file/API).

When there's a resource policy involved, the actual authorization check seems to differ between different services. In S3, the resource policy doesn't have to authorize the calling principal if the principal's IAM policies allow the call. This isn't true in KMS or IAM (trust docs). Need to examine the resource referenced in the resource policy to choose if we need the resource policy to allow in addition to the IAM policy or not.

• Resource Policy Evaluation (Local)
• Test cases for local evaluation
• Update query module to add methods or args for handling queries involving resource policies.
• Update query and argquery subcommands with args for including resource policies. (--resource-policy-string, --resource-policy-arn, and --resource-policy-file seem like a good mutually-exclusive group)
• Implement resource policy gathering/caching for S3, KMS, SNS, and SQS at a minimum.
• Implement "fetching" code for --resource-policy-arn parameter, S3, IAM, KMS, SNS, SQS for the bare minimum

Gather and Process MFA Information

Breaking change: track if a given IAM user has an MFA device, we can check for MFA bypasses in analysis with that info.

• Grab MFA information
• Go through edge-identifications to determine if caller has MFA for MFA-required actions
• Add field to Edge objects tracking if MFA is required for the Edge
• Add MFA-related findings to analysis subcommand

v1.0.0-dev Lambda Client Error

Describe the bug
UnrecognizedClientException during Lambda checks

To Reproduce
Steps to reproduce the behavior, please include information on suspected users/roles that are the source of the issue when possible:

1. Checkout branch v1.0.0-dev
2. Use Python 3 (all my dependencies are installed here, so tried to use it instead of 2. Works on master branch)
3. Run the script
   • python3 pmapper.py --profile personal graph --create
   • python3 pmapper.py graph --create
4. Majority of the script works, but then I get a invalid security token on the Lambda checks
   • I tried copying my specific profile information to the default profile to detect if it is losing the profile name at some point, but it seems to have the same behavior

Expected behavior
Lambda to succeed as other services did.

Error Stack

running edge check for service: iam
running edge check for service: lambda
Searching through Lambda-supported regions for existing functions.
Traceback (most recent call last):
File "pmapper.py", line 12, in
sys.exit(main())
File "/REDACTED_HOMEDIR/code/PMapper/principalmapper/main.py", line 168, in main
handle_graph(parsed_args)
File "/REDACTED_HOMEDIR/code/PMapper/principalmapper/main.py", line 188, in handle_graph
graph = principalmapper.graphing.graph_actions.create_new_graph(session, checker_map.keys(), parsed_args.debug)
File "/REDACTED_HOMEDIR/code/PMapper/principalmapper/graphing/graph_actions.py", line 35, in create_new_graph
return gathering.create_graph(session, metadata, service_list, sys.stdout, debug)
File "/REDACTED_HOMEDIR/code/PMapper/principalmapper/graphing/gathering.py", line 40, in create_graph
edges_result = edge_identification.obtain_edges(session, service_list, nodes_result, output, debug)
File "/REDACTED_HOMEDIR/code/PMapper/principalmapper/graphing/edge_identification.py", line 43, in obtain_edges
result.extend(checker_obj.return_edges(nodes, output, debug))
File "/REDACTED_HOMEDIR/code/PMapper/principalmapper/graphing/lambda_edges.py", line 35, in return_edges
for page in paginator.paginate(PaginationConfig={'PageSize': 25}):
File "/REDACTED_HOMEDIR/Library/Python/3.7/lib/python/site-packages/botocore/paginate.py", line 255, in iter
response = self._make_request(current_kwargs)
File "/REDACTED_HOMEDIR/Library/Python/3.7/lib/python/site-packages/botocore/paginate.py", line 332, in _make_request
return self._method(**current_kwargs)
File "/REDACTED_HOMEDIR/Library/Python/3.7/lib/python/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/REDACTED_HOMEDIR/Library/Python/3.7/lib/python/site-packages/botocore/client.py", line 661, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the ListFunctions operation: The security token included in the request is invalid.

Multi-Threaded & Netwrok - Recommendations

Hello Friend,

1. Request to make it multi-threaded program and only one CPU is loaded.

2. Second onbservation - The program requires 10GB speed network speed to work in hours, other wise with below shows the stastics with different network speed with same account and same machine.

Speed - Time taken to complete
5M - 9 Hrs
10M - 7 hours
15M - 5 hrs
10 G - 2 hrs

Requires improvement on network.

Rest all, we were happy running the program. Great work.

Regards,
Anand

Additional Edges: SageMaker

(EDIT: Separating things out into separate issues).

Amazon SageMaker is a service for data scientists. It includes several features, among which include creating compute resources that can access IAM Roles. Therefore we should look for and report any potential ways for users/roles given sagemaker:* + iam:PassRole permissions to access other IAM Roles.

Need to also review if container-related services are necessary.

New Feature: Preset(s) for Listing Connections

I'm going to put together a query preset to do two things:

• Report if one principal is connected to another.
• Report all the principals that a given principal can access.

Something like:

$ ./pmapper query "preset connected user/X role/Y"
userX can access roleY because:
   userX can create an instance profile to access roleY

$ ./pmapper query "preset connected userX"
userX can access roleY because:
   userX can create an instance profile to access roleY
userX can access roleZ because:
   userX can create an instance profile to access roleZ

Feedback wanted:

• Should these presets be separate or combined?
• Is connected a good name for this preset? Is there a better one?