xen0l/aws-gate

Better AWS SSM Session manager CLI client

aws-gate's Issues

`aws-gate bootstrap` is broken

I upgraded aws-gate and started to get errors. I guess is because, after #41 got pushed, it no longer accept session-manager-plugin to be provided by the host system. However, most of our users already have it installed by other ways.

OSError: session-manager-plugin not found

Trying to aws-gate bootstrap gives error because the needed folder is not yet created

$ aws-gate -v bootstrap
[Errno 2] No such file or directory: '/Users/alanjds/.aws-gate/bin'

After creating manually, the installation can succeed:

$ mkdir -p /Users/alanjds/.aws-gate/bin
$ aws-gate -v bootstrap
1.1.26.0
session-manager-plugin (version None) installed successfully!

After #41, aws-gate no longer accept the plugin to be provided by system.
For curiosity, why is it allowing only owned version to be used?

What about allowing usage of existing one if available, maybe with a deprecation or warning message?

Rewrite documentation on SSH

The current documentation is insufficient in terms of explaining different SSH features in aws-gate. It should contain the following:

• clear explanation when to use aws-gate ssh, aws-gate ssh-proxy and ssh ProxyCommand
• how to use scp with aws-gate and why it's not always the good idea
• why we might not accept functionality to aws-gate ssh which can be easily used with native ssh command

Turn off SSH host key verification

Currently, aws-gate ssh always asks user to verify the fingerprint of the EC2 instance we are connecting to. However, as instances can come and go, this will result in "Host key verification failed" if connecting to the instance with the same identifier.

Add support for SCP

We have a convenience integration of ssh, we should do the same for scp.

Type annotations

aws-gate should have type annotations, so we can do more linting and discover design issues.

Automate updating of Homebrew taps

We should automatically update aws-gate formula repo every time a new version is released.

Error encountered while trying to open session with aws-gate

Hello,

I am fairly new to session manager and came across aws-gate and I believe I have configured things correctly, however when attempting to establish a session to my test instance with aws-gate results in the below. I'm not sure that it matters, but I am using aws-vault to manage my aws credentials. Full debug output is at the bottom.

aws-gate session <instance-name>
----------ERROR-------
Encountered error while initiating handshake. Handshake timed out. Please ensure that you have the latest version of the session manager plugin.

I have ensured that I have the latest version of the session manager plugin on my laptop as well as the agent on the instance I'm connecting to.

I have also verified that I can establish a session with the aws cli with

aws ssm start-session --target <instance-id>

I have tried enabling debug output and the only thing that stood out was this, note the eu-west-1 at the end:

2020-05-13 11:57:43,789 - aws_gate.utils               - DEBUG - Executing "session-manager-plugin {"SessionId": "1589389053774966000-073e2ab90c4664979", "TokenValue": "token", "StreamUrl": "wss://ssmmessages.us-east-1.amazonaws.com/v1/data-channel/1589389053774966000-073e2ab90c4664979?role=publish_subscribe", "ResponseMetadata": {"RequestId": "requestid", "HTTPStatusCode": 200, "HTTPHeaders": {"x-amzn-requestid": "requestid", "content-type": "application/x-amz-json-1.1", "content-length": "646", "date": "Wed, 13 May 2020 16:57:43 GMT"}, "RetryAttempts": 0}} us-east-1 StartSession eu-west-1 {"Target": "instanceid"} https://ssm.us-east-1.amazonaws.com"

I have ensured that nothing in my environment has eu-west-1 configured, so I am not sure where that is coming from or if thats even related to the issue. Any help would be much appreciated.

Full debug output(redacted):

2020-05-13 11:57:42,891 - aws_gate.config              - DEBUG - Located config file: /Users/user/.aws-gate/config
2020-05-13 11:57:42,893 - aws_gate.utils               - DEBUG - Obtaining boto3 session object
2020-05-13 11:57:42,903 - aws_gate.utils               - DEBUG - Obtained configured AWS profiles: default profile
2020-05-13 11:57:42,904 - aws_gate.utils               - DEBUG - Obtaining boto3 session object
2020-05-13 11:57:42,912 - aws_gate.utils               - DEBUG - Obtained configured AWS profiles: default profile
2020-05-13 11:57:42,912 - aws_gate.utils               - DEBUG - Obtaining boto3 session object
2020-05-13 11:57:42,920 - aws_gate.cli                 - DEBUG - aws-vault usage detected, defaulting to the AWS profile from $AWS_VAULT
2020-05-13 11:57:42,920 - aws_gate.cli                 - DEBUG - Using AWS profile "profile" in region "us-east-1"
2020-05-13 11:57:42,921 - aws_gate.utils               - DEBUG - Deferring signal: SIGHUP
2020-05-13 11:57:42,921 - aws_gate.utils               - DEBUG - Deferring signal: SIGINT
2020-05-13 11:57:42,921 - aws_gate.utils               - DEBUG - Deferring signal: SIGTERM
2020-05-13 11:57:42,921 - aws_gate.utils               - DEBUG - Executing "session-manager-plugin --version"
2020-05-13 11:57:42,927 - aws_gate.utils               - DEBUG - Restoring signal: SIGHUP
2020-05-13 11:57:42,928 - aws_gate.utils               - DEBUG - Restoring signal: SIGINT
2020-05-13 11:57:42,928 - aws_gate.utils               - DEBUG - Restoring signal: SIGTERM
2020-05-13 11:57:42,928 - aws_gate.decorators          - DEBUG - session-manager-plugin version: 1.1.61.0 (required version: 1.1.23.0)
2020-05-13 11:57:42,928 - aws_gate.utils               - DEBUG - Obtaining boto3 session object
2020-05-13 11:57:42,937 - aws_gate.utils               - DEBUG - Obtained configured AWS profiles: default profile
2020-05-13 11:57:42,937 - aws_gate.utils               - DEBUG - No entry found in configuration file for host: instance-name
2020-05-13 11:57:42,937 - aws_gate.utils               - DEBUG - Obtaining boto3 session object
2020-05-13 11:57:42,945 - aws_gate.utils               - DEBUG - Obtaining ssm client
2020-05-13 11:57:43,045 - aws_gate.utils               - DEBUG - Obtaining boto3 session object
2020-05-13 11:57:43,056 - aws_gate.utils               - DEBUG - Obtaining ec2 boto3 resource
2020-05-13 11:57:43,169 - aws_gate.query               - DEBUG - Querying EC2 API for instance identifier: instance-name
2020-05-13 11:57:43,169 - aws_gate.query               - DEBUG - Identifier type chosen: name
2020-05-13 11:57:43,530 - aws_gate.query               - DEBUG - Found 1 maching instances
2020-05-13 11:57:43,530 - aws_gate.query               - DEBUG - Matching instance: instance-id
2020-05-13 11:57:43,530 - aws_gate.session             - INFO  - Opening session on instance instanceid (us-east-1) via profile prev-admin
2020-05-13 11:57:43,530 - aws_gate.session_common      - DEBUG - Creating a new session on instance: instance-id (us-east-1)
2020-05-13 11:57:43,788 - aws_gate.session_common      - DEBUG - Received response: {'SessionId': '1589389053774966000-073e2ab90c4664979', 'TokenValue': 'Token', 'StreamUrl': 'wss://ssmmessages.us-east-1.amazonaws.com/v1/data-channel/1589389053774966000-073e2ab90c4664979?role=publish_subscribe', 'ResponseMetadata': {'RequestId': 'requestid', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'request-id', 'content-type': 'application/x-amz-json-1.1', 'content-length': '646', 'date': 'Wed, 13 May 2020 16:57:43 GMT'}, 'RetryAttempts': 0}}
2020-05-13 11:57:43,788 - aws_gate.utils               - DEBUG - Deferring signal: SIGHUP
2020-05-13 11:57:43,788 - aws_gate.utils               - DEBUG - Deferring signal: SIGINT
2020-05-13 11:57:43,788 - aws_gate.utils               - DEBUG - Deferring signal: SIGTERM
2020-05-13 11:57:43,789 - aws_gate.utils               - DEBUG - Executing "session-manager-plugin {"SessionId": "1589389053774966000-073e2ab90c4664979", "TokenValue": "token", "StreamUrl": "wss://ssmmessages.us-east-1.amazonaws.com/v1/data-channel/1589389053774966000-073e2ab90c4664979?role=publish_subscribe", "ResponseMetadata": {"RequestId": "requestid", "HTTPStatusCode": 200, "HTTPHeaders": {"x-amzn-requestid": "requestid", "content-type": "application/x-amz-json-1.1", "content-length": "646", "date": "Wed, 13 May 2020 16:57:43 GMT"}, "RetryAttempts": 0}} us-east-1 StartSession eu-west-1 {"Target": "instanceid"} https://ssm.us-east-1.amazonaws.com"

Starting session with SessionId: 1589389053774966000-073e2ab90c4664979


SessionId: 1589389053774966000-073e2ab90c4664979 :
----------ERROR-------
Encountered error while initiating handshake. Handshake timed out. Please ensure that you have the latest version of the session manager plugin.

2020-05-13 11:58:03,082 - aws_gate.utils               - DEBUG - Restoring signal: SIGHUP
2020-05-13 11:58:03,083 - aws_gate.utils               - DEBUG - Restoring signal: SIGINT
2020-05-13 11:58:03,083 - aws_gate.utils               - DEBUG - Restoring signal: SIGTERM
2020-05-13 11:58:03,083 - aws_gate.session_common      - DEBUG - Terminating session: 1589389053774966000-073e2ab90c4664979
2020-05-13 11:58:03,200 - aws_gate.session_common      - DEBUG - Received response: {'SessionId': '1589389053774966000-073e2ab90c4664979', 'ResponseMetadata': {'RequestId': 'requestid', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'requestid', 'content-type': 'application/x-amz-json-1.1', 'content-length': '53', 'date': 'Wed, 13 May 2020 16:58:02 GMT'}, 'RetryAttempts': 0}}```

Dependency on PyYAML==5.1.2 breaks compatibility with awscli

Currently it seems that awscli depends on PyYAML<=5.1 for Python 3, which means that trying to install aws-gate AND awscli in the same Python environment doesn't work without a manual override.

Perhaps aws-gate should have a relaxed dependency, like eg. PyYAML>=5.1,<5.2 or whatever makes sense for the actual features used?

NOTE: There is a PR for awscli to bump the PyYAML dependency (aws/aws-cli#4355), but it's not there now, and it's so far unclear if it will actually solve this, as right now the aim is 5.1.1, not 5.1.2.

List should be able to output in different formats

For shell instance completion, it is necesearry to have different output format in list. The following formats might be useful: human friendly (default), parseable and JSON.

Add arguments for region and profile

So that it isn't required to use an entry in ~/.aws-gate/config for every managed instance in a multi account environment it would be good if we had optional arguments to pass the region & profile similar to the aws cli.

eg:

aws-gate session --profile staging --region eu-west-2 i-1234567890
aws-gate session --profile production --region eu-west-2 i-1234567890
aws-gate list --profile staging --region eu-west-2

Add option to query by instance name

We should be able to query by instance name.

Querying instances by autoscaling group tag is broken

aws-gate fails to query instances by autoscaling group names. We should fix this and also provide a test case.

Example input: aws:autoscaling:groupName:dummy-v001

Provide ssh proxy capabilities

Session Manager supports tunneling SSH/SCP sessions. We should provide this functionality.

"Invalid profile provided: default" when using environment variables

I'm using environment variables to configure the AWS credentials for aws-gate and don't have a default profile in .aws/config because of that. (I'm using https://github.com/99designs/aws-vault (eg aws-vault exec profile_name -- aws-gate list), but any tool that sets environment variables will work the same).

This leads to the error "Invalid profile provided: default" when running aws-gate (even though valid credentials are available in the environment). I can work around this by adding a dummy default profile in the configuration file, but it would be nice if that wasn't needed.

Provide ZSH tab completion

Once config file support is done, provide ZSH tab completion.

aws-gate can open session to terminated instance

When querying for EC2 instance details, we are not taking into consideration if the instance is running or not. We should only return only running instances.

Move to pytest fixtures

We should adopt using pytest fixtures to simplify the test code.

Always use own version of session-manager-plugin

#18 added support for downloading session-manager-plugin. aws-gate should always use it if possible.

Add bootstrap command

We should have a way to fetch and install session-manager-plugin.

when using mfa, OTP code needs to be entered twice

Hi,

Found your tool and it would cover our use case perfectly. but we are using mfa and every request triggers a new session creation (and also a new mfa request). In this case we need to enter 2 codes to do an ls or a session.

Created a pull request that would solve this by adding the credential cache to it (and pointing it to the same as the cli)

Add support for executing custom documents

Session Manager supports running custom documents that could be mapped to specific command that people need to be able to execute. Some use cases:

• create a thread dump of a Java application
• incident response evidence gathering
• restarting/starting crashed service

We should extend aws-gate to support this either via session or a completely new subcommand. AWS SSM Document creation is out of scope.

Flaky tests

Recently, the test suite started to failed recently because of flaky test. We should investigate deeper and ensure the test suite is reliable.

aws-gate install via PyPI is broken on Linux

Maybe requirements/* is missing from MANIFEST.in on the published PyPI package

Collecting aws-gate==0.3.0 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/75/aa/3afcc4c7f2eea7e5c409cc265d7b0fd9f2ad9b59cdb28aa3f06bd205a221/aws-gate-0.3.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-fbxF6Y/aws-gate/setup.py", line 31, in <module>
        install_requires=get_install_requirements('requirements/requirements.txt'),
      File "/tmp/pip-install-fbxF6Y/aws-gate/setup.py", line 13, in get_install_requirements
        with open(os.path.join(__location__, path), 'r') as f:
    IOError: [Errno 2] No such file or directory: '/tmp/pip-install-fbxF6Y/aws-gate/requirements/requirements.txt'

Add ssh support

aws-gate should be able to directly open SSH session for you (no prior configuration required)

plugin_version decorator doesn't work properly on Python 3.6

A user reported a problem that aws_gate is failing with capture_output=True in plugin_version on Python 3.6. That's possible because capture_output=True was added in Python 3.7.

We should fix this and come up with a better test.

2020-03-20 10:46:33,106 - aws_gate.cli                 - DEBUG - Using AWS profile "saml2aws" in region "eu-west-1"
2020-03-20 10:46:33,106 - aws_gate.utils               - DEBUG - Deferring signal: SIGHUP
2020-03-20 10:46:33,106 - aws_gate.utils               - DEBUG - Deferring signal: SIGINT
2020-03-20 10:46:33,106 - aws_gate.utils               - DEBUG - Deferring signal: SIGTERM
2020-03-20 10:46:33,107 - aws_gate.utils               - DEBUG - Executing "session-manager-plugin --version"
2020-03-20 10:46:33,107 - aws_gate.utils               - DEBUG - Restoring signal: SIGHUP
2020-03-20 10:46:33,107 - aws_gate.utils               - DEBUG - Restoring signal: SIGINT
2020-03-20 10:46:33,107 - aws_gate.utils               - DEBUG - Restoring signal: SIGTERM
Traceback (most recent call last):
  File "/home/david/.local/bin/aws-gate", line 11, in <module>
    main()
  File "/home/david/.local/bin/aws-gate", line 7, in main
    aws_gate.cli.main()
  File "/home/david/.local/lib/python3.6/site-packages/aws_gate/cli.py", line 262, in main
    key_size=args.key_size,
  File "/home/david/.local/lib/python3.6/site-packages/aws_gate/decorators.py", line 30, in plugin_required
    return wrapped_function(*args, **kwargs)
  File "/home/david/.local/lib/python3.6/site-packages/aws_gate/decorators.py", line 38, in wrapper
    version = execute_plugin(["--version"], capture_output=True)
  File "/home/david/.local/lib/python3.6/site-packages/aws_gate/utils.py", line 143, in execute_plugin
    return execute(PLUGIN_NAME, args, **kwargs)
  File "/home/david/.local/lib/python3.6/site-packages/aws_gate/utils.py", line 125, in execute
    result = subprocess.run([cmd] + args, env={"PATH": env}, check=True, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 423, in run
    with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'

Provide HomeBrew package

Integrate aws-gate with HomeBrew for easier distribution on macOS platforms.

Move to Github Actions

The following actions should be moved to Github Actions:

• testing
• PyPI package publishing

Add config file support

We should support config file support similar to what ~/.ssh/config is for ssh.

Listing is slow when there are multiple instances

get_instance_details in aws_gate/utils.py is running one API call per instance_id, which takes a lot of time in cases when there are multiple instances. We could speed this up somehow.,

Add verify command

Verify command should check if session-manager-plugin is installed

Extend aws-gate user agent

aws-gate should have a modified user agent to contain the aws-gate/<version> string. This will allow monitoring of version being used as well as better identity client in CloudTrail.

eu connectivity fails via KMS

Hi, xen0l. Love the tool, use it constantly. Not sure what AWS changed network side, but eu connections are failing KMS handshakes. Pretty straightforward:

# aws-gate version

Package          Version
---------------- ---------
aws-gate         0.9.3

# eu-central-1

> aws-gate session bastioneu
Starting session with SessionId: me@work-some-session-id
SessionId: me@work-some-session-id:
----------ERROR-------
Encountered error while initiating handshake. Handshake timed out. Please ensure that you have the latest version of the session manager plugin.

# us-east-1
> aws-gate session bastion

Starting session with SessionId: me@work-some-session-id
sh-4.2$ exit
Exiting session with sessionId: me@work-some-session-id


# eu via awscli
> aws ssm start-session --target some-instance-id --profile eu --region eu-central-1

Starting session with SessionId: me@work-some-session-id
This session is encrypted using AWS KMS.
sh-4.2$ exit
Exiting session with sessionId: me@work-some-session-id

Add support for aws-vault

aws-vault is used for storing AWS credentials. aws-gate should support aws-vault.

Provide bash completion

aws-gate ships with zsh completion. We should also provide completion for Bash users.

MFA

Dear,

I can connect to an instance using this command for MFA and profile usage:

aws-gate session server1 --profile profile1

However, I get prompted for my MFA token twice when using the Name tag, not when using instance ID. And they need to be different. Which is a pain. How can this be solved?

Thanks,
Tom

Allow SSH tunnelling to other VPC resources

My use case: I have a MySQL RDS instance and an EC2 Amazon Linux 2 instance in a private subnet in a VPC. The EC2 instance can connect to the RDS instance. I want to connect to the RDS instance from my desktop so I can query it using familiar GUI tools such as DBeaver.

The EC2 instance is enabled for EC2 instance connect and Session Manager, so I can use aws-gate ssh to connect to it automatically. However, this only gets me halfway.

I would like to create an SSH tunnel through the EC2 instance from a port on my desktop to the RDS instance. I don't see a way to set that up with aws-gate.

Charlie Belmer published a script that allows me to do what I need. As far as I can tell, it uses the same AWS APIs as aws-gate does to solve the problem, and extends the solution by using ssh's -L option to create a tunnel over which database tools can connect. I've pasted the code from the article below.

Would it be possible to add the -L option to aws-gate ssh?

ssh-keygen -t rsa -f temp -N ''
aws ec2-instance-connect send-ssh-public-key --instance-id i-07cec3c515bcb2e61 --availability-zone us-east-1b --instance-os-user ssm-user --ssh-public-key file://temp.pub
ssh -i temp -N -f -M -S temp-ssh.sock -L 3306:echodb-dev.cju92986bx4i.us-east-1.rds.amazonaws.com:5432 ssm-user@i-07cec3c515bcb2e61 -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" -o ProxyCommand="aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters portNumber=%p"
read -rsn1 -p "Press any key to close session."; echo
ssh -O exit -S temp-ssh.sock *
rm temp*

$ psql -h localhost -p 3306 -U master postgres
Password for user master: 
psql (12.2 (Ubuntu 12.2-1.pgdg19.10+1), server 10.6)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.

postgres=> \q

Improve documentation about IAM roles and EC2 instance setup

We shoud improve the documentation by mentioning:

• the need to install ec2-instance-connect
• providing minimal all IAM permissions to run aws-gate
• provide Athena queries / CloudWatch logs queries for querying access events from CloudTrail

Extend list output

We have a function get_instance_details, which provides us with a lot of information about the instance. We should extend aws_gate list output to include more attributes such as VPC ID, Availability zone, instance IPs and DNS names.

Plugin version comparison should work for all cases

Current version of plugin has version 1.2.7.0. My version is quite old, 1.1.23.0. Under the current logic (remove dots and do a simple comparison), 1.2.7.0 is older than 1.1.23.0.

Document ssh lowercasing behaviour

User reached out that when using aws-gate ssh OpsPublic everything works as expected. However, when you try to use scp OpsPublic.eu-west-1.default, OpenSSH automatically lowercases the host name and it case sensitivess matters. We shoudl document this.

As an alternative, it's possible to use SCP via id, e.g. scp i-123141.eu-west-1.default.

Add support for downloading Linux plugin

aws-gate bootstrap supports downloading session-manager-plugin only on macOS. We should add support for downloading it on Linux distributions as well.

Create a proper documentation

We should have a proper documentation in place, ideally something with mkdocs.

Support AWS-StartInteractiveCommand

If this falls under the scope of #256 then feel free to close it, but this feature would allow me to let my developers open consoles for whatever language they're using, instead of forcing them to navigate the instance filesystem to whatever application they are working on. It also would allow a user to use whatever shell they wanted by default, while still providing the logging that the SSH sessions don't provide. I might take a stab at creating a PR for this myself, but I wanted to let you know that the desire for this existed.

Publish aws-gate Docker container

aws-gate should be available in the form of Docker container. This should be built every time a new release happens.

Support querying instances by autoscaling group name

aws-gate should support querying instance by autoscaling group names:

aws-gate session asg:dummy-v001

Provide MSI for better Windows integration

Since 0.11.1, we are generating self-contained binaries for Windows. However, to make it work, the user has to download the binary, place it in path and this is too much manual effort. It would be great if we built MSI package for Windows, which could take care of everything. However, as I have no required experience with Windows platform nor access to it, this should be provided by the community.

Bootstrapped plugin not found in PATH

The current behaviour is that plugin found in the PATH will be used the first. However, if there is no plugin present in the PATH and we have plugin installed via aws-gate bootstrap we should use it as a last resort.

Distribute .exe file of aws-gate on Windows

My other "pip installed" tools that are installed into %USERPROFILE%\AppData\local\Programs\Python\Python39\Scripts are .exe files that can then easily be invoked on command line.

However in this directory aws-gate is just a script file (no extension), with following content:

#!c:\users\xxx\appdata\local\programs\python\python39\python.exe

import aws_gate.cli


def main():
    aws_gate.cli.main()


if __name__ == '__main__':
    main()

Because it is not an executable, you have to run it with py aws-gate and to be in the same directory as the script. This makes it a bit harder to use.

One example of another tool that installs an exe is: https://github.com/mludvig/aws-ssm-tools - maybe it can be used for inspiration.

Flaky tests

Some tests that use hypthesis are flaky and need to be fixed.

asg querying is still broken

asg querying support fixed and support added in 0.8.3 is broken as we found out that : is an allowed character for tags resulting in broken parsing logic. We should add more tests for this.