This is the ssh daemon configuration and necessary EC2 instance scripting to enable EC2 Instance Connect. Also included is various package manager configurations for packaging for various Linux distributions.
License: Apache License 2.0
This:
aws-ec2-instance-connect-config/src/bin/eic_harvest_hostkeys
Lines 18 to 20 in d998839
doesn't implement the support for Nitro instances in eic_curl_authorized_keys at
aws-ec2-instance-connect-config/src/bin/eic_curl_authorized_keys
Lines 30 to 40 in d998839
As a result, when eic_harvest_hostkeys is run on a Nitro instance, it immediately hits the exit 0 condition and never uploads its public host keys.
The proposal to make the service more isolated from the system came up during reviewing the package for inclusion in the "main" Ubuntu package set:
https://bugs.launchpad.net/ubuntu/+source/ec2-instance-connect/+bug/1835114/comments/5
There are a few options which could be enabled in the .service file which could make it more secure:
https://www.redhat.com/sysadmin/mastering-systemd
aws ec2-instance-connect send-ssh-public-key (in the AWS CLI) lets me upload an SSH for users that do not exist. This makes sense and is good.
I would like to be able to have those users created on-login, though (according to a provided user template, perhaps).
Our use case is to allow developers to come and go from the team without needing to manually create accounts for them on our fleet of hundreds of instances to grant them the ability to login to those instances as required.
so ec2-instance-connect use http://169.254.169.254/latest/meta-data/managed-ssh-keys/signer-ocsp to check the OCSP, the instance metadata usually gaves the following in normal
06B25927C42A721631C1EFD9431E648FA62E1E39
8484DD8720EC8B1747641469A796F913E9AEDE33
917E732D330F9A12404F73D8BEA36948B929DFFC
which is correct, but in one of our machine, the metadata server accidentally gaves
0A8E2B0EFF0E33554642700839AB84B1700A95FD3AAEE9F529828E38DC7A0A6D
28689B30E4C306AAB53B027B29E36AD6DD1DCF4B953994482CA84BDC1ECAC996
87DCD4DC74640A322CD205552506D1BE64F12596258096544986B4850BC72706
F55F9FFCB83C73453261601C7E044DB15A0F034B93C05830F28635EF889CF670
which all become SHA256
but the ec2-instance-connect only verify by SHA-1(https://github.com/aws/aws-ec2-instance-connect-config/blob/master/src/bin/eic_parse_authorized_keys#L53)
This failure cause all SSH connection made by ec2-instance-connect failed
not sure what is going on
We haven't changed any settings, not sure whether ec2-instance-connect need to fix it, or there's some button on Amazon console that can change the digest type
AWS instance connect (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html) enhanced the security of ssh connections.
This instance connect config connect to the imds in the "v1" way:
curl_cmd="/usr/bin/curl -s -f -m 1"
# Verify the instance ID itself
instance=$(eval "${curl_cmd}" "http://169.254.169.254/latest/meta-data/instance-id/")Instead it should do
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
curl_cmd='/usr/bin/curl -H "X-aws-ec2-metadata-token: $TOKEN" -s -f -m 1'
# Verify the instance ID itself
instance=$(eval "${curl_cmd}" "http://169.254.169.254/latest/meta-data/instance-id/")I tried to create a patch, but because the development environment is not described I tried with amazon linux 2, but the integration tests does not run. There is also a region "us-east-1" hard coded instead of $region.
How to reproduce:
- aws ec2 modify-instance-metadata-options --instance-id {{.INSTANCEID}} --http-endpoint enabled --http-token requiredmssh i-0416fa9ebc6824ed0
[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).Version: ec2-instance-connect-1.1-11.amzn2.noarch
I have some EC2 instances that routes to the internet via a web proxy. How can I configure the ec2-instance-connect-harvest-hostkeys.service to use the proxy? I do not see mention of that.
Hi,
I am trying to connect to my EC2 instance i have downloaded the pemfile. However this is a windows box
when i click get password it takes more than 4 times for it to show the password.
I have check the IAM settings it has got complete rights..
please find screenshots for the same
I am not sure where to post this
Regards
Sudeep
When running the following command on mac os:
mssh -d -r eu-west-2 i-XXXXXXXXXXXXXXX
I get the following output
2021-07-24 19:15:58,821 - botocore.httpsession - DEBUG - Certificate path: /usr/local/lib/python3.9/site-packages/botocore/cacert.pem
2021-07-24 19:15:59,629 - botocore.parsers - DEBUG - Response headers: {'x-amzn-RequestId': 'cec3279a-c220-45b7-b99b-930c0a7a8fe0', 'Content-Type': 'application/x-amz-json-1.1', 'Content-Length': '67', 'Date': 'Sat, 24 Jul 2021 23:15:59 GMT'}
2021-07-24 19:15:59,629 - botocore.parsers - DEBUG - Response body:
b'{"RequestId":"cec3279a-c220-45b7-b99b-930c0a7a8fe0","Success":true}'
2021-07-24 19:15:59,629 - botocore.hooks - DEBUG - Event needs-retry.ec2-instance-connect.SendSSHPublicKey: calling handler <botocore.retryhandler.RetryHandler object at 0x1139e8e80>
2021-07-24 19:15:59,629 - botocore.retryhandler - DEBUG - No retry needed.
2021-07-24 19:15:59,631 - EC2InstanceConnect - DEBUG - Successfully pushed the public key to i-XXXXXXXXXXXXXXX
2021-07-24 19:15:59,632 - EC2InstanceConnect - DEBUG - Generated command: ssh -o "IdentitiesOnly=yes" -i /var/folders/vk/11pm_tqj4llb5h4j36cqp7480000gn/T/tmpsb6y5b8t [email protected]
[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
2021-07-24 19:16:03,205 - EC2InstanceConnect - DEBUG - Deleting the private key file: /var/folders/vk/11pm_tqj4llb5h4j36cqp7480000gn/T/tmpsb6y5b8tYou can see the final lines suggest the temporary key was successfully pushed, yet get permission denied after that. The time sync solution reported by others did not solve my problem.
This is rather strange since I am able to mssh into other instances, all of which are Amazon Linux 2. I am doing all of this from a mac.
Any help would be really appreciated :)
This package is linked from https://github.com/widdix/aws-ec2-ssh as a replacement:
June 2019: Check out Amazon EC2 Instance Connect as a replacement for this project
However there's one component I don't see here - the bulk import of IAM users into local users onto the AWS linux instance.
Is there some other mechanism that is intended to handle that function, or is this package intended solely to provide access using a single user ID, ie: ubuntu or ec2-user depending on your linux variant?
I've got a webserver running AL2 which needs ssh access to ubuntu instances (that are launched by the webserver)
$ mssh ubuntu@i-0e523f322c8c4a31b -r eu-central-1
Permission denied (publickey).
Upon further inspection of the debug flag:
2021-03-14 16:02:23,979 - botocore.httpsession - DEBUG - Certificate path: /usr/local/lib/python3.7/site-packages/certifi/cacert.pem
2021-03-14 16:02:24,302 - botocore.parsers - DEBUG - Response headers: {'x-amzn-RequestId': 'f740e31a-d9ff-4976-b907-d3ecbefd2877', 'Content-Type': 'application/x-amz-json-1.1', 'Content-Length': '67', 'Date': 'Sun, 14 Mar 2021 16:02:24 GMT'}
2021-03-14 16:02:24,302 - botocore.parsers - DEBUG - Response body:
b'{"RequestId":"f740e31a-d9ff-4976-b907-d3ecbefd2877","Success":true}'
2021-03-14 16:02:24,302 - botocore.hooks - DEBUG - Event needs-retry.ec2-instance-connect.SendSSHPublicKey: calling handler <botocore.retryhandler.RetryHandler object at 0x7f3fbb0825d0>
2021-03-14 16:02:24,303 - botocore.retryhandler - DEBUG - No retry needed.
2021-03-14 16:02:24,304 - EC2InstanceConnect - DEBUG - Successfully pushed the public key to i-0e523f322c8c4a31b
2021-03-14 16:02:24,304 - EC2InstanceConnect - DEBUG - Generated command: ssh -o "IdentitiesOnly=yes" -i /tmp/tmps7nrux4d [email protected]
Permission denied (publickey).
2021-03-14 16:02:24,407 - EC2InstanceConnect - DEBUG - Deleting the private key file: /tmp/tmps7nrux4d
I can make the full debug log available if it helps. I get a similar result when I do this process "manually" via the aws ec2-instance-connect CLI.
Hi,
I was reviewing the general concepts of EIC a while ago and wanted to now file this report for discussion here. Because one thing I started to wonder was about was what would happen if you have e.g. remotely driven automation that might have hundreds of ssh calls per second.
Obviously one could say "push a script to the system and execute that" but many automation solutions just don't work that way. In that context EIC will work like an amplifier which means every of those ssh logins will trigger a multitude of curl calls each adding latency and overhead.
I was wondering if it would seem reasonable to you to rate-limit this.
You could use timestamps and only re-check everything once every x seconds.
The first login won't find a timestamp and has to work it out, but every later login for some time doesn't need to do the same work over and over again.
That could help scalability and drop overhead a lot at almost no loss IMHO.
I have not found a "I already got my Auth-data, fast-path-skip" in the code - if there is one that I missed please just let me know and consider this almost resolved :-)
In this section of the code curl is called with -f which causes it to return exit code 22 when the HTTP response code is >= 400. This means that when the user doesn't have any keys for example, and the metadata service returns 404, the script exits with code 22 due to the set -e in the script.
I discovered this when calling eic_run_authorized_keys as an additional fallback for my AuthorizedKeysCommand script, I was executing:
exec /usr/share/ec2-instance-connect/eic_run_authorized_keys "${@}"
Which would then cause my own script to return exit code 22, and thus no keys produced by the calling script were accepted at all either due to the non-zero code.
I am filing a bug because the logic in the script shows an intention for this to be an exit 0 scenario.
I'm trying to build a Debian packages, every time I run the cmd "make deb", I got the following error :
Now signing changes and any dsc files...
signfile dsc ec2-instance-connect_1.1.14.dsc Paul Oh <[email protected]>
gpg: skipped "Paul Oh <[email protected]>": No secret key
gpg: /tmp/debsign.8viC9z6S/ec2-instance-connect_1.1.14.dsc: clear-sign failed: No secret key
debsign: gpg error occurred! Aborting....
debuild: fatal error at line 1112:
running debsign failed
Commit 4374d3a removed hostkey harvesting, but the file /src/bin/eic_harvest_hostkeys was not removed.
This causes the rpm build process to fail with the following error:
Checking for unpackaged file(s): /usr/lib/rpm/check-files
/root/aws-ec2-instance-connect-config/rpmbuild/BUILDROOT/ec2-instance-connect-1.1-18.x86_64error: Installed (but unpackaged) file(s) found:/opt/aws/bin/eic_harvest_hostkeys
Removing the /src/bin/eic_harvest_hostkeys file resolved this.
I can't get EC2 instance connect to work on Ubuntu 22.04 due to what seems to be an issue with trust chain verification.
To reproduce:
ami-0aeb7c931a5a61206 in us-east-2).mssh ubuntu@<instance-id>.When using the --debug flag with mssh, I see that the public key is pushed successfully, but the remote rejects the connection:
...
2022-05-06 09:10:58,549 - EC2InstanceConnect - DEBUG - Successfully got instance information from EC2 API for <instance-id>
...
2022-05-06 09:10:59,189 - EC2InstanceConnect - DEBUG - Successfully pushed the public key to <instance-id>
2022-05-06 09:10:59,190 - EC2InstanceConnect - DEBUG - Generated command: ssh -o "IdentitiesOnly=yes" -i /var/folders/30/xdglsm2j3tz1rn1n7yygtm7c0000gn/T/tmp33a253uf ubuntu@<ip>
ubuntu@<ip>: Permission denied (publickey).
2022-05-06 09:10:59,612 - EC2InstanceConnect - DEBUG - Deleting the private key file: /var/folders/30/xdglsm2j3tz1rn1n7yygtm7c0000gn/T/tmp33a253uf
On the instance side, the following error is logged:
AuthorizedKeysCommand /usr/share/ec2-instance-connect/eic_run_authorized_keys ubuntu SHA256:wiFxouWj6qQ0aUZ0CAcftWZqNEf3qj2LLicCfGFcQJY failed, status 2
When I make these changes to /usr/share/ec2-instance-connect/eic_parse_authorized_keys:
@@ -17,8 +17,6 @@
# Openssl to use provided as $1
# Signer public key file path provided as $2
-set -e
-
# Set umask so only we can touch temp files
umask 077
@@ -165,7 +163,7 @@
# Verify the trust chain
if [ -d "${ca_path}" ] ; then
- verify_out=$("${OPENSSL}" verify -x509_strict -CApath "${ca_path}" -CAfile "${tmpdir}/ca-trust.pem" "${tmpdir}/cert.pem")
+ verify_out=$("${OPENSSL}" verify -x509_strict -CApath "${ca_path}" -CAfile "${tmpdir}/ca-trust.pem" "${tmpdir}/cert.pem" 2>&1)
verify_status=$?
else
# If the CA path is not a directory then do not use it - openssl will throw errors on versions 1.1.1+
@@ -173,7 +171,8 @@
verify_status=$?
fi
if [ $verify_status -ne 0 ] || [ "${verify_out}" != "${tmpdir}/cert.pem: OK" ] ; then
- fail "${is_debug}" "EC2 Instance Connect could not verify the signer trust chain. No keys have been trusted."
+ fail "${is_debug}" "EC2 Instance Connect could not verify the signer trust chain. No keys have been trusted.
+$verify_out"
fi
# Verify no certificates have been revoked... the actual error is revealed:
EC2 Instance Connect could not verify the signer trust chain. No keys have been trusted.
C = US, O = "Starfield Technologies, Inc.", OU = Starfield Class 2 Certification Authority
error 89 at 4 depth lookup: Basic Constraints of CA cert not marked critical
C = US, O = "Starfield Technologies, Inc.", OU = Starfield Class 2 Certification Authority
error 92 at 4 depth lookup: CA cert does not include key usage extension
error /dev/shm/eic-l8wtfU3e/cert.pem: verification failed
It seem like the issue is either with mssh generating invalid keys or with ec2-instance-connect performing a trust verification that is too strict.
Because of the combination of shell option set -e and curl -f, curl returns 22 when HTTP response is 4xx or 5xx, and eic_curl_authorized_keys stops immediately, and makes the following error in /var/log/secure.
error: AuthorizedKeysCommand /opt/aws/bin/eic_run_authorized_keys ...(snip)... failed, status 22
ec2-instance-connect-1.1-12.amzn2.noarch.grep error /var/log/secureMay 8 19:03:03 ip-172-31-46-138 sshd[2565]: error: AuthorizedKeysCommand /opt/aws/bin/eic_run_authorized_keys ec2-user SHA256:g7uVwGWoozM/n3GrFbjJ8kYyIF7A7UawA3nbRQDxbxk failed, status 22
May 8 19:12:32 ip-172-31-46-138 sshd[2552]: error: AuthorizedKeysCommand /opt/aws/bin/eic_run_authorized_keys ec2-user SHA256:g7uVwGWoozM/n3GrFbjJ8kYyIF7A7UawA3nbRQDxbxk failed, status 22
May 8 19:13:30 ip-172-31-46-138 sshd[2629]: error: AuthorizedKeysCommand /opt/aws/bin/eic_run_authorized_keys ec2-user SHA256:g7uVwGWoozM/n3GrFbjJ8kYyIF7A7UawA3nbRQDxbxk failed, status 22
We have an issue with using ec2-instance-connect v1.1.11 on a CentOS 7.8.2003 server since a requirement to upgrade openssl to v1.1.1g
We used this guide to update openssl: https://cloudwafer.com/blog/installing-openssl-on-centos-7/
Before this update our openssl was on v1.0.2 and ec2-instance-connect all was working correctly. Since then we can see that the key is being successfully pushed up in to the instance metadata but when we try to connect we are denied and get the following error in /var/log/secure
Jun 22 14:42:54 ip-x-x-x-x sshd[6745]: error: AuthorizedKeysCommand /opt/aws/bin/eic_run_authorized_keys <username> SHA256:FOZI43gB/86B9VuWfghrhWHPExPq4IXHzmWm8fFYYmA failed, status 1
We do have SELinux enforcing but are using the recommended policy #2 to allow. We have tested setting SELinux to permissive but we still encounter the same issue.
I’ve also tried updating to v1.1.12 of ec2-instance-connect but still no joy.
I note that there is a comment here
RE openssl v1.1.1 but not sure if this is related.Has anyone else had an issue using this with openssl v1.1.1?
I have installed "ec2-install-connect" on my ubuntu16.
Got "Permission denied (publickey)" after "Successfully pushed the public key", but I don't know how to fix it.
I was accidentally successful fixed the setup in similar situation, but I couldn't fix it again even try similar steps to fix.
Could you give me some suggestions about this situation?
ubuntu@ip-192-168-17-129:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS"
ubuntu@ip-192-168-17-129:~$ uname -a
Linux ip-192-168-17-129 4.4.0-1077-aws #87-Ubuntu SMP Wed Mar 6 00:03:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ip-192-168-17-129:~ $ dpkg -l|grep ec2-instance-connect
ii ec2-instance-connect 1.1.12+dfsg1-0ubuntu3~16.04.0 all Configures ssh daemon to accept EC2 Instance Connect ssh keys
Try to connect with this instance by
mssh -r us-east-1 -d -u <my_profile_name> ubuntu@<my-instance-id>
The result
...
2020-07-31 15:51:04,700 - botocore.hooks - DEBUG - Event needs-retry.ec2-instance-connect.SendSSHPublicKey: calling handler <botocore.retryhandler.RetryHandler object at 0x7f7306ec29b0>
2020-07-31 15:51:04,700 - botocore.retryhandler - DEBUG - No retry needed.
2020-07-31 15:51:04,701 - EC2InstanceConnect - DEBUG - Successfully pushed the public key to
2020-07-31 15:51:04,701 - EC2InstanceConnect - DEBUG - Generated command: ssh -i /tmp/tmpb_f6_tmn ubuntu@
Ubuntu 16.04.6 LTS
Permission denied (publickey).
2020-07-31 15:51:05,205 - EC2InstanceConnect - DEBUG - Deleting the private key file: /tmp/tmpb_f6_tmn
Hello, the following sentence in the README.md appears to have been left incomplete:
It then reads the ssh host keys on the machine, then creates and signs an AWS Signature Version 4 Request to
Thanks
It looks like ec2-instance-connect is available from the Ubuntu package repositories, but it's not available in Debian.
I was wondering why the packaging was specifically added only to Ubuntu and not to Debian first?
This would make it available for both distributions as well as other Debian-derived ones.
Would you consider working with the Debian project to publish the package in the Debian repositories?
When we upgraded the ec2-instance-connect package from 1.1.9 to 1.1.12 on Ubuntu 16.04 and 18.04, ec2-instance-connect overrode our AuthorizedKeysCommand and AuthorizedKeysCommandUser.
For our use case we want to support two sets of AuthorizedKeysCommand
I believe it was this commit which corrected the systemd service file that introduced this issue for us.
Here is what our sshd_config looked like (some values replaced).
AuthorizedKeysCommand /usr/bin/<some_other_key_command>
AuthorizedKeysCommandUser nobody
AllowGroups <our list of groups that are allowed with some_other_key_command>
AuthenticationMethods publickey
Match Group ec2i
AllowGroups <ec2i>
AuthorizedKeysCommand /usr/share/ec2-instance-connect/eic_run_authorized_keys %u %f
AuthorizedKeysCommandUser ec2-instance-connect
auth.logWhen attempting our default login path (non-ec2i) we could see that sshd was attempting to use the ec2 instance connect commands to look up authorized keys.
2020-03-02T16:34:40.896366+00:00 ip-192-168-1-1 sshd[1234]: Connection closed by authenticating user my_user 1.1.1.1 port 63090 [preauth]
2020-03-02T16:35:37.982098+00:00 ip-192-168-1-1 sshd[4567]: AuthorizedKeysCommand /usr/share/ec2-instance-connect/eic_run_authorized_keys my_user SHA256:Ahje+...
$ sudo service sshd status
● ssh.service - OpenBSD Secure Shell server
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
Drop-In: /lib/systemd/system/ssh.service.d
└─ec2-instance-connect.conf
Active: active (running) since Mon 2020-03-02 20:51:48 UTC; 1min 13s ago
Main PID: 1744 (sshd)
Tasks: 9
Memory: 141.8M
CPU: 2.828s
CGroup: /system.slice/ssh.service
├─1744 /usr/sbin/sshd -D -o AuthorizedKeysCommand /usr/share/ec2-instance-connect/eic_run_authorized_keys %u %f -o AuthorizedKeysCommandUser ec2-instance-con
....
We have downgraded and pinned version 1.1.9. This restored our ability to SSH via our directory backed AuthorizedKeysCommand.
$ sudo service sshd status
● ssh.service - OpenBSD Secure Shell server
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-03-02 17:43:31 UTC; 3s ago
Process: 8539 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=0/SUCCESS)
Main PID: 8540 (sshd)
Tasks: 8 (limit: 4915)
CGroup: /system.slice/ssh.service
├─8540 /usr/sbin/sshd -D
....
sshd through the service configAuthorizedKeysCommand is respected.AuthorizedKeysCommand on install. Instead print a message that tells users how to configure sshd_config for themselves.Hello,
I have written two custom Config rules that were working without any issues over the last few weeks. Yesterday, when trying to re-evaluate them, I get the "No Results Reported" error.
I rebuilt the custom rules from scratch and now Config is stuck in "Evaluating..." mode.
Is there a current issue with Config that we are unaware of? I did not change anything from when the rules ran previously with no issue.
BTW, I have posted this here because I couldn't find a dedicated AWS-Config repository. Thank you!
Although EC2 Instance Connect is technically supported on Mac instances, neither this project is compatible with macOS nor a package for installing it on macOS exists. Without key pairs, the only other way to connect to a Mac instance is via SSM.
https://github.com/aws/aws-ec2-instance-connect-config/blob/master/bin/make_rpm.sh
Makefiles are considered the defacto standard for this sort of operation. Please provide makefiles for packaging purposes.
Hi, I'm trying to install and run ec2-instance-connect on a ubuntu 18.04 machine of type m4.2xlarge. I did sudo apt-get install ec2-instance-connect and got the following error:
And do systemctl status ec2-instance-connect it shows:
I ran the same commands to setup ec2-instance-connect on other machines with same ami but different instance type(m5) and succeeded. I'm wondering if the SSL problem could be an instance type related issue, or there's some other aspects I need to investigate.
Thank you very much
Our compliance requirements include need for signed RPMs that are compatible with SELinux in targeted enforcement-mode. Had put together a process to download SRPMs from the Amazon Linux 2 AMIs whenever the "new AMI" SNS notice would hit. Found that ec2-instance-connect SRPM started showing up in the 20190612 AMI. After repackaging for Enterprise Linux 7, found that service wasn't working with our AMIs. After some investigation, determined issue was lack of SELinux compatibility. Used the AVC denials to put together a provisional SELinux exclusion.
Currently am using the following in my (ad hoc) UserData to allow use of the ec2-instance-connect (but would obviously prefer this just live in the RPM):
# Write an uncompiled SELinux type-definition to disk
cat > /root/instance-connect.te << EOF
module instance-connect 1.0;
require {
type ssh_keygen_exec_t;
type http_port_t;
type sshd_t;
class process setpgid;
class tcp_socket name_connect;
class file { execute execute_no_trans open read };
}
#============= sshd_t ==============
#!!!! This avc can be allowed using one of the these booleans:
# authlogin_yubikey, nis_enabled
allow sshd_t http_port_t:tcp_socket name_connect;
allow sshd_t self:process setpgid;
allow sshd_t ssh_keygen_exec_t:file { execute execute_no_trans open read };
EOF
# Compile and SELinux module for EC2 Instance Connect
checkmodule -M -m -o /root/instance-connect.mod /root/instance-connect.te
semodule_package -o /root/instance-connect2.pp -m /root/instance-connect.mod
semodule -i /root/instance-connect2.pp
Am looking to submit a PR, but wanted to know if the above (the policy-def in the HERE document; the rest is just a way to activate it outside of a managed packaging) is sufficient. The PR would include the compiled policie files and appropriate %post script (updates) to ensure they're activated. Is there a preference on priority level that they should be activated to (i.e., is 400 ok, or would there be a preference for a different level?).
There is nothing happening at build time that requires systemd, so building the RPM should not require systemd to be installed. See https://github.com/aws/aws-ec2-instance-connect-config/blob/master/rpmsrc/SPECS/generic.spec#L25
Hi !
So I'm using ec2-instance-connect heavily as it's easy and effective way to gain access to EC2 instances.
I'm trying though to limit egress for EC2 instances. I've created com.amazonaws.us-east-1.ec2messages and com.amazonaws.us-east-1.ec2 VPC endpoints as I thought this might be the cure for my problem but well no luck.
When I'm booting new instance it fails on pushing data to https://ec2-instance-connect.us-east-1.amazonaws.com/PutEC2HostKeys/. I can fix this with giving temporary access to 0.0.0.0/0 while creating instance (asVPC egress is not enough) but well I would have to remember this and I don't want to.
Is there some secret sauce for this ?
Are there plans to provide full support for CentOS? I'm able to build and install an RPM cleanly onto a centos7 EC2 instance but nothing seems to work:
<snipped>
2019-12-23 21:04:06,529 - botocore.retryhandler - DEBUG - No retry needed.
2019-12-23 21:04:06,532 - EC2InstanceConnect - DEBUG - Successfully pushed the public key to i-**************
2019-12-23 21:04:06,532 - EC2InstanceConnect - DEBUG - Generated command: ssh -i /var/folders/zv/sdp0xvqd2pnds9sr_qsrdc0r0000gp/T/tmp90gvm1gc centos@***.***.***.***
**centos@***.***.***.***: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).**
2019-12-23 21:04:07,248 - EC2InstanceConnect - DEBUG - Deleting the private key file: /var/folders/zv/sdp0xvqd2pnds9sr_qsrdc0r0000gp/T/tmp90gvm1gc
When attempting to package the ec2-instance-connect on a debian-based linux machine, I am receiving the following error message:
Do need to replace and add a gpg key somewhere or can I --recv-key from somewhere? This is occurring when I run the 'make deb' command to package the debian version of ec2-instance-connect.
Thanks in advance,
Michael
Running shellcheck on latest master:
$ shellcheck src/bin/eic_*
In src/bin/eic_curl_authorized_keys line 90:
keys_status="$(/usr/bin/curl -s -f -m 1 -H "${IMDS_TOKEN_HEADER}" -o /dev/null -I -w %{http_code} "${IMDS}/managed-ssh-keys/active-keys/${1}/")"
^-- SC1083: This { is literal. Check expression (missing ;/\n?) or quote it.
^-- SC1083: This } is literal. Check expression (missing ;/\n?) or quote it.
In src/bin/eic_harvest_hostkeys line 54:
/bin/echo -n "${1}" | /usr/bin/sha256sum | /bin/sed 's/\s.*$//'
^-- SC2039: In POSIX sh, echo flags are undefined.
In src/bin/eic_harvest_hostkeys line 66:
base="$(/bin/echo -n "AWS4${1}" | /usr/bin/od -A n -t x1 | /bin/sed ':a;N;$!ba;s/[\n ]//g')"
^-- SC2039: In POSIX sh, echo flags are undefined.
In src/bin/eic_harvest_hostkeys line 74:
creds_status="$(/usr/bin/curl -s -f -m 1 -H "X-aws-ec2-metadata-token: ${IMDS_TOKEN}" -o /dev/null -I -w %{http_code} "http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance/")"
^-- SC1083: This { is literal. Check expression (missing ;/\n?) or quote it.
^-- SC1083: This } is literal. Check expression (missing ;/\n?) or quote it.
In src/bin/eic_harvest_hostkeys line 151:
payloadHash=$(/bin/echo -n "${val}" | /usr/bin/sha256sum | /bin/sed 's/\s.*$//')
^-- SC2039: In POSIX sh, echo flags are undefined.
In src/bin/eic_harvest_hostkeys line 154:
requestHash=$(/bin/echo -n "${canonicalRequest}" | /usr/bin/sha256sum | /bin/sed 's/\s.*$//')
^-- SC2039: In POSIX sh, echo flags are undefined.
For more information:
https://www.shellcheck.net/wiki/SC1083 -- This { is literal. Check expressi...
https://www.shellcheck.net/wiki/SC2039 -- In POSIX sh, echo flags are undef...
I recognize that those are minor issues, but making the scripts passing shellcheck can help in automated QA.
My EC2 machine is inside a private subnet, and API calls are not allowed, except if a VPC endpoint was configured. Is there a VPC endpoint for the API call https://ec2-instance-connect.{region}.amazonaws.com/PutEC2HostKeys/ ?
When the first draft of this scripting was written, it was much more minimal - do a single read from EC2 Instance Metadata service, parse out timestamp and key body, and return unexpired keys.
As time has gone on and security pieces have been added - instance ID checks, key signing, signing certificate verification, verification the script is running on an instance - this has drastically increased in complexity to the point where shell scripting is no longer appropriate. At this point, both internal to Amazon and external from distribution vendors, public comments, etc, there is a broad, general agreement this needs re-implementation in a proper language. Furthermore, continuing to iterate without shifting to a different language has proven a major pain to developers and maintainers and will only cause further pain as time goes on.
General Language Arguments:
For some time i was wondering why my ec2 instances takes ~2 minutes to be available to ssh connections.
By looking at the /var/log/secure I've noticed that there is a 2min gap (each time) between instance startup and the daemon startup:
Sep 28 22:44:50 ... useradd[3212]: add 'ec2-user' to shadow group 'wheel'
Sep 28 22:44:50 ... useradd[3212]: add 'ec2-user' to shadow group 'systemd-journal'
Sep 28 22:47:02 ... sshd[3401]: Server listening on 0.0.0.0 port 22.
Sep 28 22:47:02 ... sshd[3401]: Server listening on :: port 22.
after some digging, i've notice this in the journalctl:
Sep 28 22:44:51 ... systemd[1]: Started OpenSSH Server Key Generation.
Sep 28 22:44:51 ... systemd[1]: Starting EC2 Instance Connect Host Key Harvesting...
Sep 28 22:44:51 ... cloud-init[3225]: Cloud-init v. 18.2-72.amzn2.0.7 running 'modules:config' at Sat, 28 Sep 2019 22:44:51 +0000. Up 9.55 seconds.
Sep 28 22:44:51 ... eic_harvest_hostkeys[3284]: % Total % Received % Xferd Average Speed Time Time Time Current
Sep 28 22:44:51 ... eic_harvest_hostkeys[3284]: Dload Upload Total Spent Left Speed
Sep 28 22:44:51 ... systemd[1]: Stopping OpenSSH Server Key Generation...
Sep 28 22:44:52 ... chronyd[2695]: Selected source 169.254.169.123
Sep 28 22:44:53 ... dhclient[3035]: XMT: Solicit on eth0, interval 3970ms.
Sep 28 22:44:57 ... dhclient[3035]: XMT: Solicit on eth0, interval 7990ms.
Sep 28 22:45:05 ... dhclient[3035]: XMT: Solicit on eth0, interval 16040ms.
Sep 28 22:45:05 ... systemd[1]: Started Dynamically Generate Message Of The Day.
Sep 28 22:45:21 ... dhclient[3035]: XMT: Solicit on eth0, interval 31440ms.
Sep 28 22:45:52 ... dhclient[3035]: XMT: Solicit on eth0, interval 61220ms.
Sep 28 22:46:54 ... dhclient[3035]: XMT: Solicit on eth0, interval 112830ms.
Sep 28 22:47:02 ... systemd[1]: ec2-instance-connect.service: main process exited, code=exited, status=7/NOTRUNNING
Sep 28 22:47:02 ... eic_harvest_hostkeys[3284]: [10.2K blob data]
Sep 28 22:47:02 ... systemd[1]: Failed to start EC2 Instance Connect Host Key Harvesting.
Sep 28 22:47:02 ... systemd[1]: Unit ec2-instance-connect.service entered failed state.
Sep 28 22:47:02 ... systemd[1]: ec2-instance-connect.service failed.
Sep 28 22:47:02 ... systemd[1]: Starting OpenSSH server daemon...
Sep 28 22:47:02 ... sshd[3401]: Server listening on 0.0.0.0 port 22.
Sep 28 22:47:02 ... sshd[3401]: Server listening on :: port 22.
its seems like the Host Key Harvesting causing the delay, and only after it fails, the sshd is started.
i've verified this by these two steps, each one individually solved the problam
Since the instnace-connect tool comes by default with AmazonLinux2, i cannot avoid having this issue on each fresh instance creation.
Are the RPM packages for CentOS and RedHat published anywhere? I thought ec2-instance-connect supported CentOS and RedHat, but I can't find any yum repo where the packages are published.
Hello!
I'm currently packaging aws-ec2-instance-connect-config for openSUSE/SLE but I'm having a hard time properly testing the package due to lack of documentation.
It would be great if there were some simple manpages which explain exactly what each command does and which parameters they have and what they do.
Thanks!
ec2-instance-connect breaks during host key harvesting for instances launched in local zones.
I have tested with Ubuntu 21.04 (Hirsute) using that distro's package (version 1.1.13-0ubuntu1) but inspection of the code here indicates the problem is also in this upstream code @
$ systemctl is-system-running
degraded
$ systemctl list-units --failed
UNIT LOAD ACTIVE SUB DESCRIPTION
● ec2-instance-connect.service loaded failed failed EC2 Instance Connect Host Key Harvesting
$ journalctl --unit ec2-instance-connect
-- Logs begin at Wed 2021-02-10 22:47:47 UTC, end at Wed 2021-02-10 22:55:46 UTC. --
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Starting EC2 Instance Connect Host Key Harvesting...
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service: Main process exited, code=exited, status=255/EXCEPTION
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service: Failed with result 'exit-code'.
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Failed to start EC2 Instance Connect Host Key Harvesting.
bash -x /usr/share/ec2-instance-connect/eic_harvest_hostkeys to observe the failure...
++ /usr/bin/curl -s -f -m 1 -H 'X-aws-ec2-metadata-token:<<TOKEN>>' http://169.254.169.254/latest/meta-data/placement/availability-zone/
+ zone=us-west-2-lax-1a
+ zone_exit=0
+ '[' 0 -ne 0 ']'
+ /bin/echo us-west-2-lax-1a
+ /bin/grep -Eq '^([a-z]+-){2,3}[0-9][a-z]$'
+ /usr/bin/head -n 1
+ exit 255
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.