basti-app / basti Goto Github PK
View Code? Open in Web Editor NEW✨ Securely connect to RDS, Elasticache, and other AWS resources in VPCs with no idle cost
Home Page: https://www.basti.app
License: MIT License
✨ Securely connect to RDS, Elasticache, and other AWS resources in VPCs with no idle cost
Home Page: https://www.basti.app
License: MIT License
Hello!
I hope you're all doing well. I would like to propose the introduction of issue templates, pull request (PR) templates, and feature request templates in the repository. These templates will bring several benefits to the collaborative development process.
Bug report template: By using a standardized bug report template, we can ensure that all bug reports include essential information such as steps to reproduce, expected behavior, and current behavior. This will help us reproduce and resolve issues more efficiently and provide a better experience for our users who encounter bugs.
Feature request template: Implementing a feature request template will provide a structured format for users to communicate their ideas and requirements for new features. This will help us understand the use case, evaluate the feasibility, and prioritize feature requests effectively. It also encourages users to provide additional context and alternative solutions, fostering valuable discussions.
PR template: The addition of a PR template will streamline the pull request process. Contributors can provide clear descriptions of their proposed changes, reference related issues, and verify that they have followed coding style guidelines, added tests (if applicable), and tested their changes locally. This will enhance the quality of the codebase, simplify the review process, and ensure that contributions align with the project's standards.
Having these templates readily available will benefit both contributors and maintainers by providing clear guidelines and reducing ambiguity when submitting and reviewing issues, feature requests, and pull requests.
I am willing to take the responsibility of creating and implementing these templates if it's ok. Please share your thoughts and let me know if you have any suggestions or concerns.
I'll PR / proposal soon.
Using basti
with Node versions lower than 14.0.0 results in errors connected to Subpath Imports and ESModules usage.
The build process has to be reconfigured to support older Node versions.
Related issue: #21
Currently, t2.micro is the default instance family and size that cannot be changed. This is fine for smaller setups that e.g. can benefit from the AWS free tier.
Adding the optional setting to change the instance family and size would be great to match an organizations requirements.
For bigger setups at scale, it's a best practice to use either reserved instances or cost savings plans that may be limited to certain EC2 instance families.
If this is the case, basti
instances would be charged at on demand pricing and would not benefit from existing cost saving measures.
Optionally setting the instance family and instance size as parameter in the CLI or in the config file.
Some of our accounts use SSO to login into AWS. Would be great if basti can support this flow.
This should be possible with a aws sso login
command.
In some cases, the SSM agent takes more than the default stop-timeout of 5 minutes to connect to SSM, therefore the EC2 can come up, but the user is not able to create an ssm session.
Also, the SSM agent could have crashed for some reason, which renders the same result: you cannot connect.
Disclaimer: this is hard to reproduce as it depends on several factors that are out of control of the user.
Basti instance is usable via SSM
See Steps to Reproduce
I see three options:
increase / make the default stop-timeout configurable
add additional reboot to the instance after init (manual reboot helped in my case)
(preferred option) make the basti instance check if it the SSM agent connected successfully so a session can be initialized. If not, (force) restart the SSM agent, wait X s, recheck. If basti instance cannot connect to SSM, perform reboot. If reboot does not help, terminate the instance.
none
Currently, users must use the custom connection target approach to connect to Elasticache instances. This involves manually setting up connectivity (at init
time) and providing Basti with the target's IP and port (at connect
time). Let's implement first-class support for Elasticache, just like for RDS. This includes:
connect
and init
commandsIf they aren't yet, logs should be sent to cloudwatch (this should include system logs and access logs).
Why? An audit trail is vital for security programs, so to use this in a production environment, logging needs to be enabled.
If logs are already enabled, then logs should be mentioned in the README.
I'm not sure why basti init asks for a connection target but then basti connect asks it again.
If there are some steps needed to init the connection this should support multi-select.
AWS CLI profile set up in the user's environment is expected to be the most common authentication method in Basti. If the user has multiple profiles set up, he or she can select a specific named profile with the AWS_PROFILE
environment variable (it's a feature of AWS SDK).
Adding --aws-profile
option would make Basti commands more readable and the profile selection feature more visible as well. The users would see that there's such an option in basti --help
output.
Command used:
AWS_PROFILE=support npx basti connect --rds-instance production-postgres-13 --local-port 5434
Output:
TypeError [ERR_PACKAGE_IMPORT_NOT_DEFINED]: Package import specifier "#src/common/debug.js" is not defined in package /Users/antonkozachuk/.npm/_npx/a74823000dc99449/node_modules/basti/dist/package.json imported from /Users/antonkozachuk/.npm/_npx/a74823000dc99449/node_modules/basti/dist/src/cli/run.js at throwImportNotDefined (internal/modules/esm/resolve.js:293:9) at packageImportsResolve (internal/modules/esm/resolve.js:574:3) at moduleResolve (internal/modules/esm/resolve.js:700:21) at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:819:11) at Loader.resolve (internal/modules/esm/loader.js:89:40) at Loader.getModuleJob (internal/modules/esm/loader.js:242:28) at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:73:40) at link (internal/modules/esm/module_job.js:72:36) { code: 'ERR_PACKAGE_IMPORT_NOT_DEFINED' }
At the moment, basti presents all the RDS instances irrespective of whether it is for an aurora cluster or nor normal RDS instance.
It would be great to present just the writer / reader endpoints especially where there are many readers in a aurora cluster.
assignPublicIp: true
I think this is wrong by design and should at least be configurable. A public ip is really not required with ssm. The only thing the instance needs to do is to connect to the amazonaws.com domains via VPC Endpoints, or a through a connection with the internet and this can also work well via a NAT GW or a Transit GW.
Improve error messages structure, as it makes it easier to solve a problem for end-user.
For example:
❌ Error checking target state. Unexpected error: The SSO session associated with this profile has expired. To refresh this SSO session run aws sso login with the corresponding profile.
Can be
❌ Error checking target state.
⚠️ The SSO session associated with this profile has expired.
To refresh the SSO session, please run the following command with the corresponding profile:
aws sso login --profile <profile_name>
Separate error message from suggested fix, when it's possible
not yet : )
It was reported that session-manager-plugin
was not installed automatically when using basti
with npx
.
Platform: MacOS
Arch: Arm64
Basti init starts an EC2 instance. Then it is in the VPC for potentially a long time (even though "off" when not in use). However, how does the instance get security patches? How is the instance monitored for security purposes? This type of documentation should be in the README so that a CISO can easily review to determine if Basti will work in their organization. Also, they would probably want to know the base image (assuming it is using a public one maintained by AWS or similar organization).
Side note: could basti work with fargate too instead of ec2?
The Basti CLI currently only logs to the terminal. This issue proposes a solution that would allow logging to log file.
A logging config option would allow users to specify a file or location (or a default location would be used otherwise) where the Basti CLI would log its activity. This would be beneficial for a number of reasons, including:
Building on issue #59 it would make sense to create a .config.yml
(or json) file in this ~/.basti
directory. This would then have a section for logging.
logs:
save_logs: true
log_line_limit: 1000
save_location: ~/.basti/logs/
save_logs
would enable or disable logging to a file
log_line_limit
is intended to limit the size of the file. This avoids log files that end up GB's in size.
save_location
allows the user to configure a save location, but default this is stored in the ~/.basti
directory.
The logging itself should always be prefixed with a timestamp. This helps with troubleshooting events in time and allows you to check with AWS logging as well.
Initially, I'd recommend that these settings can just be modified through the config.yml
file, but later a command could be added that changes these settings. basti config logs.save_logs true
for example.
Install the app without npm
It would be nice to make installation easier on osx and linux by using homebrew without npm dependency
by supporting homebrew formula to use brew install basti
.
A formula can be contributed to homebrew-core
https://docs.brew.sh/Formula-Cookbook
And updated via github action
N/A
The project requires extended reference documentation but the root README.md file is already pretty bit.
We'll set up a full-blown documentation site at https://www.basti.app
While packaging for homebrew, I ran into some shell completion issue as below:
~ basti completion zsh
basti <command>
Commands:
basti init Initialize a target to use with Basti [aliases: i]
basti connect [connection] Start port forwarding session with the selected target [aliases: c]
basti cleanup Remove all resources created by Basti [aliases: cl]
basti completion Generate completion script for your shell
Options:
-h, --help Show help [boolean]
-v, --version Show version number [boolean]
Examples:
basti init Initialize a target in interactive mode
basti connect Start connection in interactive mode
basti cleanup Cleanup Basti resources in interactive mode (requires confirmation)
Did you mean cl?
➜ ~ echo $?
1
Log an issue in here for visibility.
Instead of creating the basti resources with the command basti init
there would also be an option of creating the resources through an AWS CDK construct.
An AWS CDK construct for Basti would offer the following benefits:
Currently, users can create Basti bastion hosts using the basti init command. However, this can be difficult to use when all other resources are manged through AWS CDK, as it requires users to manually assign permissions and set up monitoring. An AWS CDK construct would make it easier for these users to create Basti bastion hosts, as it would allow them to use the AWS CDK's built-in permission and monitoring features.
The construct should do a lot, but should not be too configurable. The interface should be relatively simple. As targets are most often selected when connecting to the bastion host, the initial version should just set up the bastion host and no more. The interface in python (CDK supports many languages through JSII) would just be a class.
from basti_cdk import BastiHost
host = BastiHost(self, 'bastiHost')
The basti CDK construct can then create its own VPC and any subnets it needs. Maybe the interface can be expanded that these are created.
The construct could also be more limiting and only allow access to a specific database/database cluster.
No related PRs
I would be able to help in creating this functionality.
Currently, users must use the custom connection target approach to connect to services running on EC2 instances. This involves manually setting up connectivity (at init
time) and providing Basti with the target's IP and port (at connect
time).
The first-class EC2 support could be implemented in the following way:
basti init
basti init
run)This includes:
basti cleanup
commandWhen having multiple setups of basti in different regions such as eu-central-1 and us-west-2 basti cleanup doesn't respect the --aws-region flag when it comes to IAM resources.
basti init --aws-region eu-central-1
basti init --aws-region us-west-2
basti cleanup --aws-region us-west-2
After the command in step 4 all basti resources wich were deployed in step 2 should have been removed and all basti resources which were deployed in step 1 should stil be existing.
Unfortunately basti cleans ALL IAM resources including the role / instance profile that belongs to the eu-central-1 setup wich essentially renders that setup unusable as a connection to the instance via Session Manger is no longer possible.
Somehow make basti aware of the region of the setup which IAM resource belongs to so that basti cleanup does not remove resources which it shouldn't remove.
Apart from that thanks for proving such an elegant solution! :)
Here is the minimal set of permissions that worked for me:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:StartInstances",
"ec2:CreateTags",
"ssm:StartSession"
],
"Resource": [
"arn:aws:ec2:us-east-1:<account-id>:instance/i-<basti-instance-id>",
"arn:aws:ec2:us-east-1:<account-id>:security-group/sg-<basti-security-group-id>",
"arn:aws:ssm:*:*:document/AWS-StartPortForwardingSessionToRemoteHost"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeSecurityGroupRules",
"ec2:DescribeInstanceAttribute",
"ec2:DescribeNetworkAcls",
"ec2:DescribeSecurityGroups",
"ec2:ModifySecurityGroupRules",
"ec2:DescribeInstanceStatus"
],
"Resource": "*"
}
]
}
Hi there, we recently reduced IAM permissions for our developers which led to the situation that they can not longer run basti init
by themselves. Could you please document the required permissions?
Right now Basti isn't really pointing out which permissions are missing (Which it btw. does when running a cleanup):
Error setting up bastion. Can't create IAM role for bastion instance. Access denied by IAM.
Thanks!
Do not grant your AWS user full IAM permissions and try to run basti init
While packaging for homebrew, I noticed that the wrong arch session-manager-plugin got bundled (below is npm build for a apple arm machine):
Warning: Binaries built for a non-native architecture were installed into basti's prefix.
The offending files are:
/opt/homebrew/Cellar/basti/1.6.2/libexec/lib/node_modules/basti/node_modules/basti-session-manager-binary-darwin-arm64/session-manager-plugin (x86_64)
The bastion is currently provided the s3:GetEncryptionConfiguration
permission. This permission is not needed for general SSM operations, and my investigation has not yielded any results where Basti would need this permission.
I didn't give basti this permission in my PoC for the CDK construct, and everything seems to work fine.
The permission is added here:
Same reasoning as in #12
Sometimes you need to connect to multiple resources in your remote environment simultaneously. Currently, you need to run Basti multiple times in separate terminal windows. Being able to connect to multiple targets with a single Basti command would be useful.
In interactive (basti connect
) and automatic (basti connect <options>
) modes, multiple target selection will become cumbersome. Multiconnect should only be supported in the predefined connection mode, when all the connection targets are defined in a configuration file.
A new top-level field "connection-groups" (the name is discussable) has to be introduced. The field will reference multiple connections. The connections might also be inlined like with target inlining.
connection-groups:
prod:
connections:
- database-prod
- cache-prod
dev:
connections:
- database-dev
- cache-dev
connections:
database-dev:
...
database-prod:
...
cache-dev:
...
cache-prod:
...
Connection groups and connection names must be unique. In other words, a connection group can't have the same name as a connection. This will allow to keep the connection command concise:
basti connect <connection | connection-group>
Basti must track and display the statuses of all the simultaneous connections. The application must exit when at least one of the connections dies.
Add possibility to add custom tags that are applied to the instance and related resources.
If an organization has a tag policy / SCP combination active that enforces a set of standard tags, basti cannot be used.
Additionally, some organizations have cost allocation tags active which makes it easier to accumulate cost that are induced by basti.
Also, ownership tags are a use case that could come in handy if teams are using basti to access their resources for increased transparency.
none
Basti currently searches for a configuration file in your current working directory up to your home directory. This is great, however sometimes when you are using basti in directories above or outside the home directory it cannot find a configuration file. The solution for this would be to always check the home directory.
This could be in a .basti
directory as this also allows other information related to basti to be stored here as well as leaving room for more features in the future. The other option is to just always check for .basti.yml
(or json) files in the home directory.
Tested the tool, very impressed by the smoothness of setup and functionality.
One thing, however, which would have to be figured out, if an org wants to give this functionality to devs: is to what set of permissions to grant. Could you please describe this in documentation?
By the time this issue is created, there're two packages in this monorepo:
basti
)basti-cdk
)basti
and basti-cdk
are both versioned independently following semantic versioning.
A new major version is released in two cases
basti-cdk
property or a basti
CLI option has been renamed.init
command or basti-cdk
infrastructure is no longer compatible with the existing versions of the Basti CLI connect
command.Newer versions of Basti CLI must support connecting to any older minor and major version of the infrastructure.
basti connect
command checks if the infrastructure was initialized with the satisfying major version of Basti CLI or Basti CDK. If not, the user is asked to update to the latest version of Basti CLI (which must support the newer infrastructure).
Hello,
I'd like to reach out privately with a potential (minor) security issue / recommendation. Would you mind sharing a preferred contact method or enabling private vulnerability reporting?
Thank you!
When using CDK deploy or diff, "no operation" is expected if there is nothing to change
Running cdk diff
on a Basti CloudFormation stack reports systematically a change on EC2 tag basti:in-use
Resources
[~] AWS::EC2::Instance basti-instance/Ec2InstanceBastion bastiinstanceEc2InstanceBastionC84AB388
└─ [~] Tags
└─ @@ -9,7 +9,7 @@
[ ] },
[ ] {
[ ] "Key": "basti:in-use",
[-] "Value": "2024-06-28T12:42:12.448Z"
[+] "Value": "2024-08-09T08:19:57.890Z"
[ ] },
[ ] {
[ ] "Key": "Environment",
Exclude this tag from resources under lifecycle management in CF stack
Basti can be integrated into github actions workflows but it's a bit hacky:
Here is how I did it:
npx basti connect --rds-instance $DB_INSTANCE_NAME --local-port $DB_PORT &
npx wait-on -t 60000 tcp:$DB_PORT
Notice the &
at the end of the connect
command to put basti into background mode.
One issue is it never stops the connection.
Would be great to have official github action which can incapsulate connection/wait and teardown using runs.post
To enhance security, it's important to attach encrypted EBS volumes for EC2 instances.
NA
Perhaps we could enable the encryption configuration here . We would be pleased to collaborate if you are in agreement
NA
Remember the local port previously used for specific connect target and suggest it on the subsequent connect.
Users will have DB connections saved locally so it's important to use same port.
The Basti ec2 instance has insufficient permissions to operate. It requires the ec2:DescribeInstances
and ec2:CreateTags
permissions to be granted. But they are not. As a result, when starting a basti connection it is only functional for around 1 minute, after which the instance shuts down again as the script it is running has an error.
I have no idea why I did not notice this sooner. It might have something to do with a dependency issue that I removed to fix later, but then never fixed.
The solution is not ideal, as it is not possible (at least not to my knowledge) to restrict the resource scope to the instance alone, as it would create a circular dependency. This means that the scope for the ec2:CreateTags
action is wider than it should ideally be. I'll submit a PR that introduces some fixes for this.
Connection to the same target is repetitive right now and requires someone to carefully pick the target and use correct port.
Would be slick to have some predefined configurations set (which we can distribute to all our users) which will allow doing something like:
basti connect prod-db
basti connect staging-db
Behind the scenes it will pick correct region, target and local port.
The config format may be as simple as that:
{
"prod-db": { "target": "production-postgres-13", "localPort": 1234, "region": "us-east1" }
// ...
}
You can store this config in ~/.aws/basti
for example.
I know this can be achieved with cli args though so we can just share predefined commands.
So maybe this is an overkill.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.