aws/amazon-ec2-instance-selector

A CLI tool and go library which recommends instance types based on resource criteria like vcpus and memory

amazon-ec2-instance-selector's Issues

Suppor/Flag to filter out burstable instances

While the flag --burst-support is great to list only burstable instances. I think there might be cases where adding some flag to only list non-burstable instances would be also helpful. For example, if my application has a mixed cpu/memory/io profile with high peaks of CPU now and again, I may prefer to avoid burstable instances.

For example:

$ ec2-instance-selector --vcpus 4 --memory 16384 -a x86_64 --current-generation --non-burst-support
g4dn.xlarge
m4.xlarge
m5.xlarge
m5a.xlarge
m5ad.xlarge
m5d.xlarge
m5dn.xlarge
m5n.xlarge

Perhaps a more generic filtering out other attributes. Burstable instances however are probably the first that come to mind when thinking about production systems.

vcpus=96 returned r5.large which is 2cpu

ec2-instance-selector --region=eu-north-1 --vcpus=96 --memory-min=192 --hypervisor nitro --cpu-architecture x86_64 --deny-list '^vt.|^inf.|d\.|en\.|dn\.' --gpus 0 --network-performance-max 50 --root-device-type ebs --usage-class=spot --price-per-hour-max 1.4 --max-results 100

returned once

      - c5.24xlarge
      - c5a.24xlarge
      - c6i.24xlarge
      - m5.24xlarge
      - r5.large

couldn't reproduce it, but I'm 100% sure about this as it was in CI/CD and suddenly my 96cpu instances had one 2cpu among them

--output table-wide never return

I tried the following command ec2-instance-selector --memory 4 --vcpus 2 --cpu-architecture x86_64 -r us-east-1 -o table-wide on Windows 11 with aws-cli/2.15.11 and the command never return. Eventually the process has to be killed. I have tried the same on Fedora with same aws cli version and the outcome is the same.

Any body has similar issue?

assets (binaries) missing for release 2.1.0

https://github.com/aws/amazon-ec2-instance-selector/releases/tag/v2.1.0

Add support to filter instances by price range, and expose the prices/spot prices in output

It would be great if the support is added to filter out instances based on live price range. Also, to show the price of the instances (both spot and on-demand) in the output.

Add support of assume role with MFA

In our organization access to AWS CLI is implemented with assuming role and MFA and when I try to run ec2-instance-selector --profile <profile_name> --vcpus 4 --memory 8 --region eu-central-1 for example it fails with:

panic: AssumeRoleTokenProviderNotSetError: assume role with MFA enabled, but AssumeRoleTokenProvider session option not set.

goroutine 1 [running]:
github.com/aws/aws-sdk-go/aws/session.Must(...)
	/go/pkg/mod/github.com/aws/[email protected]/aws/session/session.go:370
main.getRegionAndProfileAWSSession(0xc0003362b, 0xc00033627, 0xc00033627, 0xc00033627, 0xc00033061)
	/amazon-ec2-instance-selector/cmd/main.go:315 +0x790
main.main()
	/amazon-ec2-instance-selector/cmd/main.go:182 +0x148

It will be great to have this functionality.

Feature-request : frequency-interruption for Ec2 Spot Instance

Issue: I cannot filter the instance to have less disruption on the application as there is no frequency interruption for Ec2 Spot Instance

New feature: Would be great to have a parameter such as --frequency-interruption

as for this page: https://aws.amazon.com/ec2/spot/instance-advisor/
in order to filter from less frequency-interruption

Adding "Sorting" in the table-wide Mode

This is no doubt a cool addition to our toolset. Right now output is sorted by "Instance Type"
Requesting Add feature like "Sort By" in the Table Wide mode.
Can we add Sorting to sort by other variables ?

Something like

ec2-instance-selector --gpus 1 -o table-wide --sort m

_

Add a Pack option to bin pack pods or other discrete resource specs on to ec2 instances

Add a new command that accepts a list of Filters which can define specs for containers/pods/jobs/etc. ec2-instance-selector will then return a set of instance type that match the filters and bin-pack the filters on the smallest number of nodes that match the criteria.

---

Examples:

$ ec2-instance-selector pack --filters "{vcpus: 1, memory: 3gb}, {vcpus: 1.5, memory: 4gb}, {vcpus: 0.5, memory: 4gb}"
  
t3.medium
m5.large

You could add any supported filter property and make them very complicated...:

$ ec2-instance-selector pack --filters "{vcpus: 1, memory: 3gb, network-interfaces-min: 3, network-performance: 100}, {vcpus: 1.5, memory: 4gb, gpus-min: 2}, {vcpus: 0.5, memory: 4gb, fpga-support: true, availability-zone: us-east-1a}"
  
g4dn.metal
f1.2xlarge

TODO / Open Questions:

1. Should the output be richer than just instance types? It probably needs to show which filter specs its fulfilling for each instance type.

2. Example of using this in Go code as a library

EFA filter

Would be nice to have a filter to select EC2 instances which support Elastic Fabric Adapter https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa.html#efa-instance-types

Where are other flag options?

There is no
--price-per-hour float Price/hour in USD (Example: 0.09) (sets --price-per-hour-min and -max to the same value)
--price-per-hour-max float Maximum Price/hour in USD (Example: 0.09) If --price-per-hour-min is not specified, the lower bound will be 0
--price-per-hour-min float

flags.

██ec2-instance-selector --version
v2.0.2
██ec2-instance-selector --help
ec2-instance-selector is a CLI tool to filter EC2 instance types based on resource criteria.
Filtering allows you to select all the instance types that match your application requirements.
Full docs can be found at github.com/aws/amazon-ec2-instance-selector

Usage:
ec2-instance-selector [flags]

Examples:
ec2-instance-selector --vcpus 4 --region us-east-2 --availability-zones us-east-2b
ec2-instance-selector --memory-min 4 --memory-max 8 --vcpus-min 4 --vcpus-max 8 --region us-east-2

Filter Flags:
--allow-list string List of allowed instance types to select from w/ regex syntax (Example: m[3-5].)
-z, --availability-zones strings Availability zones or zone ids to check EC2 capacity offered in specific AZs
--baremetal Bare Metal instance types (.metal instances)
-b, --burst-support Burstable instance types
-a, --cpu-architecture string CPU architecture [x86_64/amd64, i386, or arm64]
--current-generation Current generation instance types (explicitly set this to false to not return current generation instance types)
--deny-list string List of instance types which should be excluded w/ regex syntax (Example: m[1-2].)
-e, --ena-support Instance types where ENA is supported or required
-f, --fpga-support FPGA instance types
--gpu-memory-total string Number of GPUs' total memory (Example: 4 GiB) (sets --gpu-memory-total-min and -max to the same value)
--gpu-memory-total-max string Maximum Number of GPUs' total memory (Example: 4 GiB) If --gpu-memory-total-min is not specified, the lower bound will be 0
--gpu-memory-total-min string Minimum Number of GPUs' total memory (Example: 4 GiB) If --gpu-memory-total-max is not specified, the upper bound will be infinity
-g, --gpus int Total Number of GPUs (Example: 4) (sets --gpus-min and -max to the same value)
--gpus-max int Maximum Total Number of GPUs (Example: 4) If --gpus-min is not specified, the lower bound will be 0
--gpus-min int Minimum Total Number of GPUs (Example: 4) If --gpus-max is not specified, the upper bound will be infinity
--hibernation-support Hibernation supported
--hypervisor string Hypervisor: [xen or nitro]
-m, --memory string Amount of Memory available (Example: 4 GiB) (sets --memory-min and -max to the same value)
--memory-max string Maximum Amount of Memory available (Example: 4 GiB) If --memory-min is not specified, the lower bound will be 0
--memory-min string Minimum Amount of Memory available (Example: 4 GiB) If --memory-max is not specified, the upper bound will be infinity
--network-interfaces int Number of network interfaces (ENIs) that can be attached to the instance (sets --network-interfaces-min and -max to the same value)
--network-interfaces-max int Maximum Number of network interfaces (ENIs) that can be attached to the instance If --network-interfaces-min is not specified, the lower bound will be 0
--network-interfaces-min int Minimum Number of network interfaces (ENIs) that can be attached to the instance If --network-interfaces-max is not specified, the upper bound will be infinity
--network-performance int Bandwidth in Gib/s of network performance (Example: 100) (sets --network-performance-min and -max to the same value)
--network-performance-max int Maximum Bandwidth in Gib/s of network performance (Example: 100) If --network-performance-min is not specified, the lower bound will be 0
--network-performance-min int Minimum Bandwidth in Gib/s of network performance (Example: 100) If --network-performance-max is not specified, the upper bound will be infinity
--placement-group-strategy string Placement group strategy: [cluster, partition, spread]
--root-device-type string Supported root device types: [ebs or instance-store]
-u, --usage-class string Usage class: [spot or on-demand]
-c, --vcpus int Number of vcpus available to the instance type. (sets --vcpus-min and -max to the same value)
--vcpus-max int Maximum Number of vcpus available to the instance type. If --vcpus-min is not specified, the lower bound will be 0
--vcpus-min int Minimum Number of vcpus available to the instance type. If --vcpus-max is not specified, the upper bound will be infinity
--vcpus-to-memory-ratio string The ratio of vcpus to GiBs of memory. (Example: 1:2)

Suite Flags:
--base-instance-type string Instance Type used to retrieve similarly spec'd instance types
--flexible Retrieves a group of instance types spanning multiple generations based on opinionated defaults and user overridden resource filters

Global Flags:
-h, --help Help
--max-results int The maximum number of instance types that match your criteria to return (default 20)
-o, --output string Specify the output format (table, table-wide, one-line)
--profile string AWS CLI profile to use for credentials and config
-r, --region string AWS Region to use for API requests (NOTE: if not passed in, uses AWS SDK default precedence)
-v, --verbose Verbose - will print out full instance specs
--version Prints CLI version

Support filtering by network encryption

A subset of Nitro instance types support automatic encryption of network traffic. For some applications, this is a requirement, so I would like to have an option to filter instances by this capability.

Support AutoSpotting use cases

This looks like a neat library, and I'm planning to try it on https://github.com/AutoSpotting/AutoSpotting looking into hopefully replacing our current custom compatibility logic.

We have a bunch of additional features I mentioned below, I'd love to see at least some of these implemented in this library, so I can hopefully reduce or even entirely replace my custom logic:

• uses an instance ID(assumed to be from the current region) as comparison baseline instead of the region-agnostic instance type supported currently
• considers actually used resources on the baseline running instance instead of the maximum available in the specs, for example currently we consider the attached instance store volumes in the block device mapping not the total available, but there may be others.
• consider spot pricing of the target instance type, returning the better-priced compatible spot instance type in the same region or AZ(would be nice to be configurable between Region and AZ).
• sorts the result in increased order of the spot price.
• efficient when it comes to API calls and network traffic by storing a lot of static data(having some of it cached for a day or so may be acceptable though).

If you think this makes sense to be implemented in this project, I'll create more granular issues for all these pieces of functionality so we can discuss each of them in more detail.

support filtering inf. and vt1. instances

currently I need to --deny-list "^vt1\." --deny-list "^inf1\." to remove those instance types, couldn't find anything else on the options to not return such instances

sort by price

usacase

ec2-instance-selector --region=eu-north-1 \
                --vcpus-max=2 \
                --memory-max=2 \
                --cpu-architecture x86_64 \
                --gpus 0 \
                --root-device-type ebs

returns 3 instances - which one is the cheapest?

Pricing API change in aws-sdk-go

When updating kOps dependencies, there is a failure related to amazon-ec2-instance-selector:
https://github.com/kubernetes/kops/actions/runs/2594660135

vendor/github.com/aws/amazon-ec2-instance-selector/v2/pkg/ec2pricing/odpricing.go#L172
cannot use priceDoc (variable of type *string) as type aws.JSONValue in argument to c.parseOndemandUnitPrice

Most likely, it would have been noticed when projects deps would have been updated also, but if anyone could look into this, it would be appreciated.

Ability to filter if IsTrunkingCompatible

If on EKS you're using security group for PODs you need to now the list of supported instance types and which one have the IsTrunkingCompatible: true.

Right now the only way to get that information is trough that file: https://github.com/aws/amazon-vpc-resource-controller-k8s/blob/release/pkg/aws/vpc/limits.go#L32
More info: https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html

I'll be really useful to have it on the cli.

Cache DescribeInstanceTypes and DescribeInstanceTypeOfferings

When using ec2-instance-selector as a library in an always running application, it would be nice to have DescribeInstanceTypes and DescribeInstanceTypeOfferings calls cached. EC2 instance types don't change too often, so this cache is probably safe to stick around for hours but even moderate caching on the order of minutes could save many API calls.

---

I would envision the cache to be on by default with a conservative TTL of 15 minutes. A public method will need to be defined to clear the cache and to refresh the cache. The lib consumer could use the refresh in a ticker go routine to periodically refresh the cache without taking the cache miss latency hit. ec2-instance-selector could also providethe function that a user could kick off in a go routine for refresh.

Latest EKS AMI tag breaks querying EKS instance types

The amazon-eks-ami repo published a tag yesterday (V20220123) that breaks querying EKS instance types. The tag starts with an uppercase V, causing this check to prepend an additional v, which makes the resulting archive URL (https://github.com/awslabs/amazon-eks-ami/releases/tag/vV20220123.zip) incorrect.

$ ec2-instance-selector --service eks --memory 2 --vcpus 4 --cpu-architecture x86_64  -r us-west-2
NOTE: Unable to retrieve EKS supported instance types for version vV20220123: Unable to retrieve EKS supported instance types, got non-200 status code: 404
An error occurred when filtering instance types: Unable to retrieve EKS supported instance types, got non-200 status code: 404

support for network baseline/burst flags

Like EBS, EC2 instance network capacity also has baseline and burst bandwidth. Current --network-performance-min and max flags do not consider this, it would be nice if it did to bring it in line with how this tool can filter with the ebs baseline flags.

for example, a flag of --network-performance-min 5 GiB shows me in m5.xlarge (but it only has 1.25gbps of baseline bandwidth)
--network-performance-min 5 GiB --vcpus 4 --current-generation --output table --max-results 100 --region ap-southeast-2

Location of baseline bandwidth is in the EC2 documentation, example: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/general-purpose-instances.html#general-purpose-network-performance

java library

Does this selector have a java or javascript api to call?

on-demand pricing cache bug

It appears that when pricing data is cached, something will get out-of-sync resulting in a flood of "Could not retrieve on-demand price for instance type "

Here's maybe some reproduction steps:

## clear cache to start fresh
$ ec2-instance-selector --cache-ttl=0 -o table-wide

## populate cache with min-ttl (which is unfortunately 1 hour, but you could hard code something smaller in the go code)
$ ec2-instance-selector --cache-ttl=1 -o table-wide

## come back after 1 hour and rerun 
$ ec2-instance-selector --cache-ttl=1 -o table-wide

## You should see a bunch of these:
NOTE: Could not retrieve 30 day avg hourly spot price for instance type x2iedn.metal
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type r5ad.8xlarge
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type m3.xlarge
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type g3s.xlarge
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type t3.small
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type g5.xlarge
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type i3.4xlarge
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type i3en.large
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type r3.4xlarge
NOTE: Could not retrieve instantaneous hourly on-demand price for instance type u-6tb1.56xlarge

Filter for AVX-512 support

Hi, can we add support to filter on AVX-512 instruction set support?

Logs are in alphabetical order when tests fail

• The e2e test sorts the expected and actual test values when comparing results. This works when the tests pass, but when the tests fail the logs are also returned sorted making them unreadable.

• Example:

	🔺ACTUAL: - - - - --region AWS_DEFAULT_REGION AWS_REGION Unable a default environment environment find flag in in places: profile region region the to usual variable variable ~/.aws/config
	🔺Expected: t3a.micro t3.micro t2.micro

Add support for AWS IAM Identity Center (successor to AWS Single Sign-On) SSO credential format

We utilize AWS Identity Center Single Sign-On (SSO) enabled profiles, as outlined in the documentation at https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html. Once I am authenticated to my SSO profile and have verified its authentication, the ec2-instance-selector script encounters a failure, as demonstrated below:

$ aws sso login --no-browser --sso-session my-login1

$ aws sts get-caller-identity --profile my-profile1
{
    "UserId": "AAAAAAAAAAAAAAAAAAAAA:[email protected]",
    "Account": "0123456789012",
    "Arn": "arn:aws:sts::0123456789012:assumed-role/AWSReservedSSO_my-aws-role1_abababababababab/[email protected]"
}

$ ec2-instance-selector --version
v2.4.1

$ ec2-instance-selector --memory 8 --vcpus 4 --cpu-architecture x86_64 -r us-east-1 -o table --profile my-profile1
panic: profile "my-profile1" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

goroutine 1 [running]:
github.com/aws/aws-sdk-go/aws/session.Must(...)
	/go/pkg/mod/github.com/aws/[email protected]/aws/session/session.go:381
main.getRegionAndProfileAWSSession(0xe9b920?, 0xc00032b860)
	/amazon-ec2-instance-selector/cmd/main.go:507 +0x52a
main.main()
	/amazon-ec2-instance-selector/cmd/main.go:245 +0x1535

My sanitized .aws/config
[sso-session my-login1
sso_start_url = https://mysite.awsapps.com/start
sso_region = us-east-1
sso_registration_scopes = sso:account:access

[profile my-profile1]
sso_session = my-login1
sso_account_id = 0123456789012
sso_role_name = my-aws-role1
region = us-east-1
output = json

Ability to filter burstable instances by baseline load

We are making our own instance right-sizing/recommendation tool and it would be nice if we could filter burstable instances by a minimum percentage of baseline utilization.

Service Specific Instance Types

Some services like EKS and EMR use ec2 instance types but are only compatible with a subset of instance types.

I propose a service flag to allow users to specify a service like eks or emr to further filter the instance types:

$ ec2-instance-selector --vcpus-min=2 --memory=4 --service=eks
a1.large
c5.large
c5a.large
c5ad.large
c5d.large
c6g.large
c6gd.large
t2.medium
t3.medium
t3a.medium
t4g.medium

Instance types not supported by EMR

When running a command like the following, it's returning m5n.2xlarge and m5ad.2xlarge which are not supported by EMR.

ec2-instance-selector --vcpus-min 8 --vcpus-max 8 --vcpus-to-memory-ratio 1:4 --cpu-architecture x86_64 --current-generation --gpus 0 --service emr-5.32.0 --max-results 50 -o table-wide --price-per-hour-max 0.428 -u spot --region eu-west-1

Do we need to specify another parameter or is there an issue with the API call?

Select same spec instance types from a specific instance type

It would be cool to select same spec instance types from a specific instance type. Like below.

ec2-instance-selector instance-type m5.xlarge
# OUTPUT, it can be list
m4.xlarge, m5.xlarge, m5a.xlarge, m5ad.xlarge, m5d.xlarge, m5dn.xlarge, m5n.xlarge

This feature is useful for dynamically creating CloudFormation template with AWS::EKS::Nodegroup.InstanceTypes. Now, I tried an unattractive way with AWS CLI...

CPUS=aws ec2 describe-instance-types --instance-types m5.xlarge --query 'InstanceTypes[].VCpuInfo.DefaultVCpus | [0]'
# Fetch other specs: MEMORY and  GPUS
ec2-instance-selector --vcpus=$CPUS --memory=$MEMORY --cpu-architecture=x86_64 --gpus $GPUS --burst-support=false

ap-south-1 region is not recognized

Hello,

ec2-instance-selector \
  --usage-class spot \
  --cpu-architecture x86_64 \
  --availability-zones ap-south-1a,ap-south-1b,ap-south-1c \
  --vcpus 1 --memory 1

produces the following output

An error occurred when filtering instance types: The location passed in (ap-south-1a) is not a valid zone-id, zone-name, or region name

Specifying storage requirements fails to return any instances

If I ask for a list of instances that have 2 CPUs and 4GB memory, I get a reasonable list returned:

$ ./ec2-instance-selector --vcpus-min 2 -m 4G
a1.large
c5.large
c5a.large
c5ad.large
c5d.large
c6a.large
c6g.large
c6gd.large
c6gn.large
c6i.large
c6id.large
c7g.large
t2.medium
t3.medium
t3a.medium
t4g.medium

However, if I add any sort of storage requirement to this, I get nothing back:

$ ./ec2-instance-selector --vcpus-min 2 -m 4G --instance-storage-min 256G
NOTE: The criteria was too narrow and returned no valid instance types. Consider broadening your criteria so that more instance types are returned.

I don't understand why no instances are being returned since the (EBS) root volume of an instance can be any reasonable size ... can't they?

Support unsetting select filter flags with the usage of Suite Flags

Suite Flags is a good simplification for common / popular use cases. Overriding of values set for filter flags when suite flags are used is allowed, however, there is no way to unset certain filter flags like

ec2-instance-selector --base-instance-type m4.xlarge -o table --unset=memory-max,vcpus-max

Add a "truncated" message to stderr if instance types > Max Results

Use latest release, r5.xlarge have 4 vcpu ,32gb memory , it should list by below criteria but it's not.

$ ./ec2-instance-selector-linux-amd64 --vcpus 4 --memory-min 8 -a x86_64
c5.xlarge
c5a.xlarge
c5d.xlarge
c5n.xlarge
d2.xlarge
g3s.xlarge
g4dn.xlarge
i2.xlarge
i3.xlarge
i3en.xlarge
inf1.xlarge
m1.xlarge
m2.2xlarge
m3.xlarge
m4.xlarge
m5.xlarge
m5a.xlarge
m5ad.xlarge
m5d.xlarge
m5dn.xlarge
$ ./ec2-instance-selector-linux-amd64 --vcpus 4 --memory-min 16 -a x86_64
d2.xlarge
g3s.xlarge
g4dn.xlarge
i2.xlarge
i3.xlarge
i3en.xlarge
m2.2xlarge
m4.xlarge
m5.xlarge
m5a.xlarge
m5ad.xlarge
m5d.xlarge
m5dn.xlarge
m5n.xlarge
p2.xlarge
r3.xlarge
r4.xlarge
r5.xlarge
r5a.xlarge
r5ad.xlarge

with same logic the --vcpu-min works well. Is it design like this?

Bug - Invalid filtering type with service eks

This is an excellent tool. Thanks for creating it :)

I have filtered valid instances for eks like this but EKS complained that instance c6i.xlarge is not a supported one. I am using terraform resource aws_eks_node_group

❯ ec2-instance-selector --version
v2.0.3

❯ ec2-instance-selector --service eks --memory 8 --vcpus 4 --cpu-architecture x86_64 -r ap-southeast-2
c5.xlarge
c5a.xlarge
c5ad.xlarge
c5d.xlarge
c6i.xlarge
inf1.xlarge

--cpu-manufacturer intel shows AMD EPYC based c5ad and m5ad

ec2-instance-selector --cpu-manufacturer intel --vcpus 16 --vcpus-to-memory-ratio 1:2 --cpu-architecture x86_64 --hypervisor nitro --ebs-optimized --network-performance-min 5 --usage-class spot --service eks --region eu-central-1 -o table-wide --sort-by spot-price

This lists c5ad.4xlarge.. same for m5ad with --vcpus-to-memory-ratio 1:4
https://aws.amazon.com/blogs/aws/new-amd-epyc-powered-amazon-ec2-m5ad-and-r5ad-instances/

Ability to filter Ipv4 per ENIs

When we want to calculate the max pods in the EKS case of an instance type we need to know both the number of ENIs and the number of Ip per ENIs.

Actually we need to do the check manually in order not to have problems in the EKS clusters.

OD Pricing for some Regions do not work

The OD Pricing API uses "Region Description" US East (N. Virginia) to filter instead of a region name like us-east-1.

If you try to use some other regions, the SDKs region descriptions are not returning what the pricing API expects.

For example, if you want to query for a price in eu-central-1, the SDK gives back Europe (Frankfurt) but the pricing API expects EU (Frankfurt).

This is the code that is incorrect (at least for some regions): https://github.com/aws/amazon-ec2-instance-selector/blob/main/pkg/ec2pricing/odpricing.go#L214

Pricing info for EMR

Issue

• When designating --service emr , the prices reflect EC2 On-demand costs:
  • ex: m5.12xlarge returns $2.304

Expected Behavior

• Price should reflect EC2 on-demand costs plus the EMR premium from the emr pricing table
• From the pricing table, the EMR premium for m5.12xlarge is $0.27/hr, so the output should be $2.304+$0.27 = $2.574

Repro steps

[25/03/21 9:49:22] ➜  amazon-ec2-instance-selector git:(main) ./build/ec2-instance-selector --service emr --price-per-hour-min 0.09 -o table-wide --max-results 5 --region us-east-1 --allow-list=m5\.\*

Instance Type  VCPUs   Mem (GiB)  Hypervisor  Current Gen  Hibernation Support  CPU Arch  Network Performance  ENIs    GPUs    GPU Mem (GiB)  GPU Info  On-Demand Price/Hr
-------------  -----   ---------  ----------  -----------  -------------------  --------  -------------------  ----    ----    -------------  --------  ------------------
m5.12xlarge    48      192        nitro       true         false                x86_64    10 Gigabit           8       0       0                        $2.304
m5.16xlarge    64      256        nitro       true         false                x86_64    20 Gigabit           15      0       0                        $3.072
m5.24xlarge    96      384        nitro       true         false                x86_64    25 Gigabit           15      0       0                        $4.608
m5.2xlarge     8       32         nitro       true         true                 x86_64    Up to 10 Gigabit     4       0       0                        $0.384
m5.4xlarge     16      64         nitro       true         true                 x86_64    Up to 10 Gigabit     8       0       0                        $0.768

Filter by EBS Throughput ?

Is it possible to show or filter the results by EBS Throughput https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-optimized.html ?

Thanks

Unable to connect to EC2 instance

I have shared my ec2 instance screenshot, I am unable to connect to my ec2 instance using console. I have lost my key pair file as well, so i am unable to access it. Let me share error message getting on console:

Moreover, SSH in-bound traffic rule is allowed in my security-group. I don't know to resolve this issue, i tried to reboot instance many times but didn't work for me. Do you guys have any suggest to get out of it. Thanks for focusing

Windows Version

Hello,
Is there a way to run this utility on Windows? It will be great to have this for windows too.

Sluggishness of t3.a EC2 instance

Hi,
I had launched a t3a.large instance to deploy our microservices. The Available RAM of the ec2 instance is around 4GB and CPU usage is not more than 40% most of the time[As per cloud watch monitoring details]. But, Deployed microservices are providing a delayed response compared to microservices running in other instances. Even SSH to the server is taking more than 10 seconds.

can anyone help me to identify the root cause of the sluggishness?

Filter memory per vcpus

It will be nice to have the ability to filter instance providing an amount of memory per vCPUs considering that all vCPUs will be utilized.

Add support for EC2 Multicard

On d1.24xlarge and p4d.24xlarge the reference to the 4x100 Gigabit networking is because of a feature called EC2 Multicard. It might be easier to just add a column called EC2Multicard Support with the number of multicards supported which in this case for dl1 and p4d is 4. Future instances will support this and this feature will continue to expand.

Instance Type  VCPUs   Mem (GiB)  Hypervisor  Current Gen  Hibernation Support  CPU Arch  Network Performance  ENIs    GPUs    GPU Mem (GiB)  GPU Info             On-Demand Price/Hr           
-------------  -----   ---------  ----------  -----------  -------------------  --------  -------------------  ----    ----    -------------  --------             ------------------           
dl1.24xlarge   96      768        nitro       true         false                x86_64    4x 100 Gigabit       60      8       256            Habana Gaudi HL-205  -No Price Filter Specified-
p4d.24xlarge   96      1,152      nitro       true         false                x86_64    4x 100 Gigabit       60      8       320            NVIDIA A100          -No Price Filter Specified-

Cache local data

Performance can be pretty bad when running queries and it's likely due to having to fetch data every time.

eg:

time ec2-instance-selector --memory-min 128 --vcpus-min 33 --cpu-architecture x86_64 -r us-east-1 -o table --max-results 200 --allow-list "m5\.*"

________________________________________________________
Executed in   21.87 secs      fish           external
   usr time  676.80 millis    0.25 millis  676.56 millis
   sys time  246.06 millis    1.39 millis  244.67 millis

I would be nice if the background data could be cached for a little while (5-60m would be a good default)

Support for AWS_PROFILE env var and shared configuration cred setup

At the moment the CLI does not support the use of shared configuration credentials setup (https://github.com/aws/aws-sdk-go#configuring-credentials) and selecting the aws profile by using the standard AWS_PROFILE environment variable.

To make the shared config credentials, I just had to set up the environment variable "AWS_SDK_LOAD_CONFIG=1" and as per the documentation pass the parameter "--profile". Although I could use --profile $AWS_PROFILE to pass the profile, ideally I'd like for that to be implicit, like in the aws cli.

Filter and display instance storage information

Would be cool if you could filter by type of instance storage (HDD/SSD/NVME) and amount of storage, i.e. --instance-storage-type, --instance-storage-min and --instance-storage-max. And then have the instance storage information displayed in the output.

If I get the time later then I might make an attempt at implementing this myself.

Relevant info from aws-sdk-go:

https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#DescribeInstanceTypesInput

    //    * instance-storage-info.disk.count - The number of local disks.
    //
    //    * instance-storage-info.disk.size-in-gb - The storage size of each instance
    //    storage disk, in GB.
    //
    //    * instance-storage-info.disk.type - The storage technology for the local
    //    instance storage disks (hdd | ssd).
    //
    //    * instance-storage-info.nvme-support - Indicates whether non-volatile
    //    memory express (NVMe) is supported for instance store (required | supported)
    //    | unsupported).
    //
    //    * instance-storage-info.total-size-in-gb - The total amount of storage
    //    available from all local instance storage, in GB.
    //
    //    * instance-storage-supported - Indicates whether the instance type has
    //    local instance storage (true | false).

https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#InstanceTypeInfo

    // Describes the instance storage for the instance type.
    InstanceStorageInfo *InstanceStorageInfo `locationName:"instanceStorageInfo" type:"structure"`

    // Indicates whether instance storage is supported.
    InstanceStorageSupported *bool `locationName:"instanceStorageSupported" type:"boolean"`

https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#InstanceStorageInfo

https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#DiskInfo