Skip to content

Commit

Permalink
fetch spot current price to speed up loading data without caching
Browse files Browse the repository at this point in the history
  • Loading branch information
bwagner5 committed Oct 13, 2024
1 parent 14d4dfc commit 8ff1b10
Show file tree
Hide file tree
Showing 20 changed files with 327 additions and 309 deletions.
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<h4>A CLI tool and go library which recommends instance types based on resource criteria like vcpus and memory.</h4>

<p>
<a href="https://golang.org/doc/go1.17">
<a href="https://golang.org/doc/go1.23">
<img src="https://img.shields.io/github/go-mod/go-version/aws/amazon-ec2-instance-selector?color=blueviolet" alt="go-version">
</a>
<a href="https://opensource.org/licenses/Apache-2.0">
Expand All @@ -25,7 +25,7 @@

## Summary

There are over 270 different instance types available on EC2 which can make the process of selecting appropriate instance types difficult. Instance Selector helps you select compatible instance types for your application to run on. The command line interface can be passed resource criteria like vcpus, memory, network performance, and much more and then return the available, matching instance types.
There are over 800 different instance types available on EC2 which can make the process of selecting appropriate instance types difficult. Instance Selector helps you select compatible instance types for your application to run on. The command line interface can be passed resource criteria like vcpus, memory, network performance, and much more and then return the available, matching instance types.

If you are using spot instances to save on costs, it is a best practice to use multiple instances types within your auto-scaling group (ASG) to ensure your application doesn't experience downtime due to one instance type being interrupted. Instance Selector will help to find a set of instance types that your application can run on.

Expand Down Expand Up @@ -334,6 +334,9 @@ Filter Flags:
-e, --ena-support Instance types where ENA is supported or required
-f, --fpga-support FPGA instance types
--free-tier Free Tier supported
--generation int Generation of the instance type (i.e. c7i.xlarge is 7) (sets --generation-min and -max to the same value)
--generation-max int Maximum Generation of the instance type (i.e. c7i.xlarge is 7) If --generation-min is not specified, the lower bound will be 0
--generation-min int Minimum Generation of the instance type (i.e. c7i.xlarge is 7) If --generation-max is not specified, the upper bound will be infinity
--gpu-manufacturer string GPU Manufacturer name (Example: NVIDIA)
--gpu-memory-total string Number of GPUs' total memory (Example: 4 GiB) (sets --gpu-memory-total-min and -max to the same value)
--gpu-memory-total-max string Maximum Number of GPUs' total memory (Example: 4 GiB) If --gpu-memory-total-min is not specified, the lower bound will be 0
Expand Down Expand Up @@ -385,7 +388,8 @@ Suite Flags:
Global Flags:
--cache-dir string Directory to save the pricing and instance type caches (default "~/.ec2-instance-selector/")
--cache-ttl int Cache TTLs in hours for pricing and instance type caches. Setting the cache to 0 will turn off caching and cleanup any on-disk caches. (default 168)
--cache-ttl int Cache TTLs in hours for pricing and instance type caches. Setting the cache to 0 will turn off caching and cleanup any on-disk caches.
--debug Debug - prints debug log messages
-h, --help Help
--max-results int The maximum number of instance types that match your criteria to return (default 20)
-o, --output string Specify the output format (table, table-wide, one-line, interactive)
Expand Down
4 changes: 2 additions & 2 deletions THIRD_PARTY_LICENSES
Original file line number Diff line number Diff line change
Expand Up @@ -818,8 +818,8 @@ THE SOFTWARE.

------

** github.com/imdario/mergo; version v0.3.11 --
https://github.com/imdario/mergo
** dario.cat/mergo; version v1.0.1 --
dario.cat/mergo

Copyright (c) 2013 Dario Castañé. All rights reserved.
Copyright (c) 2012 The Go Authors. All rights reserved.
Expand Down
15 changes: 13 additions & 2 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,9 @@ const (
defaultRegionEnvVar = "AWS_DEFAULT_REGION"
defaultProfile = "default"
awsConfigFile = "~/.aws/config"
spotPricingDaysBack = 30
// 0 means the last price
// increasing this results in a lot more API calls to EC2 which can slow things down
spotPricingDaysBack = 0

tableOutput = "table"
tableWideOutput = "table-wide"
Expand Down Expand Up @@ -100,6 +102,8 @@ const (
freeTier = "free-tier"
autoRecovery = "auto-recovery"
dedicatedHosts = "dedicated-hosts"
debug = "debug"
generation = "generation"
)

// Aggregate Filter Flags
Expand Down Expand Up @@ -206,6 +210,7 @@ Full docs can be found at github.com/aws/amazon-` + binName
cli.BoolFlag(freeTier, nil, nil, "Free Tier supported")
cli.BoolFlag(autoRecovery, nil, nil, "EC2 Auto-Recovery supported")
cli.BoolFlag(dedicatedHosts, nil, nil, "Dedicated Hosts supported")
cli.IntMinMaxRangeFlags(generation, nil, nil, "Generation of the instance type (i.e. c7i.xlarge is 7)")

// Suite Flags - higher level aggregate filters that return opinionated result

Expand All @@ -219,9 +224,10 @@ Full docs can be found at github.com/aws/amazon-` + binName
cli.ConfigStringFlag(profile, nil, nil, "AWS CLI profile to use for credentials and config", nil)
cli.ConfigStringFlag(region, cli.StringMe("r"), nil, "AWS Region to use for API requests (NOTE: if not passed in, uses AWS SDK default precedence)", nil)
cli.ConfigStringFlag(output, cli.StringMe("o"), nil, fmt.Sprintf("Specify the output format (%s)", strings.Join(cliOutputTypes, ", ")), nil)
cli.ConfigIntFlag(cacheTTL, nil, env.WithDefaultInt("EC2_INSTANCE_SELECTOR_CACHE_TTL", 168), "Cache TTLs in hours for pricing and instance type caches. Setting the cache to 0 will turn off caching and cleanup any on-disk caches.")
cli.ConfigIntFlag(cacheTTL, nil, env.WithDefaultInt("EC2_INSTANCE_SELECTOR_CACHE_TTL", 0), "Cache TTLs in hours for pricing and instance type caches. Setting the cache to 0 will turn off caching and cleanup any on-disk caches.")
cli.ConfigPathFlag(cacheDir, nil, env.WithDefaultString("EC2_INSTANCE_SELECTOR_CACHE_DIR", "~/.ec2-instance-selector/"), "Directory to save the pricing and instance type caches")
cli.ConfigBoolFlag(verbose, cli.StringMe("v"), nil, "Verbose - will print out full instance specs")
cli.ConfigBoolFlag("debug", nil, nil, "Debug - prints debug log messages")
cli.ConfigBoolFlag(help, cli.StringMe("h"), nil, "Help")
cli.ConfigBoolFlag(version, nil, nil, "Prints CLI version")
cli.ConfigStringOptionsFlag(sortDirection, nil, cli.StringMe(sorter.SortAscending), fmt.Sprintf("Specify the direction to sort in (%s)", strings.Join(cliSortDirections, ", ")), cliSortDirections)
Expand Down Expand Up @@ -273,6 +279,10 @@ Full docs can be found at github.com/aws/amazon-` + binName
fmt.Printf("An error occurred when initialising the ec2 selector: %v", err)
os.Exit(1)
}
if flags[debug] != nil {
debugLogger := log.New(os.Stdout, time.Now().UTC().Format(time.RFC3339)+" DEBUG ", 0)
instanceSelector.SetLogger(debugLogger)
}
shutdown := func() {
if err := instanceSelector.Save(); err != nil {
log.Printf("There was an error saving pricing caches: %v", err)
Expand Down Expand Up @@ -417,6 +427,7 @@ Full docs can be found at github.com/aws/amazon-` + binName
FreeTier: cli.BoolMe(flags[freeTier]),
AutoRecovery: cli.BoolMe(flags[autoRecovery]),
DedicatedHosts: cli.BoolMe(flags[dedicatedHosts]),
Generation: cli.IntRangeMe(flags[generation]),
}

if flags[verbose] != nil {
Expand Down
67 changes: 33 additions & 34 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,56 +1,55 @@
module github.com/aws/amazon-ec2-instance-selector/v2

go 1.20
go 1.23

require (
github.com/aws/aws-sdk-go v1.46.6
github.com/aws/aws-sdk-go-v2 v1.24.0
github.com/aws/aws-sdk-go-v2/config v1.26.1
github.com/aws/aws-sdk-go-v2/service/ec2 v1.128.0
github.com/aws/aws-sdk-go-v2/service/pricing v1.21.6
dario.cat/mergo v1.0.1
github.com/aws/aws-sdk-go-v2 v1.32.2
github.com/aws/aws-sdk-go-v2/config v1.27.43
github.com/aws/aws-sdk-go-v2/service/ec2 v1.182.0
github.com/aws/aws-sdk-go-v2/service/pricing v1.32.2
github.com/blang/semver/v4 v4.0.0
github.com/charmbracelet/bubbles v0.16.1
github.com/charmbracelet/bubbletea v0.24.2
github.com/charmbracelet/lipgloss v0.7.1
github.com/evertras/bubble-table v0.15.2
github.com/imdario/mergo v0.3.16
github.com/charmbracelet/bubbles v0.20.0
github.com/charmbracelet/bubbletea v1.1.1
github.com/charmbracelet/lipgloss v0.13.0
github.com/evertras/bubble-table v0.17.0
github.com/mitchellh/go-homedir v1.1.0
github.com/muesli/termenv v0.15.2
github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/spf13/cobra v1.7.0
github.com/spf13/cobra v1.8.1
github.com/spf13/pflag v1.0.5
go.uber.org/multierr v1.11.0
)

require (
github.com/atotto/clipboard v0.1.4 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.16.12 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.10 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.7.2 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.9 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.18.5 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.5 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.26.5 // indirect
github.com/aws/smithy-go v1.19.0 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.41 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.17 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.2 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.2 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.2 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.32.2 // indirect
github.com/aws/smithy-go v1.22.0 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/containerd/console v1.0.4-0.20230313162750-1ae8d489ac81 // indirect
github.com/charmbracelet/x/ansi v0.2.3 // indirect
github.com/charmbracelet/x/term v0.2.0 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/mattn/go-isatty v0.0.18 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.14 // indirect
github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/reflow v0.3.0 // indirect
github.com/rivo/uniseg v0.2.0 // indirect
github.com/sahilm/fuzzy v0.1.0 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.7.0 // indirect
golang.org/x/term v0.6.0 // indirect
golang.org/x/text v0.4.0 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/sahilm/fuzzy v0.1.1 // indirect
golang.org/x/sync v0.8.0 // indirect
golang.org/x/sys v0.24.0 // indirect
golang.org/x/text v0.16.0 // indirect
)
Loading

0 comments on commit 8ff1b10

Please sign in to comment.