Comments (10)
To reduce SSM costs we can query 10 secrets at once via GetParametersByPath. Your security concerns are reasonable.
Here is an example to store the variables in the env vars so you can use getenv()
in you config files or elsewhere.
$ssmPath = '[Your environment SSM path e.g. /production/awesome-app/]';
$ssm = new \Aws\Ssm\SsmClient([
'region' => getenv('AWS_REGION'),
'version' => getenv('SSM_VERSION'),
]);
$options = [
'MaxResults' => 10,
'Path' => $ssmPath,
'Recursive' => true,
'WithDecryption' => true,
];
do {
$result = $ssm->getParametersByPath($options);
$options['NextToken'] = $result->get('NextToken');
foreach ($result->get('Parameters') as $parameter) {
putenv(substr($parameter['Name'], strrpos($parameter['Name'], '/') + 1) . '=' . $parameter['Value']);
}
} while ($result->get('NextToken'));
from bref.
I see. That turned out to be expensive and not cost effective for us.
It cost $0.025 per 1,000 API calls to SSM.
It cost 1 API call to decrypt to one parameter.
We average about 6 secrets that need encryption per Lambda Function.
We average 100,000 Lambda executions a day.
Worst case scenario, that is 600,000 SSM API calls per day.
600 * $0.025 = an average cost of $15/day or $450/mnth
We were not seeing the worst case scenario, we did some caching, and tried to work around with it. But ultimately, it cost us far too much because our workload is not consistent throughout the day, and then I run the various bits of work in async parallel across multiple differing Lambda functions. AWS has expanded our concurrent jobs to 20K, and most of the work we do is in spikes of 10-15K jobs in parallel for 30 minutes or so and it can be an hour before we spike again. That means the majority are cold starts.
So we went with AWS KMS instead and found it much more cost efficient.
For one thing, it only costs $0.03 per 10,000 requests.
As we see it, the simplest way that KMS is used to better secure secrets in Lambda projects:
- A customer master key (CMK) is generated in KMS, with access only to production lambda functions (an IAM role) and the administrator account
- All static secrets are stored in an encrypted .env file.
- These .env files containing encrypted secrets (e.g. DB password, mail credentials) as well as any unencrypted non-secrets (e.g. DB user, DB host, etc) can then even be committed to your favorite version control system, if desired.
- Lambda bootstrap function is modified so that it uses the KMS decrypt function (part of AWS SDK) on the encrypted .env file and then exports the values to environment variables, ensuring unencrypted secrets are never stored anywhere but the memory.
- Because the bootstrap is doing the decryption only on cold starts, the values are effectively cached in memory, only costing 1 API call per cold start for all environment variables.
It is important to us that the secrets are never stored on disk, anywhere, at any time, unencrypted. It is also important to us to minimize the cost in API calls as well as the performance impact of decryption.
Thus, we came up with a way to manage encrypted .env files for this purpose.
I supose for smaller Lambda projects that only make a few 1,000 API calls a month, SSM could be cost effective. That is, simple enough to implement that it is worth the few pennies to use.
However, would never use a solution that writes the secrets to disk, even in the lambda environment, because it is critical security flaw that would result in Financial, Governmental, and Healthcare (and anyone else that requires rigorous security practices) institutions could never leverage it.
Addmittedly, there are always multiple ways to solve this sort of problem, and if we can come up with a more cost effective method, without sacrificing security, I will implement it as fast as possible. If for not other reason than it would have a positive impact on Signature Tech Studio's bottom line. :-)
from bref.
I was more thinking about documenting how to use AWS secret manager (as well as another similar tool I forgot the name). These are the recommended way to store and access secrets in Lambda.
from bref.
We wrote a composer package https://github.com/stechstudio/laravel-env-security to manage secrets in Laravel. It would be easy enough to port that over to some bref specific tooling.
from bref.
I don't like env files but it could be reasonable for some projects. I use and prefer a kind of Managing Secrets for Larger Projects and Teams. Is it possible to store a config cache file at runtime in Lambda and it's populated for the next request? This has the advantage to only load "once" the secret keys.
With this, you can use the get_secret
function in your PHP config file and if config cache file is available no AWS SSM API call is needed. See zend-config-aggregator for an example.
return [
'my_secret' => get_secret('MY_SECRET_AWS_KEY'),
]
For development it is useful to use getenv
in the get_secret()
method to allow also to inject env variables. If no env variable is available a AWS SSM API call is executed. With this approach we have two possibilities combined.
from bref.
Thank you that is super useful! That shows that we definitely need a guide for all of that 😄
from bref.
GetParametersByPath
... that does put the costs back on par! As long as we have ten or less secret variable to manage, the performance would be on par as well (1 API call). I dig it.
from bref.
Some notes:
-
AWS SSM Parameter Store is a sub-service of "SSM" and it allows to store values
Cost: free
-
AWS Secrets Manager is basically the same thing with more features:
- audit log of changes
- auto-renewal of secrets
- finer access control
Cost: $0.4/month per secret + $0.05 per 10,000 API calls.
The fixed cost of Secrets Manager is too bad because it was a very good solution. For bigger projects it might be interesting.
Here is a solution that doesn't require the Lambda to fetch the secrets on every execution, or even on every cold start. This is a way to import the value in template.yaml
:
Environment:
Variables:
FOO: '{{resolve:ssm:MY_PARAMETER:1}}'
Note that 1
is the version number.
The downside with SSM is that we must create one parameter per environment variable.
With Secrets Manager we can create 1 secret and store multiple values inside via JSON. Then we can reference them using the :
as a separator for a "JSON query":
Environment:
Variables:
FOO: '{{resolve:secretsmanager:MY_PARAMETER:FOO:BAR}}'
No need to set a version as well.
Note as well that with this technique environment variables are fixed on deploy. They can't be changed dynamically: we must redeploy the application. (note that this is my interpretation and I haven't tested this further)
from bref.
I am trying to define a parameter in template.yaml
and reference it in the environment variables, but that doesn't work. For reference I have asked on StackOverflow: https://stackoverflow.com/questions/55286991/how-to-define-and-use-at-the-same-time-a-ssm-parameter-in-cloudformation
from bref.
I have opened #277 to solve this issue.
In the future we may want to document how to fetch secrets from PHP. That would allow to update configuration values without having to redeploy the application. One step at a time though!
from bref.
Related Issues (20)
- Handle "Response too big" in Console functions HOT 2
- PlanetScale removed the Hobby tier
- Long running CLI commands repeat every 2 minutes HOT 1
- Troubleshooting Bref Docker Deployment with Octane Handler
- Symfony CLI DATABASE_URL Not Found HOT 1
- Can I fire putJobSuccessResult event from AWS Lambda which is developed with Bref? HOT 1
- Feedback for “Custom domain names” HOT 1
- I am getting this issue :-
- OPENSSL_3.3.0' not found (required by /opt/lib/libssl.so.3) HOT 10
- Stopping 1.x maintenance
- Feedback for “Serverless Laravel - Getting started” HOT 3
- NOTICE since upgrading to v2 HOT 1
- Brefv2 500 Errors Not In Logs HOT 5
- PHP ext-intl is required on layer 8.3 HOT 1
- CURL SSL_ERROR_SYSCALL calling another Lambda HOT 7
- Env always return null after update packages to bref 2 version
- max_execution_time not working HOT 3
- Laravel 11, PHP 8.3 mysql driver not enabled, also can't use postgress
- Bref Serverless v4 HOT 12
- php (not fpm) with BREF_LOOP_MAX=1 is 2x slower than php-fpm
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bref.