b3ross/dotenvi

A simple library for generating dotenv files

dotenvi's Issues

throlling aws error

In times of heavy development we can have some failures Throttling: Rate exceeded.

Error describing stack marketplace-api-infrastructure-stack error=Throttling: Rate exceeded

I would like to provide a high level information on what is API throttling and how can we resolve the same below :

-> Amazon throttles API requests for each AWS account on a per-region basis to help the performance of services to ensure that all calls to the Amazon APIs (whether they originate from an application, calls to a command line interface, or the Amazon console) don't exceed the maximum allowed API request rate.

-> Once a user has crossed the threshold of API calls, they run into 'Rate Exceeded' error message

-> These limits are internal information and is not shared by the backend team and hence I can not share the same with you. 

~~~ Steps to resolve the throttling ~~~

This issue can be mitigated by implementing exponential backoffs [1] and/or polling [2] in your configuration code. The idea behind Exponential Backoff is to use progressively longer waits between retries for consecutive error responses. This will decrease the number of API calls made and you will not receive the "Rate Exceeded" error. Examples of how to do so can be found here as well [3]. This will introduce some small delay and randomization of API calls made from your account, reducing rate spikes and the chance of experiencing any throttling events. A description of each can be found below.

-> Polling:
Your application might need to call an API repeatedly to check for an update in status. Before you start polling, give the request time to potentially complete. When you begin polling, use an appropriate sleep interval between successive requests. For best results, use an increasing sleep interval.
Note: This is similar to using random delays as you have used.

-> Calculating the sleep interval(Exponential Backoff):
When you have to poll or retry an API request, we recommend using an exponential backoff algorithm to calculate the sleep interval between API calls. The idea behind exponential backoff is to use progressively longer waits between retries for consecutive error responses.

Also here is an interesting article that I would recommend you to read : 

>> https://medium.com/pablo-perez/dealing-with-throttling-in-cloudformation-part-i-56993706c676 
>> https://medium.com/pablo-perez/dealing-with-throttling-in-cloudformation-part-ii-ece8f6711b74 

Add end-to-end tests

Write a test that leverages example/env.yml and validates the results.

Casing in resolver names

Resolvers can only be lowercase.

default_env: &default_env
  SESSION_SECRET: ${randomString:32}

Can't work because randomString doesn't match:

const regex = new RegExp('\\${([a-z]+):(.*)}');

dotenvi/src/rewriter.ts

Line 28 in f42c0a1

const regex = new RegExp('\\${([a-z]+):(.*)}');

I think just changing this to

const regex = new RegExp('\\${([a-zA-Z]+):(.*)}');

would at least allow for uppercase names. But maybe \w would be even better, to allow alphanumeric and underscores?

Strings with curly braces cause memory error

env.yaml:

default_env: &default_env
  TEST_TPL: "a string with {something} in curly brackets"

❯ dotenvi -s default_env

runs for a while and then:

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two-Factor Authentication, make configure the auth-only level is supported. semantic-release cannot publish with the default auth-and-writes level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

Add a way to add an optional stage in the credstash resolver

Remove `@types/aws-sdk` as it has it's own typings

When installing dotenvi now, seeing the following warning:

dotenvi > @types/[email protected]: This is a stub types definition for aws-sdk (https://github.com/aws/aws-sdk-js). aws-sdk provides its own type definitions, so you don't need @types/aws-sdk installed!

a way to run this when env.yml isn't in root

Support for Node 6?

Currently, when running in Node 6, you get the following error:

Could not write .env file: TypeError: util_1.promisify is not a function

I imagine there are a minimal set of changes to allow for support in Node 6. Thoughts?

Export to .json rather than .env

What's the possibility of including an option to create a JSON file (e.g., config.json) rather than an .env file?

AWS Secret Manager

Any plans to include secret manager out of the box?

undefined variable results in no variable

if cred: or cft: variable doesn't exist, it fails. it would be nice if undefined printed a warning and then remove the variable from .env vs a failure

Add support for recursive evaluations

For example

BUCKET_NAME: foo-bar-${env:IA_ENV}

Optional value should allow a blank string

I'm trying to use an optional value and I'm getting the following error:

I'm trying to add a value in one environment where the string is the empty string:

This code doesn't allow for blanks, but it seems it should:

Need a way to specify an output directory other than the current directory

A flag like --output-dir?

`optional` functionality broken

Version:

development:
  OPTIONAL_VARIABLE: ## Optional variable syntax.  Undefined variables will otherwise cause failures
    value: ${env:SOME_POSSIBLY_UNDEFINED_VARIABLE}
    optional: true

Running the following command: yarn dotenvi -s development yields the following error:

Environment variable SOME_POSSIBLY_UNDEFINED_VARIABLE is undefined
Could not write .env file: Error: Resolver env didn't return any value
    at Rewriter.<anonymous> (/Users/dancaddigan/Code/goldcaddy77/warthog-starter/node_modules/dotenvi/dist/rewriter.js:48:35)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/dancaddigan/Code/goldcaddy77/warthog-starter/node_modules/dotenvi/dist/rewriter.js:4:58)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Apparently dotenvi doesn't respect `AWS_PROFILE` or `AWS_DEFAULT_PROFILE`

@goldcaddy77 to add more information

Handling numbers

Currently, the following will fail:

default_env: &default_env
  PORT: 3000

The workaround is to wrap in quotes, but might want to support numbers.