Comments (9)
Using Jinja2 templating everywhere would also be a nice workaround for YAML limitation: it can have anchor/alias for a mapping/object, but not for a sequence.
We would just define the sequence as a variable and then put it into places where we want it using Jinga2 tags.
from gitlabform.
We wrapped a few tools together into a CLI and threw it into a docker container:
The tools include:
gitlabform
for managing the configuration of existing repositoriespython-gitlab
for creating and deleting repositories and groupsansible-vault
for managing secrets
Feel free to make your own wrapper based on ours, or contribute. MRs welcome!
from gitlabform.
...and this could remove uglyness of group, project and usernames used in the integration tests being both hardcoded in provided configs as well as in variables for all the rest of the code. With Jinja2 and env variables support we could just set the variables as env variables and then reference them in the configs. We could even randomize thise entities names to make the tests run on new entities each time and not run into problems with GitLab having a delay with actually deleting resources, to which you run into when you have static resource names.
I am starting to love this idea! ๐
from gitlabform.
Hi @weakcamel , thank you for this report.
Frankly I have not thought about adding secure secrets handling to gitlabform yet.
At Egnyte we deploy gitlabform using Puppet, so we keep its config in a git repo, with all the values that are secret encrypted using asymmetric encryption - https://github.com/voxpupuli/hiera-eyaml. So on the machine which runs gitlabform once a day the config file is deployed in unencrypted form. This allows us to give developers access to the config repo and the public key so they can add MRs to add their new secrets encrypted. It is not perfect, but it's is safe enough for us thanks to other means of security that we use.
I think you can use hiera-eyaml even without using Puppet as it has cli to generate encryption keys, encrypt single value, edit encrypted file, decrypt single value, decrypt whole file - all the features to deploy the flow similar to ours. There is also https://github.com/mozilla/sops that seems to work in a similar way and has more features, but I have not use it myself.
Of course, as always, any PRs to gitlabform adding securing of the secrets into it are welcome too!
from gitlabform.
For your information (and for the record): for now I've worked around that with Jinja2 templates.
It's not quite ideal, however using vault-cli helped me to very easily fill in the secrets directly from Hashicorp Vault.
Once peopledoc/vault-cli#113 is finished, I'm going to also break down the config.yml
into smaller bits as I expect we may end up with quite a sizeable configuration.
I may still look at adding secret support (or maybe Jinja2 support?) to Gitlabform at some point, but probably not just yet.
from gitlabform.
I had the same problem, ended up just using sed in the gitlab-ci.yml to replace like so:
sed -i -e 's/SED_REPLACE_JIRA_PASSWORD/'"$JIRA_PASSWORD"'/g' config.yml
It will still end up in the log though ๐ข
from gitlabform.
How about adding support for automatic replacement of some strings with environment variables + changing logging code to not print any values with verbosity level up to โverboseโ (they would still be printed with โdebugโ on though).
This would allow setting them as secrets in GitLab project, setting them from Vault in a pre-script etc. and seems easy to implement.
What do you think?
from gitlabform.
...or we could add built-in Jinja2 support to automatically pre-process the while config with built-in code similar to answers for https://stackoverflow.com/q/25862071/2693875 to be able to reference env variables anywhere.
This would generalize @mkjmdski โs code for Jinja2 support for file contents and setting GitLab URL and token from env variables.
from gitlabform.
Yes, an ability to grab values from environment or to pass arbitrary Jinja variables on via CLI to gitlabform (e. g. -e foo=bar -e baz=qux
) and so on would be great! The option with environment vars would probably be more useful if someone wanted to make the logs public (would not expose the secrets in command-line parameters).
The change in logging also sounds like a reasonable, simple way to hide the secrets.
from gitlabform.
Related Issues (20)
- Migrate `resource_group` configuration feature to use python-gitlab library
- Migrate `schedules` configuration feature to use python-gitlab library
- Migrate `tags` configuration feature to use python-gitlab library HOT 1
- Migrate group's `group_members` configuration feature to use python-gitlab library HOT 1
- Change to protected_environment deploy_access_levels is not detected HOT 1
- Error while processing group sharing HOT 10
- Resource Groups - don't fail if key not exist HOT 10
- Use of expires_at in project members fails due to debug function HOT 4
- Project Transfer feature's few acceptance tests failing after GitLab 16.7 release HOT 2
- Add `enforce` support for `hooks` configuration HOT 3
- Add support for member_role_id HOT 7
- Is there a way to manage CI/CD settings at project level for "Keep artifacts from most recent successful jobs" using gitlab forms. HOT 3
- black formatting issue after upgrading markupsafe to version 2.1.4
- Entities marked as delete are remove and then added again when not present HOT 3
- 3.9.0 appears to break while setting a project webhook with a token HOT 8
- Transfer project does not work when gitlabform run with ALL_DEFINED or ALL as the target HOT 2
- Uploading Project Avatar HOT 6
- Merge request approvals settings of group HOT 4
- Feature Flags support HOT 5
- Add retry feature HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gitlabform.