agilestacks / components Goto Github PK
View Code? Open in Web Editor NEWStack components in Hub format
License: Apache License 2.0
Stack components in Hub format
License: Apache License 2.0
WARN: Env var
TF_VAR_name=postgres-db
set bycomponent.postgresql.name
overriden bydns.name
toarkadi123
Also, when deployed together with postgresql
component:
Parameter
component.postgresql.name
environment variable setupCOMPONENT_NAME
does not match new setupTF_VAR_name
We set this early in happy-meal's parent k8s-aws stack:
- name: dns
brief: DNS
kind: user
parameters:
- name: name
brief: DNS name
env: TF_VAR_name
I as a developer want to use Agile Stacks as a developer accelerator for myself and peers in my team
I as a user want to execute backup
and restore
operation for my etcd cluster.
I as a user can select [EBS or EFS] as MongoDB component storage option. Hub component should support this
0.8.3 PostgreSQL chart use persistence.storageClass
in values.yaml
to specify PVC spec.storageClassName
in templates/pvc.yaml
. Not having a value assigned cause PVC to get stuck with error.
storageClass
to gp2
or -
for default storage class.ebs-gp2
StorageClass not being created by Kubernetes aws-ebs provisioner - kubectl get sc
.I as a user can select [EBS or EFS] for Postgres component storage option. Hub component should support this
I as a user want to make a backup/restore operation for my postgresql-rds component
Enable SSO for Wowza admin panel using Dex.
Extract ecr
out of stack-k8s-aws. Instead
ecr` must be implemented as hub component.
Research Spartakus (https://github.com/kubernetes-incubator/spartakus) to collect Kubernetes usage statistics
To avoid collisions I want to store tfstate
by following convention:
s3://mybucket/<component>/<domain>/<component_name>.tfstate
or
s3://mybucket/<component>/<domain>/<component_name>/terraform.tfstate
where <component>
is the constant (but not user defined input).
Tyk is a lightweight, open source API Gateway and Management Platform enables you to control who accesses your API, when they access it and how they access it. Tyk will also record detailed analytics on how your users are interacting with your API and when things go wrong.
Tyk Overview:
https://github.com/TykTechnologies/tyk
Kubernetes-based Tyk setup:
https://github.com/TykTechnologies/tyk-kubernetes
OpenID Connect for integration with Okta IDP:
https://tyk.io/docs/security/your-apis/openid-connect/
This will allow us to (a) apply components to clouds other than AWS and (b) pass parameters to components to configure behavior - currently its only on/off switch.
Related https://app.zenhub.com/workspace/o/agilestacks/stack-k8s-aws/issues/55
Regarding TLS enabled or not Trafiek should be correct served from app or apps subdomain.
Valid correspondent links should be generated on Happy Meal.
component.jenkins.instances
and component.jenkins.namespaces
are not Jenkins component parameters.
Research telepresence as for local development
Reproducible every time on my machine:
--- /usr/bin/make [deploy] (../../../components/prometheus)
Creating /Users/arkadi/Work/agilestacks/components/prometheus/.helm/repository
Creating /Users/arkadi/Work/agilestacks/components/prometheus/.helm/repository/cache
Creating /Users/arkadi/Work/agilestacks/components/prometheus/.helm/repository/local
Creating /Users/arkadi/Work/agilestacks/components/prometheus/.helm/plugins
Creating /Users/arkadi/Work/agilestacks/components/prometheus/.helm/starters
Creating /Users/arkadi/Work/agilestacks/components/prometheus/.helm/cache/archive
Creating /Users/arkadi/Work/agilestacks/components/prometheus/.helm/repository/repositories.yaml
$HELM_HOME has been configured at /Users/arkadi/Work/agilestacks/components/prometheus/.helm.
Not installing Tiller due to 'client-only' flag having been set
Happy Helming!
Client: &version.Version{SemVer:"v2.6.2", GitCommit:"be3ae4ea91b2960be98c07e8f73754e67e87963c", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.6.2", GitCommit:"be3ae4ea91b2960be98c07e8f73754e67e87963c", GitTreeState:"clean"}
helm --kube-context="arkadi50.kubernetes.delivery" --tiller-namespace="kube-system" install charts/kube-prometheus \
--name prometheus-kube-prom \
--namespace prometheus \
--replace \
--wait \
--values values-kube-prometheus.yaml
Error: render error in "kube-prometheus/charts/exporter-node/templates/service.yaml": template: kube-prometheus/charts/exporter-node/templates/service.yaml:12:18: executing "kube-prometheus/charts/exporter-node/templates/service.yaml" at <.Values.service.type>: can't evaluate field type in type interface {}
make[1]: *** [install_kube_prom] Error 1
As a user of Kubernetes stack with EFK component deployed I want to open Kibana UI and receive a nice pre-configured dashboard that allows me to search thru Kubernetes and pods logs.
Currently, I'm presented with a cryptic index setup interface. This is very unappealing. In general, I have little interest diving into Kibana internals to setup UI. It should be done once by a component developer.
Currently, PostgreSQL parameters configured on Template screen does not influence anything in the deployment. PostgreSQL Helm Chart we use has no instances
- only one database is created with the name from component.postgresql.database
parameter. storage
is volume
. storageType
is unused.
@akranga commented on Mon Jul 24 2017
Gitlab has OKTA plugin. We don't know if OKTA plugin provides all desired functionality.
We need to have clickable demo to review as a team of gaps and findings
I as a user want to use nginx as an alternative to the traefik as ingress controller for my kubernetes cluster. This request came during the meeting with innominds
Rolling update updates replica but not updates container image
If I put unsupported(?) key into component's parameters in stack's params.yaml, like this:
- component: pg1-rds
name: component.postgresql
parameters:
- name: rds.instanceType
default: db.t2.small
(here 'default' key is not supported(?) and is result of a copy-paste from component params)
elaborate phase would not complain and proceed without warning.
With "infected" parameter elaborate result looks following:
- name: component.postgresql.rds.instanceType
component: pg1-rds
default: db.t2.small
While make deploy:
2017/12/07 15:16:00 component.postgresql.rds.instanceType|pg1-rds => ``
And finally fail like this:
2017/12/07 15:24:24 deploy ***pg1-rds*** (3/3)
2017/12/07 15:24:24 Parameters flattened (pg1-rds):
2017/12/07 15:24:24 :cloud.vpc.id|pg1-rds => `` (env:TF_VAR_vpc_id)
2017/12/07 15:24:24 :component.postgresql.name|pg1-rds [postgresql-rds] => `` (env:COMPONENT_NAME)
2017/12/07 15:24:24 :component.postgresql.rds.name|pg1-rds [postgresql] => `` (env:TF_VAR_rds_name)
2017/12/07 15:24:24 :component.postgresql.database|pg1-rds => `` (env:TF_VAR_database_name)
2017/12/07 15:24:24 :component.postgresql.user|pg1-rds [postgres] => `` (env:TF_VAR_database_username)
2017/12/07 15:24:24 :component.postgresql.password|pg1-rds => `` (env:TF_VAR_database_password)
2017/12/07 15:24:24 :component.postgresql.rds.instanceType|pg1-rds [db.t2.micro] => `` (env:TF_VAR_instance_type)
2017/12/07 15:24:24 Parameters expanded:
2017/12/07 15:24:24 component.name => `pg1-rds`
2017/12/07 15:24:24 cloud.vpc.id => `vpc-73f30718` (env:TF_VAR_vpc_id)
2017/12/07 15:24:24 component.postgresql.name => `pg1-rds` (env:COMPONENT_NAME)
2017/12/07 15:24:24 component.postgresql.rds.name => `postgresql` (env:TF_VAR_rds_name)
2017/12/07 15:24:24 component.postgresql.database => `devrktest` (env:TF_VAR_database_name)
2017/12/07 15:24:24 component.postgresql.user => `postgres` (env:TF_VAR_database_username)
2017/12/07 15:24:24 component.postgresql.password => `qwerty123` (env:TF_VAR_database_password)
2017/12/07 15:24:24 component.postgresql.rds.instanceType => `` (env:TF_VAR_instance_type)
2017/12/07 15:24:24 Wrote state `.hub/dev-rk.kubernetes.delivery.state`
2017/12/07 15:24:24 Component parameters:
2017/12/07 15:24:24 cloud.availabilityZone => `eu-central-1c`
2017/12/07 15:24:24 cloud.kind => `aws`
2017/12/07 15:24:24 cloud.region => `eu-central-1`
2017/12/07 15:24:24 cloud.sshKey => `agilestacks`
2017/12/07 15:24:24 cloud.vpc.id => `vpc-73f30718` (env:TF_VAR_vpc_id)
2017/12/07 15:24:24 component.etcd.cluster.enabled => `true`
2017/12/07 15:24:24 component.name => `pg1-rds`
2017/12/07 15:24:24 component.postgresql.database => `devrktest` (env:TF_VAR_database_name)
2017/12/07 15:24:24 component.postgresql.database|pg1-rds => `devrktest`
2017/12/07 15:24:24 component.postgresql.name => `pg1-rds` (env:COMPONENT_NAME)
2017/12/07 15:24:24 component.postgresql.name|pg1-rds => `pg1-rds`
2017/12/07 15:24:24 component.postgresql.password => `qwerty123` (env:TF_VAR_database_password)
2017/12/07 15:24:24 component.postgresql.password|pg1-rds => `qwerty123`
2017/12/07 15:24:24 component.postgresql.rds.instanceType => `` (env:TF_VAR_instance_type)
2017/12/07 15:24:24 component.postgresql.rds.instanceType|pg1-rds => ``
2017/12/07 15:24:24 component.postgresql.rds.name => `postgresql` (env:TF_VAR_rds_name)
2017/12/07 15:24:24 component.postgresql.user => `postgres` (env:TF_VAR_database_username)
2017/12/07 15:24:24 component.postgresql.user|pg1-rds => `postgres`
2017/12/07 15:24:24 dns.baseDomain => `kubernetes.delivery`
2017/12/07 15:24:24 dns.name => `dev-rk`
2017/12/07 15:24:24 terraform.bucket.name => `terraform.agilestacks.com`
2017/12/07 15:24:24 terraform.bucket.region => `us-east-1`
2017/12/07 15:24:24 No templates for component `pg1-rds`
2017/12/07 15:24:24 Component environment:
2017/12/07 15:24:24 COMPONENT_NAME=pg1-rds
2017/12/07 15:24:24 TF_VAR_database_name=devrktest
2017/12/07 15:24:24 TF_VAR_database_password=qwerty123
2017/12/07 15:24:24 TF_VAR_database_username=postgres
2017/12/07 15:24:24 TF_VAR_instance_type=
2017/12/07 15:24:24 TF_VAR_rds_name=postgresql
2017/12/07 15:24:24 TF_VAR_vpc_id=vpc-73f30718
--- /usr/bin/make [deploy] (../../../components/postgresql-rds)
yes yes | terraform init -get=true -no-color -force-copy \
-backend=true -input=false \
-backend-config="bucket=terraform.agilestacks.com" \
-backend-config="region=us-east-1" \
-backend-config="key=pg1-rds/postgresql.dev-rk.kubernetes.delivery/terraform.tfstate" \
-backend-config="profile=agilestacks" \
. .terraform/postgresql.dev-rk.kubernetes.delivery
Initializing configuration from: "."...
Initializing the backend...
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your environment. If you forget, other
commands will detect it and remind you to do so if necessary.
cd .terraform/postgresql.dev-rk.kubernetes.delivery && \
terraform plan -no-color -refresh=true -module-depth=-1 -out=terraform.tfplan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_vpc.selected: Refreshing state...
data.aws_subnet_ids.selected: Refreshing state...
The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed. Cyan entries are data sources to be read.
Your plan was also saved to the path below. Call the "apply" subcommand
with this plan file and Terraform will exactly execute this execution
plan.
Path: terraform.tfplan
+ aws_db_instance.postgresql
address: "<computed>"
allocated_storage: "32"
apply_immediately: "<computed>"
arn: "<computed>"
auto_minor_version_upgrade: "true"
availability_zone: "<computed>"
backup_retention_period: "30"
backup_window: "04:00-04:30"
character_set_name: "<computed>"
copy_tags_to_snapshot: "false"
db_subnet_group_name: "${aws_db_subnet_group.all.name}"
endpoint: "<computed>"
engine: "postgres"
engine_version: "9.6.2"
final_snapshot_identifier: "terraform-aws-postgresql-rds-snapshot"
hosted_zone_id: "<computed>"
identifier: "postgresql"
identifier_prefix: "<computed>"
kms_key_id: "<computed>"
license_model: "<computed>"
maintenance_window: "sun:04:30-sun:05:30"
monitoring_interval: "0"
monitoring_role_arn: "<computed>"
multi_az: "false"
name: "devrktest"
option_group_name: "<computed>"
parameter_group_name: "default.postgres9.6"
password: "<sensitive>"
port: "5432"
publicly_accessible: "false"
replicas.#: "<computed>"
resource_id: "<computed>"
skip_final_snapshot: "true"
status: "<computed>"
storage_encrypted: "false"
storage_type: "gp2"
tags.%: "1"
tags.Name: "postgresql-rds"
timezone: "<computed>"
username: "postgres"
vpc_security_group_ids.#: "<computed>"
+ aws_db_subnet_group.all
arn: "<computed>"
description: "Managed by Terraform"
name: "<computed>"
name_prefix: "postgresql"
subnet_ids.#: "2"
subnet_ids.2366951888: "subnet-3f7ec654"
subnet_ids.991094391: "subnet-252e8368"
tags.%: "1"
tags.Name: "postgresql-all-subnets"
+ aws_security_group.default
description: "Managed by Terraform"
egress.#: "<computed>"
ingress.#: "1"
ingress.3847841800.cidr_blocks.#: "1"
ingress.3847841800.cidr_blocks.0: "10.0.0.0/16"
ingress.3847841800.from_port: "5432"
ingress.3847841800.ipv6_cidr_blocks.#: "0"
ingress.3847841800.protocol: "tcp"
ingress.3847841800.security_groups.#: "0"
ingress.3847841800.self: "false"
ingress.3847841800.to_port: "5432"
name: "<computed>"
owner_id: "<computed>"
tags.%: "1"
tags.Name: "postgresql-db"
vpc_id: "vpc-73f30718"
Plan: 3 to add, 0 to change, 0 to destroy.
cd .terraform/postgresql.dev-rk.kubernetes.delivery && \
terraform apply -no-color -Xshadow=false terraform.tfplan
aws_db_subnet_group.all: Creating...
arn: "" => "<computed>"
description: "" => "Managed by Terraform"
name: "" => "<computed>"
name_prefix: "" => "postgresql"
subnet_ids.#: "" => "2"
subnet_ids.2366951888: "" => "subnet-3f7ec654"
subnet_ids.991094391: "" => "subnet-252e8368"
tags.%: "" => "1"
tags.Name: "" => "postgresql-all-subnets"
aws_security_group.default: Creating...
description: "" => "Managed by Terraform"
egress.#: "" => "<computed>"
ingress.#: "" => "1"
ingress.3847841800.cidr_blocks.#: "" => "1"
ingress.3847841800.cidr_blocks.0: "" => "10.0.0.0/16"
ingress.3847841800.from_port: "" => "5432"
ingress.3847841800.ipv6_cidr_blocks.#: "" => "0"
ingress.3847841800.protocol: "" => "tcp"
ingress.3847841800.security_groups.#: "" => "0"
ingress.3847841800.self: "" => "false"
ingress.3847841800.to_port: "" => "5432"
name: "" => "<computed>"
owner_id: "" => "<computed>"
tags.%: "" => "1"
tags.Name: "" => "postgresql-db"
vpc_id: "" => "vpc-73f30718"
aws_db_subnet_group.all: Creation complete (ID: postgresql0022058fd2db50223d88c54366)
aws_security_group.default: Creation complete (ID: sg-f66b1c9c)
aws_db_instance.postgresql: Creating...
address: "" => "<computed>"
allocated_storage: "" => "32"
apply_immediately: "" => "<computed>"
arn: "" => "<computed>"
auto_minor_version_upgrade: "" => "true"
availability_zone: "" => "<computed>"
backup_retention_period: "" => "30"
backup_window: "" => "04:00-04:30"
character_set_name: "" => "<computed>"
copy_tags_to_snapshot: "" => "false"
db_subnet_group_name: "" => "postgresql0022058fd2db50223d88c54366"
endpoint: "" => "<computed>"
engine: "" => "postgres"
engine_version: "" => "9.6.2"
final_snapshot_identifier: "" => "terraform-aws-postgresql-rds-snapshot"
hosted_zone_id: "" => "<computed>"
identifier: "" => "postgresql"
identifier_prefix: "" => "<computed>"
kms_key_id: "" => "<computed>"
license_model: "" => "<computed>"
maintenance_window: "" => "sun:04:30-sun:05:30"
monitoring_interval: "" => "0"
monitoring_role_arn: "" => "<computed>"
multi_az: "" => "false"
name: "" => "devrktest"
option_group_name: "" => "<computed>"
parameter_group_name: "" => "default.postgres9.6"
password: "<sensitive>" => "<sensitive>"
port: "" => "5432"
publicly_accessible: "" => "false"
replicas.#: "" => "<computed>"
resource_id: "" => "<computed>"
skip_final_snapshot: "" => "true"
status: "" => "<computed>"
storage_encrypted: "" => "false"
storage_type: "" => "gp2"
tags.%: "" => "1"
tags.Name: "" => "postgresql-rds"
timezone: "" => "<computed>"
username: "" => "postgres"
vpc_security_group_ids.#: "" => "1"
vpc_security_group_ids.2015187407: "" => "sg-f66b1c9c"
Error applying plan:
1 error(s) occurred:
* aws_db_instance.postgresql: 1 error(s) occurred:
* aws_db_instance.postgresql: Error creating DB Instance: InvalidParameterValue: Invalid DB Instance class:
status code: 400, request id: 80a6b980-347f-4c9f-8ca6-5bf28497ecf3
Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.
make[1]: *** [apply] Error 1
---
2017/12/07 15:24:48 Failed to deploy pg1-rds: exit status 2
make: *** [deploy] Error 1
Proposed behaviour:
elaborate phase failing with message saying that there is no value for param.
Identify and implement customizations for Spinnaker that will work for both
hub
componentMigrate Jenkins component from stack-k8s-aws to hub-component
Currently the values of component parameters entered on Template creation screen aren't used anywhere. Some parameters doesn't relate to what component actually supports.
In Makefile we test for presence of a Helm release with:
ifeq ($(findstring $(COMPONENT_NAME), $(shell $(helm) list -q | xargs)),)
So in case, for example, automation-hub-db
and automation-hub-dbclient
are both installed, then reinstallation of automation-hub-db
is not possible.
I as an admin want to deploy a new version of prod or stage without losing user data
Hey,
Looks like subj.
Steps to reproduce (on stack):
Error applying plan:
1 error(s) occurred:
Note: got it first time, while performing same sequence many time already -
could be a glitch of TF or AWS itself too.
Enable PostgreSQL RDS with existing postgresql-rds component. The thing to watch for is parameters names that are the same for postgresql and postgresql-rds - parameters.component
must be set to address specific component.
I as a AgileStacks user want to know if I can provision a new stack by checking if quotas allow me to do that. I might want able to put quota information on dashboards
This can help: https://awslimitchecker.readthedocs.io/en/latest/
By default nvidia-docker doesn't support CoreOS gives favor to Ubuntu. It is not clear if we should add support of Ubuntu
OS in addition to CoreOS
As a user I want to deploy a Zookeeper from Happy Meal screen. Zookeeper is a component requested by one of our potential clients
An error occurs while producing partial operations on stack with COMPONENT=gitlab-ce defined.
`COMPONENT_NAME=gitlab` set by `component.gitlab.name` does not match component name `gitlab-ce`
The same message outputs as a warning if no COMPONENT argument is defined.
Currently the component is very basic. There are only postgres
database and user with hardcoded password supersecret
.
Parameterize component to accept component.postgresql.database
and component.postgresql.user
. Generate strong password output into component.postgresql.password
.
Alternatively, introduce PostgreSQL operator.
By default nvidia-docker
doesn't support CoreOS. However it should be possible to port this from Ubuntu and export required installables from specially created a docker container (ubuntu perhaps
). This task aim to proof or disproof this concept
Drop connection to vault pod
/bin/sh: line 0: kill: (8942) - No such process
/bin/sh: line 0: kill: (8940) - No such process
Start proxy to pod/vault-service-vault-2633935217-lsdk3
nohup kubectl --context="arkadi13.kubernetes.delivery" --namespace="automation-hub" port-forward vault-service-vault-2633935217-lsdk3 8200:8200 &
Connecting to vault at http://127.0.0.1:8200
curl: (7) Failed to connect to 127.0.0.1 port 8200: Connection refused
Waiting for pod/vault-service-vault-2633935217-lsdk3 to connect (1)...
error: unable to forward port because pod is not running. Current status=Pending
curl: (7) Failed to connect to 127.0.0.1 port 8200: Connection refused
Waiting for pod/vault-service-vault-2633935217-lsdk3 to connect (2)...
We're happy if GitLab doesn't work for a non-TLS-enabled stack - and it's Makefile may check for that and exit with an error, but breaking Okta login for everyone is not an option.
(py) arkadi@hal9017 ~/Work/agilestacks/dev14-14 $ kubectl logs --since=5m dex-554615886-9mdsd
time="2017-11-30T11:05:41Z" level=error msg="Connector \"okta\" returned error when creating callback: expected callback URL \"http://auth.arkadi124.arkadi121.kubernetes.delivery/callback\" did not match the URL in the config \"https://auth.arkadi124.arkadi121.kubernetes.delivery/callback\""
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.