Comments (8)
We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/100895292.
from diego-release.
Hey @camelpunch,
You're right, the manifest generation should be better documented in general, and in particular as it pertains to other infrastructures. In the mean time, here's what the BOSH-Lite instructions are, and I'll explain what needs to change:
cd ~/workspace/diego-release
./scripts/generate-deployment-manifest \
~/deployments/bosh-lite/director.yml \
manifest-generation/bosh-lite-stubs/property-overrides.yml \
manifest-generation/bosh-lite-stubs/instance-count-overrides.yml \
manifest-generation/bosh-lite-stubs/persistent-disk-overrides.yml \
manifest-generation/bosh-lite-stubs/iaas-settings.yml \
manifest-generation/bosh-lite-stubs/additional-jobs.yml \
~/deployments/bosh-lite
The first argument to the script is just a stub with the director UUID:
---
director_uuid: YOUR_DIRECTOR_UUID
The second is a property overrides stub. You can see a full list of overridable properties with this command:
grep '(( property_over' manifest-generation/diego.yml | cut -d'(' -f3- | cut -d' ' -f2 | sort | uniq
Your manifest will "compile" with out overriding any properties, but there are some that you need to override for your cluster to work, and it's impossible to have a sane default for them because it depends a lot on things like your networking setup. Probably the simplest thing you can do to start for this stub is something like this:
property_overrides:
etcd:
require_ssl: false
executor:
allow_privileged: true
garden-linux:
allow_networks: ["10.10.0.2/32"] # This is the DNS for your AWS VPC; it depends on your CIDR range.
btrfs_store_size_mb: 25000
skip_cert_verify: true
This is similar to how it's deployed in the test environment, but a few key things will be missing:
- syslog forwarding to a service like papertrail
- encrypted traffic between etcd clients and servers, as well as peer-to-peer traffic amongst etcd servers
- the receptor will not register a route with the router, so you can't do things like point xray at it
- I don't know whether SSH access to your containers will work
The third, fourth, and sixth arguments allow you to override instance counts for jobs, persistent disk settings for jobs (specifically etcd), and colocate additional templates from additional releases. You probably don't need to modify these to get started, and can just use the relevant BOSH-Lite stubs.
The final (seventh) argument is a directory where you need to have a file called cf.yml
which is your deployment manifest for the cf that you're deploying your diego alongside.
The third argument is an IaaS settings stub. Here's a quick way to see what it requires:
grep '(( iaas_settings' manifest-generation/diego.yml | cut -d'(' -f3- | cut -d' ' -f2 | sort | uniq
Here's an example that works assuming your VPC has been set up accordingly (with some names/IDs you'll have to change to match your IaaS configuration):
iaas_settings:
compilation_cloud_properties:
availability_zone: us-east-1b
instance_type: c3.large
resource_pool_cloud_properties:
- cloud_properties:
availability_zone: us-east-1b
elbs:
- NAME_OF_ELB_FOR_DIEGO_CONTAINER_SSH_ACCESS
name: access_z1
- cloud_properties:
availability_zone: us-east-1c
elbs:
- NAME_OF_ELB_FOR_DIEGO_CONTAINER_SSH_ACCESS
name: access_z2
- cloud_properties:
availability_zone: us-east-1d
elbs:
- NAME_OF_ELB_FOR_DIEGO_CONTAINER_SSH_ACCESS
name: access_z3
- cloud_properties:
availability_zone: us-east-1b
name: brain_z1
- cloud_properties:
availability_zone: us-east-1c
name: brain_z2
- cloud_properties:
availability_zone: us-east-1d
name: brain_z3
- cloud_properties:
availability_zone: us-east-1b
name: cc_bridge_z1
- cloud_properties:
availability_zone: us-east-1c
name: cc_bridge_z2
- cloud_properties:
availability_zone: us-east-1d
name: cc_bridge_z3
- cloud_properties:
availability_zone: us-east-1b
ephemeral_disk:
size: 174080
type: gp2
name: cell_z1
- cloud_properties:
availability_zone: us-east-1c
ephemeral_disk:
size: 174080
type: gp2
name: cell_z2
- cloud_properties:
availability_zone: us-east-1d
ephemeral_disk:
size: 174080
type: gp2
name: cell_z3
- cloud_properties:
availability_zone: us-east-1b
name: database_z1
- cloud_properties:
availability_zone: us-east-1c
name: database_z2
- cloud_properties:
availability_zone: us-east-1d
name: database_z3
- cloud_properties:
availability_zone: us-east-1b
name: route_emitter_z1
- cloud_properties:
availability_zone: us-east-1c
name: route_emitter_z2
- cloud_properties:
availability_zone: us-east-1d
name: route_emitter_z3
stemcell:
name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent
version: latest
subnet_configs:
- name: diego1
subnets:
- cloud_properties:
security_groups:
- SOME_SECURITY_GROUP
subnet: SOME_SUBNET_ID
dns:
- 10.10.0.2
gateway: 10.10.50.1
name: default_unused
range: 10.10.50.0/25
reserved:
- 10.10.50.2 - 10.10.50.9
static:
- 10.10.50.10 - 10.10.50.31
- name: diego2
subnets:
- cloud_properties:
security_groups:
- SOME_SECURITY_GROUP
subnet: SOME_OTHER_SUBNET_ID
dns:
- 10.10.0.2
gateway: 10.10.114.1
name: default_unused
range: 10.10.114.0/25
reserved:
- 10.10.114.2 - 10.10.114.9
static:
- 10.10.114.10 - 10.10.114.31
- name: diego3
subnets:
- cloud_properties:
security_groups:
- SOME_SECURITY_GROUP
subnet: YET_ANOTHER_SUBNET_ID
dns:
- 10.10.0.2
gateway: 10.10.178.1
name: default_unused
range: 10.10.178.0/25
reserved:
- 10.10.178.2 - 10.10.178.9
static:
- 10.10.178.10 - 10.10.178.31
The hardest part is setting up AWS with the correct VPC, CIDRs, subnets, ELBs, security groups, and AZs. However, you may be able to piggyback off of an existing setup for CF on AWS if you have one. Especially if you're using a tool like CloudFormation to automate setting up your AWS environment, it might not be that bad.
from diego-release.
Thanks @Amit-PivotalLabs! I do need to have the receptor register a route with the gorouter, as I'm doing this just to test that https://github.com/pivotal-cf-experimental/diegotaskscheduler (submits tasks and monitors them) is possible to deploy in a CF instance.
Great detail though, I'll have a play with this.
from diego-release.
Combine this with your property overrides:
property_overrides:
receptor:
cors_enabled: true # not sure if this is needed in general, or some special thing for xray
domain_names: ["receptor.YOUR_DOMAIN"] # subdomain can be anything, not necessarily "receptor"
register_with_router: true
username: RECEPTOR_BASIC_AUTH_UN
password: RECEPTOR_BASIC_AUTH_PW
from diego-release.
Cheers. I was hoping to get away with YOUR_DOMAIN being an IP. I guess I don't know it up front, though?
from diego-release.
There's numerous options here.
On one end of the spectrum, you're deploying alongside a full CF, and you have an ELB balancing traffic in front of your routers with wildcard DNS pointing at your ELB. In that case, you use the domain from your wildcard record. This is the type of solution you'd use in a production setting.
On the other end, you could just spin up a jumpbox behind the firewall and work from there, and target the IP of one of the access VMs (where the receptor processes live). If you want to know up front, you could configure one of the access VMs to use a static IP, and make sure to allocate one for it in your deployment manifest that falls within the appropriate CIDR and doesn't clash with any other static IPs (e.g. the ones used for the database jobs).
There's all sorts of options in the middle, depending on how/whether you have DNS, an ELB, any HA Proxies, static IPs for the gorouters, etc.
from diego-release.
Hi, @camelpunch,
Yes, we'd like to improve documentation about the AWS-specific information to supply for a Diego deployment. As @Amit-PivotalLabs pointed out, this information is primarily concentrated in the iaas-settings.yml
stub, although depending on the desired deployment configuration you may also want to set values in persistent-disk-overrides.yml
as well. I've added https://www.pivotaltracker.com/story/show/100909610 to address this, and I expect we'll get to it soon as part of polishing Diego's documentation for BOSH operators.
Thanks,
Eric, CF Runtime Diego PM
from diego-release.
Great. I'll close this then. Sorry for going silent on it: it turned out not to be worth the effort for what I was doing.
from diego-release.
Related Issues (20)
- [PR REVIEW]: allow sending network traffic usage for app metrics
- [EXECUTOR PR REVIEW]: send container network traffic metrics via logging client HOT 1
- [REP PR REVIEW]: enhance test to expect new network traffic usage fields
- Make BBS more resilient to API port being unavailable HOT 3
- RetireActualLRP is not emitting events HOT 3
- [EXECUTOR/REP PR REVIEW]: disable log rate limit metrics for tasks HOT 2
- Add support for docker images with attestation information HOT 3
- [BBS PR REVIEW]: BBS - Make BBS DesiredLRPHandler send the Stop/Update LRP requests to rep in parallel
- Rep unable to removed cached items after management API restart HOT 2
- Calculate the CPUWeight directly in the Executor HOT 10
- Make max-containers setting configurable HOT 7
- [REP/BBS PR REVIEW]: Add tags to task logs HOT 1
- [BBS] Use scheduling info instead of the whole desiredLRP HOT 5
- [BBS] Application CPU assignment HOT 3
- [Executor] Send error to app logs if starting a container fails HOT 14
- Use SHA algorithm for content digest in URLUploader HOT 2
- Add CPU Entitlement gauge metric & Deprecate CPU Entitlement counter metric HOT 10
- [BBS] Add request metrics for BBS endpoints HOT 5
- [Envoy] Envoy proxy healthchecks
- Convert components to a go module HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diego-release.