pivotal/cf-onboarding

Iterating on Onboarding Week with lightning speed and cunning

CF Onboarding

Onboarding Tracker: Local

Onboarding Tracker Local is a hands-on tour of some of the systems you'll be working on / with during your time as a CF Pivot. It's good solo-able material, and good preparation for the paired Onboarding Week.

Whether you are Engineering, Product, Design, Docs, or one of dozens of other roles that make our engine run and you're curious about the essentials of the CF or Concourse user experience, then this backlog is for you.

Onboarding Tracker: OSS (a.k.a. Onboarding Week)

Onboarding Tracker OSS is intended to be a facilitated exploration of Open Source Cloud Foundry, embarked upon with other Cloud Foundry newbies. Unlike Tracker Local, it is more of a puzzle than a tour. It provides:

1. A self-paced learning environment paired with others who are learning too.
2. A coherent, if cursory, overview of a complicated product.
3. Empathy for the customer who uses that product.
4. The opportunity to struggle through these problems and really learn the material.
5. A little knowledge of the breadth of work 70+ teams are doing around the world.

To run an Onboarding Week in your office, read the facilitation docs and join the #cf-onboarding-week channel on Pivotal Slack.

Usage

Import stories to Tracker (from source)

The stories in this repo are divided by epic (e.g. Deploying with GCP, Redis CUPS, etc.) They are provided in .prolific format. To grab the most recent versions of stories from master or another branch:

1. Clone this repo
2. Run ./build local or ./build oss (requires Docker)
3. Import your newly created csv file (onboarding-tracker.csv) to a new Tracker project
4. WARNING: concatenating CSVs is a risky/inadvisable business, so stories generated by this script are slightly buggy in inconsequential ways (e.g. the first letter or two of a single story title goes missing).

Contributing

Depending on personal preferance, either edit stories in the .prolific text files themselves or convert the prolific file to a CSV and upload it to a temporary Tracker project, reverting back to .prolific format once you're ready to make a PR. The second one takes more time, but removes the risk of accidental prolific syntax errors.

cf-onboarding's Issues

Improve everything about the DNS instructions

Epic: Container networking

Have an epic that would cover container networking.

Add create-security-group story

A feature of security groups that we didn't use in the security group stories was creating and binding a new security group. This seems to involve writing a json file and then either running a cf command or directly making requests against the cc api (?!).

docs: https://docs.cloudfoundry.org/concepts/asg.html#procedures

Epic: Create a BOSH release

Most CF software is intended to be consumed as a BOSH release, so it would be really helpful to have an epic which goes through the entire process.

EPIC: opsman and ERT

Most of the epics and stories in this backlog are focused on the open source cloud foundry and bosh, however many teams work on PCF and PCF tiles. It would awesome if there was an epic which went through the opsman and ERT lifecycle.

In addition to this, it would be great to also install a service tile and go through the interaction between PCF and a service.

cc @kkallday

Additional direction needed in Nameservers section

In the Nameservers section, select the Use custom nameservers (enter below) radio button.
Otherwise, your Namerserver will remain the default.

https://www.pivotaltracker.com/story/show/150192734

Image in first Concourse story is broken

Story: set up proxy variables for use with firewall

Broken links on story: "Create an admin user in UAA using the uaac CLI"

The link: https://docs.cloudfoundry.org/adminguide/uaa-user-management.html leads to "page not found". The link is referenced in both the "How?" and "Resources" sections.

Need to specify username when setting up redis server

If you create your redis db with RedisLabs, you need to specify your username when creating the cf service via cf cups...e.g.

cf cups redis -p "host, port, username, password"

This username will be whatever email address you used to sign up for the service with.

(https://www.pivotaltracker.com/story/show/142993563)

[Local Track] Need instruction to login as Admin user in story to "Create an org and space"

In this story https://www.pivotaltracker.com/story/show/150296148

Some of us got hung up because we were logged in as User rather than Admin. It would help to have an instruction to login as Admin in this story.

Register a domain name image link is broken

GCP load balancer instructions are misleading

Let people know to ignore the "Update your DNS records to point to your GCP load balancer" step in lbs instructions.

Story: add a networking/monitoring story that uses dig to investigate

Story: Please explain GCP concepts

In this story, we set up iam service accounts. It would be helpful to have some explanation of what iam and service accounts are.

Story: create security story to generate TLS certs/keys

Pull App Development module out of GCP Track

• Clarifies difference between local (Part 1, soloing) and OSS (Part 2, pairing) tracks
• Happens at next release

Better introduction for how to build Docker images

This intro on docker's website really helped us understand how to build a docker image better than the article linked to in the story (https://www.pivotaltracker.com/story/show/143145055). We ended up installing Docker for Mac and using it that way.

Bug: word "repo" in cf-deployment deploy story should be a link

Make nmap networking story instructions clearer

Describe what traffic we expect to block

Bug: How to avoid hitting external IP quota

And what is creating so many of them? And why are they not being destroyed when whatever is using them is destroyed?

Point out where to change the number of compilation workers

https://github.com/pivotal-cf/onboarding-week/blob/9f0a832bc848bc696eddea6946d33bd579e403be/deploy_oss.prolific#L260

This should be more clear. In the .yml, you should edit the workers line under the compilation section.

Have a pre-existing Concourse environment

Have a ready to go Concourse env so one could have a track that is dedicated to only setting up pipelines and not the concourse env.

Rewrite the TLS cert story to explain each openssl flag and argument

..instead of just being copy-pastable

Add Concourse load balancer story

Explain how to create a load balancer for Concourse without bbl that will work with deployment instructions so that the CF lb doesn't have to be torn down.

Mention cf --trace and bbl --debug earlier / at all

link for setting up local concourse vm on vagrant gives a 404 error

http://concourse.ci/vagrant.html gives me a 404 error :(

Please configure GITBOT

Pivotal uses GITBOT to synchronize Github issues and pull requests with Pivotal Tracker.
Please add your new repo to the GITBOT config-production.yml in the Gitbot configuration repo.
If you don't have access you can send an ask ticket to the CF admins. We prefer teams to submit their changes via a pull request.

Steps:

• Fork this repo: cfgitbot-config
• Add your project to config-production.yml file
• Submit a PR

If there are any questions, please reach out to [email protected].

Create bosh ssh story for bosh troubleshooting section

Create separate local and OSS modules for Concourse

... so that the instructions are less confusing

Not clear where to get app instance index from

not super clear where to get app-instance-index from when SSH'ing into our newly deployed app. Realized that it's the zero index of the number of instances you specified in the manifest.yml, but took us a second... (https://www.pivotaltracker.com/story/show/142969525)

Add yml checker resource to yml stories

Redis Password is not user password

story: https://www.pivotaltracker.com/story/show/145951205

The password is not the one used when creating your account. The database defaults to not having a password, and you can configure it to require one. I don't think it needs the username either, just the password (if you set it up).

Bug: spring-music-app can't be resurrected after it's killed, or restarted once it's stopped

This has been observed once and may be a user configuration error.

Epic: implement a basic Concourse resource

Include instructions on how to run through stories in Intro story

• Deliver and Accept stories that you feel confident about
• Leave stories in the Delivered state if you have follow up questions for your facilitator or would like to revisit something about them

Generate a TLS cert story should be clearer

https://www.pivotaltracker.com/story/show/150192732

It isn't immediately clear from the story that the user must set the path for -keyout and -out. It would be clearer if the intended command were expressed in a way similar to the below:

openssl req -x509 -newkey rsa:2048 -keyout CHOOSE-PRIVATE-KEY-FILE-NAME.pem -out CHOOSE-CERT-FILE-NAME.pem -nodes

Update Security Groups Story

story: https://www.pivotaltracker.com/story/show/145951219

Claims that the security group should be all_open on gcp, but we see dns and public_networks. They still seem to be named reasonably -- dns allows port 53 on tcp or udp.

From the story:

#### Expected Result
If you've deployed a full Cloud Foundry on GCP you should have only one security group, applied to both staging and running apps: `all_open`.

If you're working with PCF Dev, you should see three security groups, one of which is named `all_pcfdev`.

For either circumstance if you run `cf security-groups <SECURITY-GROUP NAME>` you'll see that they do exactly what it sounds like they do&mdash;leave everything open by allowing containers to access all IPs on any port.

Because of the `all_open` (or `all_pcfdev`) security group any other group would be redundant.

The output we see:

→ cf security-groups
Getting security groups as onboarding
OK

     Name              Organization   Space
#0   public_networks
#1   dns

→ cf security-group public_networks
Getting info for security group public_networks as onboarding
OK

Name    public_networks
Rules
	[
		{
			"destination": "0.0.0.0-9.255.255.255",
			"protocol": "all"
		},
		{
			"destination": "11.0.0.0-169.253.255.255",
			"protocol": "all"
		},
		{
			"destination": "169.255.0.0-172.15.255.255",
			"protocol": "all"
		},
		{
			"destination": "172.32.0.0-192.167.255.255",
			"protocol": "all"
		},
		{
			"destination": "192.169.0.0-255.255.255.255",
			"protocol": "all"
		}
	]

No spaces assigned

→ cf security-group dns
Getting info for security group dns as onboarding
OK

Name    dns
Rules
	[
		{
			"destination": "0.0.0.0/0",
			"ports": "53",
			"protocol": "tcp"
		},
		{
			"destination": "0.0.0.0/0",
			"ports": "53",
			"protocol": "udp"
		}
	]

No spaces assigned

CPU quota / compilation worker story needs better context

@SimonParker
This story stumped us; our CPU quota is exactly 24, so per instructions, we need to decrease the number of compilation workers.
In the cloud config generated from bosh, we have compilation.workers: 6. We didn't know if that was too high, or what number we needed to decrease it to. 5? 4?
We decided to plow ahead and when we hit the story requiring bosh -d cf deploy, see if we got the CPU quota error, and fix it then. (We did wonder whether the reason this fix is done preemptively was because hitting the CPU quota is somehow expensive?)

Bug: clarify cloud-config story

Be explicit that user must run bosh update-cloud-config to update cloud config

Story Deploy CF using BOSH + cf-deployment should be more explicit about link to repo

The story https://www.pivotaltracker.com/story/show/142952109 contains instructions to clone the repo, which is linked earlier in the story. This is a little confusing as there is not really an indication that the link goes to a repo, or an association between the word "repo" and the link.

Suggestion:

Make the word "repo" the link to the repository.

Create an OPS file story

Both bbl and bosh accept ops files. Add a story that involves creating an ops-file (rather than just passing one in).

Update "Set up your CF target domain" story

• Admin password is no longer uaa_scim_users_admin_password

Clarify the use of the -o switch when using bosh deploy

The stories relating to bosh deploy should clarify what -o does and what the files in the operations folder do.

The wording makes the use of the operation folder feel like it must be applied, but in reality the files under operations are just options one can activate, and at any time after the creation of a bosh deployed cluster as long as one executes the bosh deploy.

Rephrase README.md and FACILITATION.md to be less engineering focused

Setting up GCP account story instructions are misleading

@SimonParker
Some confusion on this story. Text reads:
"Then, follow these steps to set up your GCP service account, replacing ACCOUNT_NAME with the name you just chose and PROJECT_ID with your assigned project's name"
We're not finding any reference to choosing a name for this account. Presumably, we can make up anything we want? The fact that we pull it from a previous step suggests it needs to correspond with something, but we don't know what.

Include GCP Quota warning in story, not just in FAQs

Add additional route to application question

@SimonParker
A question we weren't able to answer for ourselves:
Why does cf create-route exist? Wouldn't it be enough to only have cf map-route? It sounds like creating but not mapping a route leaves it useless, and the process of mapping a route is exactly the same whether or not you created the route previously.

CPU quota / compilation worker story needs better context

Add detail to the domain registration story

re: inter- and intra-component routing in CF