microsoft/code-with-engineering-playbook

This is the playbook for "code-with" customer or partner engagements

ISE Engineering Fundamentals Customer/Partner Engineering Playbook

An engineer or data scientist working on an ISE project...

• Has responsibilities to their team – mentor, coach, and lead.
• Knows their playbook. Follows their playbook. Fixes their playbook if it is broken. If they find a better playbook, they copy it. If somebody could use your playbook, give them yours.
• Leads by example. Models the behaviors we desire both interpersonally and technically.
• Strives to understand how their work fits into a broader context and ensures the outcome.

This is our playbook. All contributions welcome! Please feel free to submit a pull request to get involved.

Note: If you are reading this on GitHub - head over to https://microsoft.github.io/code-with-engineering-playbook/ for a better reading experience

Why Have A Playbook

• To increase overall efficiency for team members and the whole team in general.
• Reduce the number of mistakes and avoid common pitfalls.
• Strive to be a better engineer and learn from other people's shared experience.

"The" Checklist

If you do nothing else follow the Engineering Fundamentals Checklist! It's here to help follow the Engineering Fundamentals.

Structure of a Sprint

A breakdown of sections according to the structure of an Agile sprint.

General Guidance

• Keep the code quality bar high.
• Value quality and precision over ‘getting things done’.
• Work diligently on the one important thing.
• As a distributed team take time to share context via wiki, teams and backlog items.
• Make the simple thing work now. Build fewer features today, but ensure they work amazingly. Then add more features tomorrow.
• Avoid adding scope to a backlog item, instead add a new backlog item.
• Our goal is to ship incremental customer value.
• Keep backlog item details up to date to communicate the state of things with the rest of your team.
• Report product issues found and provide clear and repeatable engineering feedback!
• We all own our code and each one of us has an obligation to make all parts of the solution great.

QuickLinks

• Engineering Fundamentals Checklist
• Structure of a Sprint

Engineering Fundamentals

• Accessibility
• Agile Development
• Automated Testing
• Code Reviews
• Continuous Delivery (CD)
• Continuous Integration (CI)
• Design
• Developer Experience
• Documentation
• Engineering Feedback
• Observability
• Security
• Privacy
• Source Control
• Reliability

Fundamentals for Specific Technology Areas

• Machine Learning Fundamentals
• User-Interface Engineering

Contributing

See CONTRIBUTING.md for contribution guidelines.

code-with-engineering-playbook's Issues

Suggestion: Repository should match its own source control best practice

Source control page has requirements that don't seem to have been met yet for this repo

• agree on their branch, release and merge strategy
• define their approach to commit history (linear or non-linear)
• agree on how to name branches (e.g. user/your_alias/feature_name)

Can we agree the answers for these questions, for this repo, and add to the contributing page?

Bash best practices

It's pretty common that we use lots of bash scripts to wire things together.

Is it worth including a section on how to lint/check bash?

Currently use the following to run shellcheck as part of an azuredevops pipeline.

- bash: |
    echo "This checks for formatting and common bash errors. See wiki for error details and ignore options: https://github.com/koalaman/shellcheck/wiki/SC1000"
    export scversion="stable"
    wget -qO- "https://storage.googleapis.com/shellcheck/shellcheck-${scversion?}.linux.x86_64.tar.xz" | tar -xJv
    sudo mv "shellcheck-${scversion}/shellcheck" /usr/bin/
    rm -r "shellcheck-${scversion}"
    shellcheck ./scripts/*.sh
  displayName: "Validate Scripts: Shellcheck"

Guidance on how to include the right SMEs

Guidance on rules who to include and how do you find the right people to look at your changes.

Suggestion: Secrets management for k8s

Possibly put an example for k8s in the secrets management section.

Verify the Code Review Recipe for Python

Guidance on how to CR the language and what tools can be used in the process to support the CR process

Suggestion: Terraform best practices.

Proposing we have a section relating to best practices wrt Terraform.

Verify Code Review Recipe for C#

Validate the content for the specific language and make sure it is up to date.

Use the following template to structure the content:

1. linters / code formatters
2. enforcement in build pipelines
3. Starter Code Review Checklist
4. tools/extensions/plugins for VS Code and VS
5. Language specifics

Verify the Code Review Recipe for Bash

Guidance on how to CR the language and what tools can be used in the process to support the CR process

Pull request description template

• design description if applicable
• solution description
• what needs to be included
• Links to bugs/tasks
• instruction on how to apply and run the code

Ensure timely code review process

How to avoid delays in code reviews. Tips and tricks on how a process can be improved:

• triage for CR?
• CR champion per project?

Add recipe content for running remote retrospectives with Retrium

The boston and dallas teams have had quite a bit of success with running retrospectives with remote teams using Retrium. This article should describe how to implement the concepts defined in the root retrospective readme using Retrium.

Verify Code Review Recipe for JavaScript and TypeScript

Guidance on how to CR the language and what tools can be used in the process to support the CR process

Investigate: measurable CRs

• more measurable code review
• how can we measure it?
• can we collect metrics?

add guidance on git branching strategies

Under https://github.com/microsoft/code-with-engineering-playbook/tree/master/source-control
There should be guidance around when git branching strategy like git flow is ideal, as compared to branching strategies like release flow / truck based development. The post https://codeburst.io/trunk-based-development-vs-git-flow-a0212a6cae64 lists some of the considerations

content for definition-of-ready

This is for the new proposed outline in #53

This content should define the criteria a team should use to determine if a story is ready to pull into a sprint. Example is, team understands the implementation, how to prove work meets acceptance, estimated, etc.

add conceptual content for agile-ceremonies/stand-ups

This is for the new proposed outline in #53

Using the suggested template below as a guide, describe the concept of standups and what role they play in scrum. This should include (but not limited to)

• Inputs: What are team members expected to do before standup (i.e. update tasks status and hours remaining, have a plan for what they are working on next)
• Participants
• Outputs: What decisions or actions are needed to conclude the meeting?

Suggested Template

Goals

What are the intended outcomes of the ceremonies?

Participation

What members of the team should participate?

Impact

What positive impact has been observed as a result of this practice?

Measures

How might one measure the impact of this ceremony positive or negative?

Facilitation Guidance

How might one go about running the ceremony? Is there a general pattern that the ceremony should follow?

TypeScript recipe suggests deprecated tslint

The TypeScript recipe suggests TSLint as a linter. TSLint is being deprecated in favor of improving ESLint's TypeScript support. These docs can be updated with some instructions on using ESLint to lint TypeScript code (e.g. https://github.com/typescript-eslint/typescript-eslint/blob/master/docs/getting-started/README.md).

I can open a PR with updates (tomorrow since it's EOD 😄)

Verify the Code Review Recipe for JavaScript

Guidance on how to CR the language and what tools can be used in the process to support the CR process

Guidance on why we need CRs

There is a goals section already in there. Add additional reasoning why we know, CRs are an important process.

content for planning

This is for the new proposed outline in #53

Using the suggested template below as a guide, describe the concept of planning in scrum. This should include (but not limited to)

• Inputs: What are the pre-reqs for planning (i.e. estimated & prioritized stories)
• Participants
• Outputs: What decisions or actions are needed to conclude the meeting?
• Determining team capacity
• Tasking stories in sprint backlog
  • Size of tasks
  • Hour estimates
  • Updating todo hours
• Dividing work into WIP (work-in-progress) or Workstreams

Suggested Template

Goals

What are the intended outcomes of the ceremonies?

Participation

What members of the team should participate?

Impact

What positive impact has been observed as a result of this practice?

Measures

How might one measure the impact of this ceremony positive or negative?

Facilitation Guidance

How might one go about running the ceremony? Is there a general pattern that the ceremony should follow?

Suggestion: tidy up the "Structure of Sprint" page so it is customer friendly

My understanding of the 'code-with engg playbook' is that it is a resource that we give our customers so that they get a picture of what to expect from a code-with engagement. The 'Tech Lead's Checklist' does that by providing a set of core principles that we don’t compromise on to ensure that we follow sound engineering practices. The 'Structure of Sprint' page starts to do that with the Day 1/Day 2/Day 3 etc breakdown format, but my concern is that there is too much information on this page. For e.g. under Project Start, whilst it is great to have a list of activities that happen at project start, guidance about creating good user stories can be moved elsewhere. Also, there are some topics that are already talked about in the Tech lead checklist that may not be necessary on this page.

Minimalism should be removed

https://github.com/Microsoft/code-with-engineering-playbook/blob/master/Engineering/BestPractices/MinimalismRules.md

It seems this manifesto popped up in hackernews a few months ago and hasn't even got much attention. Rightfully so because these are not widely accepted principles like the agile manifesto. And therefore doesn't belong in a Microsoft engineering playbook.

In fact, some items in this are gibberish IMO.

• Perfect is enemy of good, first do it, then do it right, then do it better
• Basics, first. Follow always a top-down thinking starting by the best-practices of CS.
• Think different. Simple is harder than complex, which means you'll need to use your creativity.

While some other items are a no-brainer or things that can be inferred from existing agile practices.

If we want to add a manifesto into the playbook, let's add the agile manifesto.

PR - size guidance

Guidance on how to keep PR small and focused

Tooling: Add Visual Studio & VS Code guidance

Add guidance on how to optimize the CR experience using Visual Studio and VS Code to support an easier review process vs. using the web UI of git.com

1. supported plugins
2. step by step setup guide
3. tips and tricks

Suggestion: Customer Facing Retrospective Guidelines

It would be helpful to have a different doc for customer facing retrospectives. What are some guidelines for the types of feedback should MSFT employees be presenting/not-presenting in front of the customer. What is the objective for the retro? Is it to come away with action items for the dev crew (i.e. extract feedback from the customer about how we can do better)? Is it to provide feedback to the customer about how they can do better? Or both?

This came up in a discussion today between myself and Michael Green. I came here to see if anything had been documented for this. The current document does not provide guidance on the nuances of doing this with the customer.

Guidance on test code review

• test coverage should not decrease
• tests should be reviewed - ensure cases are covered

Verify the Code Review Recipe for Go

Guidance on how to CR the language and what tools can be used in the process to support the CR process

content on estimation

This is for the new proposed outline in #53

This content should guide a new team in how to establish how they will be doing estimation.

• Agreed scale (i.e. fibonacci)
• Conversion ration of points in scale back to unit of time (i.e. 1 point = 1 day for 1 contributor)
• Establishing baseline stories (i.e. pick story that represents a 5 and one that represents 2 if using fibonacci)

Setup instructions ADO for CR

• step by step guidance on how to setup your ADO project to enable CR and enforce them
• tricks for working with ADO (tools)

Document How to Gain Access to a Customer's Azure DevOps

Oftentimes a customers use Azure DevOps, but do not have AAD set up correctly.

Without AAD set up, it is impossible to grant permissions to @microsoft.com addresses. Since setting up AAD is not always intuitive for the customer, people often suggest to use a non-Microsoft MSA account. Then the customer would end up granting project access to @hotmail.com or @outlook.com addresses, which isn't something CELA would support.

The steps to properly set up VSTS with @microsoft accounts as members would help us "do the right thing" when collaborating with customers and help us get to the "coding with" quicker.

Create Code Review Recipe for Markdown

Guidance on how to CR the language and what tools can be used in the process to support the CR process

add conceptual content for building test cases/plans at testing/test-cases

This is for the new proposed outline in #53

This content should provide guidance on how to build test cases to assert acceptance is met for a given story. Guidance should include but not limited to

• Avoiding false positives with positive and negative test cases
• test case structure (pre-conditions, actions, assertions/verification)
• guidance on organizing/grouping test cases into test plans (i.e. full regression, smoke test, functional area regression)

This article should also provide justification for why test case development is important. For example, test cases become a living documentation of how to test areas of the system and can be easily handed over to the customer after the engagement.

Defining this content is important as its foundational to automated testing. Automation should be a reflection of the concepts defined in this article.

content on grooming

This is for the new proposed outline in #53

Using the suggested template below as a guide, describe the concept of grooming and what role it plays in scrum. This should include (but not limited to)

• Inputs: What are requirements to have this meeting (i.e. stories ready for estimation)
• Participants required
• Outputs: What decisions or actions are needed to conclude the meeting?

May also elaborate on

• Criteria for knowing when story is ready to estimate (i.e. implementation is understood, testable, clear understanding of what success looks like)
• Estimation
  • Voting without influencing
  • Resolving estimation conflicts (two people are sizing differently)

Suggested Template

Goals

What are the intended outcomes of the ceremonies?

Participation

What members of the team should participate?

Impact

What positive impact has been observed as a result of this practice?

Measures

How might one measure the impact of this ceremony positive or negative?

Facilitation Guidance

How might one go about running the ceremony? Is there a general pattern that the ceremony should follow?

Guidance on code reviews/PRs with respect to process/tooling added/improved

The Code Reviews guidance should have:

• Guidance on PRs/CRs with respect to development process
  • Labeling/marking tasks waiting for review, how to make sure efficient PR/CR velocity etc.
• Additional references to instructions on how to set up branch policies (AzDO/GitHub)

Add Engineering Feedback content

Add related content to:

• General Guidance (README.md)
• Tech Lead's Engineering Fundamentals Checklist (TECH-LEADS-CHECKLIST.md)

Create new section for Engineering Feedback

Edits for Logging section of Pillars

While https://github.com/microsoft/code-with-engineering-playbook/tree/master/observability#logging had a good first run of content, it could be formatted/edited in a more structured way.

Proposal for the structure of the 3 sections of the Observability Pillars:

• Summarize what the particular is and how it fits into a dev/ops workflow
• Point out primary use case examples
• Best Practices & Guidance for the particular pillar (may expand into its own doc over time)
• Links To Useful Tools & Further Guidance

Verify Code Review Recipe for Terraform

Guidance on how to CR the language and what tools can be used in the process to support the CR process

Edits for Metrics section of Pillars

While https://github.com/microsoft/code-with-engineering-playbook/tree/master/observability#metrics had a good first run of content, it could be formatted/edited in a more structured way.

Proposal for the structure of the 3 sections of the Observability Pillars:

• Summarize what the particular pillar is and how it fits into a dev/ops workflow
• Point out primary use case examples
• Best Practices & Guidance for the particular pillar (may expand into its own doc over time)
• Links To Useful Tools & Further Guidance

Edits for Tracing section

While https://github.com/microsoft/code-with-engineering-playbook/tree/master/observability#logging had a good first run of content, it could be formatted/edited in a more structured way.

Proposal for the structure of the 3 sections of the Observability Pillars:

• Summarize what the particular is and how it fits into a dev/ops workflow
• Point out primary use case examples
• Best Practices & Guidance for the particular pillar (may expand into its own doc over time)
• Links To Useful Tools & Further Guidance

WIP: Weekly Sprint Developer Loop

Monday Sprint Planning

Wednesday EOD finish stories

Thursday Morning PR

Thursday EOD PR review / fix

Friday Create demo video

Friday EOD Retro

Process: code reviewer per story/PBI

• having a code reviewer assigned per user story / task?
• that would help with discipline and planning the effort
• how do you make the effort predictable with changing PR size?

content on scrum-of-scrums

This is for the new proposed outline in #53

Using the suggested template below as a guide, describe the concept of scrum of scrums . This should include (but not limited to)

• Inputs: What are team members expected to do before scrum of scrums (i.e. determine dependent work or work that may affect other teams)
• Participants
• Outputs: What decisions or actions are needed to conclude the meeting?

Suggested Template

Goals

What are the intended outcomes of the ceremonies?

Participation

What members of the team should participate?

Impact

What positive impact has been observed as a result of this practice?

Measures

How might one measure the impact of this ceremony positive or negative?

Facilitation Guidance

How might one go about running the ceremony? Is there a general pattern that the ceremony should follow?

Create a Code Review Recipe for Java

Guidance on how to CR the language and what tools can be used in the process to support the CR process

Guidance on how to review infrastructure as code changes

How to reviews changes to things like terraform scripts

Recipe: Create Cloud Native Example/Tutorial

Create a recipe/example/tutorial for adding telemetry to an application

WIP: ML integration, conceptual content + recipes

This is a WIP issue to start fleshing out some of the ML/Engineering combined content for the playbook.
 

• Secret management for ML (Python specific, introduction of environs library, how to load from jupyter).
• Testing ML (what and how to unit test for ML)
• ML Code reviews?!
• Stories for ML (use of hypothesis and detail on what will try in hypothesis investigation).
• Estimation for ML (keep hypothesis investigations timeboxed).
• Some breakdown of how to assign stories between an Eng team and an ML team?! Some discussion about collaboration practices perhaps?
• Recipes on MLFlow and AML (combination and only AML)

update content of questions for daily Stand-up Meetings

In the playbook, the Stand-ups are described as
"...The stand-up is a ceremony that is held each day of the sprint. In this ceremony, each contributor will answer three simple questions. This will repeat until each contributor has answered the three following questions.

• What did you work on yesterday?
• What are you working on today?
• Do you have any impediments/blockers? (defer discussion / resolution to “the parking lot”, described below)
  ..."
  The Scrum Guide 2017 (lastest version) states the questions around the sprint goal, what has the purpose of guaranteeing that the scrum team answers them with the correct focus of completing the sprint goal instead of answering based on any other activity they may also be involved in and that is not addressing the sprint goal. Otherwise they should consider to move to activities that address the goal to make sure the goal can be completed by the end of the sprint.

Below you can find the description from Scrum Guide 2017 for the daily scrum:

[Scrum guide 2017 - changed text for Daily Scrum]
"The structure of the meeting is set by the Development Team and can be conducted in different ways if it focuses on progress toward the Sprint Goal. Some Development Teams will use questions, some will be more discussion based. Here is an example of what might be used:

• What did I do yesterday that helped the Development Team meet the Sprint Goal?
• What will I do today to help the Development Team meet the Sprint Goal?
• Do I see any impediment that prevents me or the Development Team from meeting the Sprint Goal?"

Automated code review tools

Update language areas with tools to support the review process.

• FxCop
• StyleCop

and the like should be called out and specific instructions on how to enable and enforce them in a pipeline.

Invalid link on code reviews main readme

The link to Best Kept Secrets of Peer Code Review document on Code Reviews main README does not work. The link currently working is https://static1.smartbear.co/smartbear/media/pdfs/best-kept-secrets-of-peer-code-review_redirected.pdf

Golang: leaktesting

Recently I've been working on this library to provide simple locking in Azure between components written in Golang. https://github.com/lawrencegripper/goazurelocking/

One element that I found really useful, due to the use of goroutines for autorenewal, was using a leak tester to ensure all the routines are correctly closed. https://github.com/fortytw2/leaktest

Do you think this is worth adding to the docs for golang? Happy to submit a PR.