python/core-workflow

Issue tracker for CPython's workflow

core-workflow

Issue tracker and relevant tools for CPython's workflow.

Core workflow tools

Name	Description	Issue tracker	Owner/Maintainer
python/bedevere	Bot to help identify missing information for CPython pull requests	GitHub	Brett Cannon
python/blurb	blurb add on the command line	GitHub
python/blurb_it	blurb add on the web	GitHub	Mariatta
python/cherry-picker	Command line tool for backporting CPython pull requests	GitHub	Mariatta
python/miss-islington	Bot for backporting CPython pull requests	GitHub	Mariatta
ambv/cla-bot	CLA enforcement bot for Python organization projects		Łukasz Langa
berkerpeksag/cpython-emailer-webhook	Webhook to send every CPython commit to python-checkins mailing list	GitHub	Berker Peksag

core-workflow's Issues

Hook up Travis for PR/CI testing

See https://github.com/brettcannon/cpython-ci-test/blob/master/.travis.yml for the initial configuration.

Move the PEPs to Read the Docs

Various things need to happen (as outlined as issues in https://github.com/python/peps/issues), but getting the PEPs to build instantly would be good. Could even create peps.python.org.

Automatically merge with passing tests and a +1 Review

It would be great if a bot could automatically merge PRs that meet all of the following:

• All status checks are green
• A CLA check is good.
• Someone with write permission to the repository has given it an approved review with no reviews from someone with write permission requesting changes.

Right now if I review someone's PR either one of two things happen:

• The status checks are green and the CLA check is green, in which case I press the merge button.
  • This case is trivial though it does require an extra button click that could be eliminated.
• The status checks are pending or the CLA check hasn't happened yet.
  • This is the case I'm most interested in, what happens here is at some point in the future the status checks will go green but there is no notification that this has happened, so this relies on someone coming around again and notice that the merge button can be pushed.

cherry_picker.py opens wrong page

When I run cherry_picker.py it opens pages like https://github.com/python/cpython/compare/3.5...iy-storchaka:c85a266-3.5?expand=1 that compares changes between branches 3.5 and c85a266-3.5 in the python/cpython repository. Since branch c85a266-3.5 in the python/cpython repository doesn't exist there isn't anything to compare. However branch c85a266-3.5 is created in my fork of the python/cpython repository.

CLA form asks for "username", BPO field is "Login name"

Helping someone sign the CLA today, they were confused by the fact that the CLA form asks for "bugs.python.org username" but the actual field of interest on the user details page in Roundup is "Login name".

(It didn't help that the user details page in Roundup shows up under your numeric user ID rather than your login name)

Don't wait for allowed_failures on travis builds?

Currently we have a matrix item in Travis for allowed_failure. This is fine to do, but travis won't mark the build successful or failed until that job has finished even though it won't ultimately effect the outcome. One option we can do is to set:

matrix:
  fast_finish: true

Which will tell travis to report the status of the job as soon as all of the non allowed failure builds are done, while still running the allowed failure build.

Block directly pushing to the cpython repository

This won't happen until people are comfortable with GitHub, but once the setup is mostly automated and we are happy with the CI setup, core devs should be forced to go through the PR system and not push directly to the repo. This prevents accidents where something is overlooked. It also makes sure core devs are aware of what it's like for outside contributors so we can always work to make that process easier for everyone.

Require AppVeyor to pass

Not sure if we consider it stable enough, but it would be nice to have as a goal to make https://ci.appveyor.com/project/python/cpython be required just like we do for Travis.

@zware do you think this is ready to be flipped on? And is it fast enough?

Delete old branches?

We currently have useless branches for 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 3.0, 3.1, and 3.2. I suggest tagging the HEAD of each branch (x.x-EOL, perhaps?) and then deleting the branch. Having those branches makes the branch list unwieldy in the GitHub UI, and also (but less so) in command line usage. Keeping those branches might also make unfamiliar contributors think that they are valid targets for changes.

Add the mention bot?

One feedback people have been having so far is that the firehouse of notifications from Github on python/cpython is too much and they're having to unwatch the repository. That is a reasonable thing to do (I, for instance, do not watch the repository). However that can make it harder to get people to review PRs since they'll no longer be notified of news ones and they'll have to go out of their way to look for them.

One solution to this problem is https://github.com/facebook/mention-bot. It will attempt to intelligently notify people on new PRs (by CCing them) who it thinks is a good fit to review the code (effectively, which people were working around this area most recently). This bot also has a feature that might help with the module owner issue mentioned in #29, which it allows people to configure themselves to always get notified for certain PRs that touch certain files. For people who never want to be automatically notified, they can add themselves to the configuration blacklist and it will never select them as a potential reviewer. It has a number of other features (skipping if the person is a contributor, delaying the mention for X amount of time to let a PR get a review "naturally" before notifying people, etc) that attempt to reduce the amount of notifications it gives as well.

Move build status to README.rst?

The build status tags and links are currently in .github/CONTRIBUTING.rst, which is really only seen when you navigate to it. Should we move it to README.rst for higher visibility?

Auto-approve backport PRs

Is it possible and reasonable to auto-approve or self-approve backport PRs? Usually backport PRs need no change or need only trivial change. And now a bug fix may need a core dev to approve up to 4 times, for almost same contents. Worse is PRs are not grouped and you may need to search for the backport PRs.

Travis tests are broken

After all Python tests are passed at the end of the log:

$ echo '#include "Python.h"' > test.cc && $CXX -c test.cc -o /dev/null -I ./Include -I .

/home/travis/build.sh: line 62: -c: command not found

The command "echo '#include "Python.h"' > test.cc && $CXX -c test.cc -o /dev/null -I ./Include -I ." exited with 127.

Done. Your build exited with 1.

This blocks merging even simple PRs.

Add a setup command to cherry_picker

The idea is to simplify the setup by having cherry_picker do the checking out of one's fork and setting the origin. It's minor, but it makes getting started easier and it means that if any steps change in the future then people can be unaware as they just need to remember the setup command instead of assuming they know what to do and missing the new step(s).

When cherry_picker fails, drop you off in the checkout

I just had a cherry-pick fail, but because I was running cherry_picker from its own git repo it took me a minute to realize that I couldn't find the failed branch because I had not cd cpython yet. It might work out better to automatically cd into the checkout upon failure (with an appropriate message that this was done).

Move the devguide to Read the Docs

Either redirect to http://cpython-devguide.readthedocs.io/, or make http://cpython-devguide.readthedocs.io/ be dev.python.org and redirect there.

Require code coverage not drop in a PR

Codecov allows for PRs to be blocked from merging if the code coverage drops too much. As of right now I have seen coverage for things like pydoc vary too much to do that yet. We should probably work towards figuring out what is leading to that coverage variance and fix it so we can require coverage never gets worse.

If people do come across modules whose coverage seems to drop unexpectedly, please mention them here so we know what modules need to have their tests fixed.

Bot to automatically generate cherry-picking PRs

As pulled from PEP 512:

Since the decision has been made to work with cherry-picks instead of forward merging of branches, it would be convenient to have a bot that would generate pull requests based on cherry-picking for any pull requests that affect multiple branches. The most likely design is a bot that monitors merged pull requests with key labels applied that delineate what branches the pull request should be cherry-picked into. The bot would then generate cherry-pick pull requests for each label and remove the labels as the pull requests are created (this allows for easy detection when automatic cherry-picking failed).

Consider how to handle failed cherry-picks

From @ncoghlan on March 12, 2017 4:57

I think this utility has a lot of potential to evolve into the bot that actually does the automated cherry-pick PR creation, and one of the things we'll need to address there is what to do when the routine cherry pick fails.

At the moment, if the first cherry-pick fails, it leaves the checkout in a messy state, so all the subsequent branch creation commands fail.

I'm thinking one way of tackling this would be to:

Complain and exit early if the checkout is in a messy state, and suggesting running git stash to move the local changes aside.
If any of the cherry-picks fail do an immediate git reset --hard and move on to the next branch.
Does that approach sound reasonable?

Copied from original issue: Mariatta/chic_a_cherry_picker#15

Use webhooks to regenerate docs.python.org

That way the dev docs are always up-to-date.

cherry_picker.py requires Python 3.6

cherry_picker.py uses f-strings and requires Python 3.6. Is this absolutely necessary? The version of Python shipped with Ubuntu LTS (and perhaps other popular distributions) is 3.5.

Generate Misc/ACKS from author data

As pulled from PEP 512:

Traditionally the Misc/ACKS file [20] has been managed by hand. But thanks to Git supporting an author value as well as a committer value per commit, authorship of a commit can be part of the history of the code itself.

As such, manual management of Misc/ACKS will become optional. A script will be written that will collect all author and committer names and merge them into Misc/ACKS with all of the names listed prior to the move to Git. Running this script will become part of the release process.

The script should also generate a list of all people who contributed since the last execution. This will allow having a list of those who contributed to a specific release so they can be explicitly thanked.

More diffs in notification emails

If it's possible to support in GH, I would like to see diffs (unified) in PRs and commit emails. While it increases the size of the emails, I think it would be a huge productivity increase. Without the diffs, I have to stop and click on a link in my email, then go to my browser, just to see if the change is anything I can or want to comment on.

If the diffs were in my email notifications, I could do a quick review immediately. By scanning the email I could:

• Decide that I don't care about this change, delete it and move on
• Make a quick determination that the change looks good and do a quick +1 turnaround
• See some changes that I have questions or concerns about but defer my response to when I have sufficient time
• Have a quick comment about a change which I could send in a quick reply

I only want to go to the GH web ui when I really need to interact with the PR or commit, and that should be a minority of changes to Python.

Adjust coverage failure threshold

Codecov lets us specify how much of a coverage drop is acceptable. Since we are seeing some PRs fail by just adding an extra comment line, we should probably set a threshold of some sort to say a slight drop is acceptable. I'm thinking maybe 0.01% is okay?

And the configuration can be validated.

Add links back to bugs.python.org

When the bpo-\d+ pattern is picked up in a comment we should rewrite it to be linked to bugs.python.org appropriately. And if the link is detected in the title then a link should be added as a comment on the PR.

Provide labels to help track PRs that need to be cherry-picked

The question is what format to use for the labels. Feel free to vote for the format you prefer by leaving an appropriate reaction on the comment.

Add a --dry-run option to preview cherry-pick actions

When first learning to use the cherry-pick script, a --dry-run is handy for building confidence before actually running it to generate backport PRs.

Original pre-migration PR: Mariatta/chic_a_cherry_picker#13

Shut down Rietveld instance on bugs.python.org

Once all the in-progress reviews are finished, we can shut down Rietveld. We might want to archive the review comments and put them somewhere before we do the shutdown.

Drop the gcc build from Travis

We currently have 5 builds on Travis:

1. clang
2. gcc
3. C++ header compatibility
4. docs
5. code coverage

I'm not willing to drop 1, 4, or 5. Perhaps we should drop the gcc build as redundant next to clang and let any gcc buildbots acts as our fallback in case we hit a weird compatibility issue?

Automate the issue creation for PR that don't refer to any existing issue.

Brought up by @ezio-melotti in python/devguide#104 :)

Run coverage.py with coverage/fullcoverage/encodings.py

https://github.com/nedbat/coveragepy/blob/master/coverage/fullcoverage/encodings.py is a hack which makes sure that coverage.py is imported early enough to cover modules that are imported during interpreter startup. Previously it wasn't included with coverage.py, but @nedbat was nice enough to add fullcoverage to coverage.py in nedbat/coveragepy@3962bc4 for its next release. So this issue is to make sure that no one forgets to take advantage of fullcoverage once it's available on PyPI.

Review dismissal

It appears that a 'request changes' review blocks merge even if another core-dev approves. I'm not sure whether two approvals would override a 'request changes', but it is possible to dismiss a review; we should decide criteria for dismissing another core-dev's review here and document it in the devguide.

For example, python/cpython#27 was blocked by my outdated review, even though @methane had made the changes I had requested and @ncoghlan had approved the change. Frankly, I had forgotten that I had reviewed that PR, so I would have had no problem with either of the other involved core-devs dismissing my review.

Improve hglookup.py to directly map svn and hg commits to git

https://gist.github.com/brettcannon/f8d97c92b0df264cd4db008ffd32daf9

For the migration, hglookup.py just maps svn commits to svn.python.org and hg commits to hg.python.org. It would be nice to set it up so we have a mapping of svn and hg commits to git hashes and simply send everyone to python/cpython directly.

How to import my cherry_picker script to here while maintaining full commit history?

Since @ncoghlan has started contributing to it as well, I'd like to keep the history intact if possible :)

Check PRs for whitespace

See https://hg.python.org/hooks/file/tip/checkwhitespace.py for the hg.python.org solution. Need to check both code and docs. Could either set up a bot to do the check on the diff directly or somehow use Travis and its TRAVIS_PULL_REQUEST environment variable to check whitespace only on the changing code.

Consider using a prefix in the title of backport PRs

Can we use a prefix like [backport to X.Y] or [3.6] for backport PRs? For example:

[backport to X.Y] bpo-1234: Fix comment in tokenizer.c

This way we can easily see which PRs are backports on GitHub and we can also set appropriate email filters.

python/cpython doesn't have tags

There are no tags on https://github.com/python/cpython

Perhaps, autoclone script forgot to use push --tags option?

Generate Misc/NEWS from individual files

As pulled from PEP 512:

Traditionally the Misc/NEWS file [19] has been problematic for changes which spanned Python releases. Oftentimes there will be merge conflicts when committing a change between e.g., 3.5 and 3.6 only in the Misc/NEWS file. It's so common, in fact, that the example instructions in the devguide explicitly mention how to resolve conflicts in the Misc/NEWS file [21] . As part of our tool modernization, working with the Misc/NEWS file will be simplified.

The planned approach is to use an individual file per news entry, containing the text for the entry. In this scenario each feature release would have its own directory for news entries and a separate file would be created in that directory that was either named after the issue it closed or a timestamp value (which prevents collisions). Merges across branches would have no issue as the news entry file would still be uniquely named and in the directory of the latest version that contained the fix. A script would collect all news entry files no matter what directory they reside in and create an appropriate news file (the release directory can be ignored as the mere fact that the file exists is enough to represent that the entry belongs to the release). Classification can either be done by keyword in the new entry file itself or by using subdirectories representing each news entry classification in each release directory (or classification of news entries could be dropped since critical information is captured by the "What's New" documents which are organized). The benefit of this approach is that it keeps the changes with the code that was actually changed. It also ties the message to being part of the commit which introduced the change. For a commit made through the CLI, a script could be provided to help generate the file. In a bot-driven scenario, the merge bot could have a way to specify a specific news entry and create the file as part of its flattened commit (while most likely also supporting using the first line of the commit message if no specific news entry was specified). If a web-based workflow is used then a status check could be used to verify that a new entry file is in the pull request to act as a reminder that the file is missing. Code for this approach has been written previously for the Mercurial workflow at http://bugs.python.org/issue18967 . There is also tools from the community like https://pypi.python.org/pypi/towncrier , https://github.com/twisted/newsbuilder , and http://docs.openstack.org/developer/reno/ .

Discussions at the Sep 2016 Python core-dev sprints led to this decision compared to the rejected approaches outlined in the Rejected Ideas section of this PEP. The separate files approach seems to have the right balance of flexibility and potential tooling out of the various options while solving the motivating problem.

Identify potential new developers through authorship analysis

It should be possible to do a rough analysis of authorship information of PRs -- and the activity on those PRs -- to make it easier to figure out who to consider giving commit privileges to.

Link to developer guide in CPython repo description

python/cpython#118 adds a link to the Developer Guide to the top of the README.

It would be useful if the repo description linked to the developer guide in addition to linking to the main python.org website.

BPO notifications should ignore PR branches

If you use the online merge conflict resolution on a PR branch that was created directly in the online editor, GitHub will merge the target branch into the PR branch.

This currrently retriggers BPO notifications for all the commits merged into the PR branch.

Limiting notifications to master and X.Y branches should avoid the problem.

checking code style for pull request

In Github GUI, it's difficult to check 79 columns limitation.
How about adding pycodestyle --diff or something similar as non strict check?

Backup GitHub information

Better safe than sorry.

Link to GitHub PR from bpo emails

When someone creates a pull request that links to a bpo issue, there would be email sent to the issue subscribers / nosy list, for example:

Changes by Mariatta Wijaya <[email protected]>:


----------
pull_requests: +70

1. Does 70 here refer to GitHub pull request number 70? (eg python/cpython#70)
2. Instead of a plaintext +70, can it be a link to the said pull request?

Thanks :)

Add the branch number to cherry pick commits?

From @ncoghlan on March 12, 2017 4:46

I like having the '[X.Y]' prefixes in cherry pick commit messages, so I went exploring to figure out how we could add them automatically here.

As far as I can tell, the best available option seems to be a combination of git commit --amend -m and http://stackoverflow.com/questions/7293008/how-to-read-last-commit-comment which would involve doing:

git log -1 --pretty=%B

to get the full text of the current commit message, prepending the appropriate "[X.Y] " header, and then writing it back into the cherry-pick commit with git commit --amend -m.

Is that a change you might be interested in including if I put together a PR?

Copied from original issue: Mariatta/chic_a_cherry_picker#14

Bot to check for issues in PR titles

Can be a status check to make sure they are formatted correctly.

Merge commit messages

Should we have some guidance on what the commit message should be for our squash merges? The default is something like:

bpo-12345: PR Title (#PRNumber)

* bpo-12345: Commit message from first commit on the PR

* Commit message from second commit

* Commit message from a cleanup commit

Should we explicitly clean up the message, specifically by removing unnecessary cleanup commit messages and manually merging relevant parts of remaining commit messages?

Hook up Codecov for coverage reporting on C code

See https://github.com/brettcannon/cpython-ci-test/blob/master/codecov.yml for an initial configuration (only differs from the default thanks to the new config for comments).

Rationale for choice of pinned repositories

I just picked 6 pinned repositories for the Python GitHub organisation, this issue records my rationale for that, and may also serve as a venue for arguing for some different choices.

• cpython - chosen as reference interpreter
• peps - chosen as the standards setting mechanism for the ecosystem as a whole
• devguide - chosen as the "contributor manual" for the above two repositories
• pythondotorg - chosen as the main web presence for the community as a whole
• typeshed - chosen as a common point of collaboration for static analyser maintainers
• performance - chosen as a common point of collaboration for benchmarking interpreter performance

Only run CI checks when appropriate files have changed

E.g. don't run code coverage or the test suite if only the docs changed (or vice-versa).