Giter Site home page Giter Site logo

bors-ng / bors-ng Goto Github PK

View Code? Open in Web Editor NEW
1.5K 17.0 180.0 4.79 MB

๐Ÿ‘ A merge bot for GitHub Pull Requests

Home Page: https://bors.tech/

License: Apache License 2.0

JavaScript 2.04% Elixir 91.97% CSS 1.13% HTML 3.05% Shell 0.27% Dockerfile 0.36% Smarty 0.24% HCL 0.95% Procfile 0.01%
bors continuous-integration elixir github travis-ci appveyor

bors-ng's Introduction

Deprecation notice: Though I'll still merge bug fixes if they're provided, new features will not be accepted. If you want to implement a workflow like this, use GitHub's built-in merge queue. See TMIB 76 for more info.

A merge bot for GitHub pull requests

Bors-NG implements a continuous-testing workflow where the main branch never breaks. It integrates GitHub pull requests with a tool like GitHub Actions that runs your tests.

Other resources:

But don't GitHub's Protected Branches already do this?

Most CI systems, like Jenkins and GitHub Actions both run the test suite on every branch after it's pushed to and every pull request when it's opened, and GitHub can block the pull requests if the tests fail on them. To understand why this is insufficient to get an evergreen main branch, imagine this:

  • Pull Request #1: Rename bifurcate() to bifurcateCrab()

    Change the name of this function, as well as every call site that currently exists in the main branch. I've thought of making it a method on Crab instead of on Sword, but then it would be bifurcateWithSword(), which hardly seems like an improvement.

  • Pull Request #2: bifurcate() after landing, in addition to before

    Adds another call to bifurcate(), to make sure it gets done even if we skip the pre-landing procedure.

When both of these pull requests are sitting open in the backlog, they will both be tested with the main branch. Assuming they both pass, GitHub will happily present the Big Green Merge Button. Once they both get merged, the main branch will go red (Method bifurcate() not found).

In addition to the testing requirements, GitHub can also be set to block pull requests that are not "up to date" with the main branch, meaning that problems like this can't show up. This fixes the problem, by requiring that the main branch only contain a snapshot of the code that has passed the tests, but it requires maintainers to manually:

  1. "Update the pull requests," merging them onto main without changing main itself
  2. Wait for the test suite to finish
  3. Merge the pull request when it's done, which is a trivial operation that can't break the test suite thanks to step 1

And it has to be done for every pull request one at a time.

This is similar to, but less efficient than, the process that bors automates. Instead of merging, you add reviewed pull requests to a "merge queue" of pull requests that are tested against the main branch by copying it to a staging branch and merging into that. When the status of staging is determined (either pass or fail), bors reports the result back as a comment and merges staging into main if it was a pass. Then it goes on to the next one. Based on the assumption that the tests usually pass once they're r+-ed, bors actually tests them in batches (and bisects if a batch fails).

Note that bors is not a replacement for your CI system. It just implements this workflow.

How it works

Bors is a GitHub Application, so (assuming you already have GitHub Actions set up), getting bors set up requires three steps:

  1. Add the app to your repo in GitHub. Click here to use the publicly hosted instance.

  2. Commit a bors.toml with these contents:

    status = ["ci"]
    
  3. Set up a workflow step with the same name. For example:

    ci-success:
      name: ci
      if: ${{ success() }}
      needs:
        - exfmt
        - test
      runs-on: ubuntu-latest
      steps:
        - name: CI succeeded
          run: exit 0
    

To use it, you need to stop clicking the big green merge button, and instead leave a comment with this in it on any pull request that looks good to you:

bors r+

As commits are reviewed, bors lumps them into a queue of batches. If everything passes, there will just be two batches; the one that's running, and the one that's waiting to be run (and is accumulating more and more pull requests until it gets a chance to run).

To run a batch, bors creates a merge commit, merging the main branch with all the pull requests that make up the batch. Instead of pushing the merge commit immediately, however, it will instead push it to the staging branch. They'll look like this:

Merge #5 #7 #8

5: Rename `bifurcate()` to `bifurcateCrab()`
7: Call `bifurcate()` in the `onland` event handler
8: Fix crash in `drive()`

If the build passes, the main branch gets fast-forwarded to meet the staging branch. Since the main branch contains the exact contents that were just tested, bit-for-bit, it's not broken. (at least, not in any way that the automated tests are able to detect)

If the build fails, bors will follow a strategy called "bisecting". Namely, it splits the batch into two batches, and pushes those to the queue. In this example, the first batch will look like this:

Merge #5 #7

5: Rename `bifurcate()` to `bifurcateCrab()`
7: Call `bifurcate()` in the `onland` event handler

This batch will still fail, because the second patch inserts a call to a function that the first patch removes. It will get bisected again, as a result.

The second will still pass, though.

Merge #8

8: Fix crash in `drive()`

This one will work, causing it to land in the main branch, leaving the first two still in the backlog.

Merge #5

5: Rename `bifurcate()` to `bifurcateCrab()`

This one will pass, since the PR it conflicts with (#7) is sitting behind it in the queue.

Merge #7

7: Call `bifurcate()` in the `onland` event handler

When a batch cannot be bisected (because it only contains one PR), it gets kicked back to the creator so they can fix it.

Note that you can watch this process running on the dashboard page if you want.

As a convenience, you can also run bors try, which will kick off a build the same way r+ would, but without actually pushing it to the main branch even if it does succeed. To help keep them separate, r+ merge commits go in staging and try builds go in trying.

The original bors used a more simple system (it just tested one PR at a time all the time). The one-at-a-time strategy is O(N), where N is the total number of pull requests. The batching strategy is O(E log N), where N is again the total number of pull requests and E is the number of pull requests that fail.

How to run it on your local machine

If you're using a macOS or Linux command line with Docker on it, ./script/setup && ./script/server will set up a local instance, with a mocked-out GitHub instance, using Docker to pull in all the underlying dependencies. The web server ends up running on http://localhost:8000/. You can get an Elixir REPL running in the same context as the webserver by running repl instead of server. To run the tests, run test instead of server.

If you log in, it will log you in with the user "space." There won't be any repositories, and space will not have admin perms. You can use the User model to give space admin rights, and the WebhookController and GitHub ServerMock to create the repo.

Setting it up without Docker, like on Windows home edition

The main things you'll need to run Bors on your laptop are:

  • Familiarity with the command line
  • Elixir, with a full installation of OTP (the esl-erlang package is sufficient)
  • PostgreSQL; the configuration for it is in config/dev.exs
  • Stock C compilation tools, because some of bors's dependencies use NIFs
  • A git client, which you probably already have for downloading this repository
  • NodeJS, to perform asset compilation

I use Portable PostgreSQL, the Chocolatey packages for Elixir, Git, and NodeJS, and the Visual C++ build tools from Microsoft.

You can then run it using mix:

$ mix ecto.create
$ mix ecto.migrate
$ mix phx.server

If you want to use MySQL, add the -r flag:

$ mix ecto.create -r BorsNG.Database.RepoMysql

And it'll run with the GitHub API mocked-out.

To run tests, run:

$ mix test
$ mix dogma
$ mix dialyzer

How to set up your own real instance

Step 1: Register a new GitHub App

The first step is to register a new GitHub App on the GitHub web site.

App settings

The GitHub App name, description, and homepage URL are irrelevant, though I suggest pointing the homepage at the dashboard page.

The user authorization callback URL should be at <DASHBOARD URL>/auth/github/callback.

Leave the setup URL blank.

The webhook URL should be at <DASHBOARD URL>/webhook/github.

The webhook secret should be a randomly generated string. The mix phx.gen.secret command will work awesomely for this. Keep this handy to specify the same value in the bors configuration (you can also edit this value later if you need to).

Required GitHub App permissions

Permission summary

For each of these sections, set the following overall section permissions and check the following webhook event checkboxes. Explanations for why bors-ng needs each of these permissions are below.

  • Repository metadata: Read-only (no choice)
    • Repository (Repository created, deleted, publicized, or privatized)
  • Repository administration: No access
  • Commit statuses: Read & write
    • Status (Commit status updated from the API)
  • Deployments: No access
  • Issues: Read & write
    • Issue comment (Issue comment created, edited, or deleted)
  • Pages: No access
  • Pull requests: Read & write
    • Pull request (Pull request opened, closed, reopened, edited, assigned, unassigned, review requested, review request removed, labeled, unlabeled, or synchronized)
    • Pull request review (Pull request review submitted, edited, or dismissed)
    • Pull request review comment (Pull request diff comment created, edited, or deleted)
  • Repository contents: Read & write
    • (no checkboxes)
  • Single file: No access
  • Repository projects: No access
  • Organization members: Read-only
    • Team (Team is created, deleted, edited, added to/removed from a repository)
    • Member (Collaborator added to, removed from, or has changed permissions for a repository)
    • Membership (Team membership added or removed)
    • Organization ( User invited to, added to, or removed from an organization)
  • Organization projects: No access
  • Checks: Read & Write
    • Check run (Check run created from the API)
    • Check suite (Check suite created from the API)

Permission explanations

Repository metadata will be read-only. Must be set to receive Repository events to automatically remove entries from our database when a repo is deleted.

Commit statuses must be set to Read & write to report a testing status (this is the older version). Also must get Status events to integrate with CI systems that report their status via GitHub.

Issues must be set to Read & write because pull requests are issues. Issue comment events must be enabled to get the "bors r+" comments. If Issues is set to Read-only, repos will end up with pull requests that are marked as simultaneously merged and opened.

Pull requests must be set to Read & write to be able to post pull request comments. Also, must receive Pull request events to be able to keep the dashboard working, and must get Pull request review and Pull request review comment events to get those kinds of comments.

Repository contents must be set to Read & write to be able to create merge commits.

Checks must be set to Read & write to report a testing status (this is the newer version). Also must get Check run events to integrate with CI systems that report their status via GitHub.

Organization members must be set to Read only to synchronize repository contributors and bors reviewers.

After you click the "Create" button

GitHub will send a "ping" notification to your webhook endpoint. Since bors is not actually running yet, that will fail. This is expected.

You'll need the following values from your GitHub App for configuring bors-ng:

  • Private key (generate one and download the file)
  • OAuth credentials
  • ID (appears beneath the app logo and "Owned by" in the right hand column)

Internal app?

GitHub Apps can be set as "Internal" or "External". When the App is set to be internal, then whichever organization/user it belongs to will be the only one allowed to install it.

This setting can be chosen while first creating the app, or it can be changed afterward at one of these URLs (the switch is on the bottom of the page):

  • If the app is owned by an organization: https://github.com/organizations/<ORGANIZATION>/settings/apps/<APP NAME>/advanced
  • If the app is owned by a user: https://github.com/settings/apps/<APP NAME>/advanced

If an "External" app is installed on any external repositories, then the "Make Internal" button will be grayed out.

Step 2: Set up the server

bors-ng is written in the Elixir programming language, and it uses PostgreSQL as the backend database. Whatever machine you plan to run it on needs to have both of those installed.

bors-ng is built on the Phoenix web framework, and they have docs on how to deploy phoenix apps already. Where you deploy will determine what the dashboard URL will be, which is needed in the previous steps, so this decision needs to be made before you can set up the GitHub App.

You'll need to edit the configuration with a few bors-specific variables.

Deploying on Heroku (and other 12-factor-style systems)

The config file in the repository is already set up to pull the needed information from the environment, so you can configure bors by setting the right env variables and deploy the app from this repository into Heroku:

You can do using Heroku's one-button-deploy system:

Deploy on Heroku

Or you can do it manually:

Note: The GITHUB_INTEGRATION_ID is now called the App ID on GitHub.

$ heroku create --buildpack "https://github.com/HashNuke/heroku-buildpack-elixir.git" bors-app
$ heroku buildpacks:add https://github.com/gjaldon/heroku-buildpack-phoenix-static.git
$ heroku addons:create heroku-postgresql:hobby-dev
$ heroku config:set \
    MIX_ENV=prod \
    POOL_SIZE=18 \
    PUBLIC_HOST=bors-app.herokuapp.com \
    ALLOW_PRIVATE_REPOS=true \
    COMMAND_TRIGGER=bors \
    SECRET_KEY_BASE=<SECRET1> \
    GITHUB_CLIENT_ID=<OAUTH_CLIENT_ID> \
    GITHUB_CLIENT_SECRET=<OAUTH_CLIENT_SECRET> \
    GITHUB_INTEGRATION_ID=<ISS> \
    GITHUB_INTEGRATION_PEM=`base64 -w0 priv.pem` \
    GITHUB_WEBHOOK_SECRET=<SECRET2> \
    [BORS_LOG_LEVEL=<debug|info|warn|...>]
$ git push heroku master
$ heroku run POOL_SIZE=1 mix ecto.migrate

WARNING: bors-ng stores some short-term state inside the web dyno (it uses a sleeping process to implement delays, specifically). It can recover the information after restarting, but it will not work correctly with Heroku's replication system. If you need more throughput than one dyno can provide, you should deploy using a system that allows Erlang clustering to work.

Deploying using Docker (and compatible container orchestration systems)

Pre-built Docker images are available at Docker Hub for the current master (as bors-ng:latest).

The Dockerfile in the project root can be used to build the image yourself. It relies on multi-stage builds as introduced in Docker 17.05, to generate a slim image without the Erlang, Elixir and NodeJS development tools.

Most of the important configuration options should be set at runtime using environment variables, not unlike the Heroku instructions. All the same recommendations apply, with some extra notes:

  • ELIXIR_VERSION can be set as a build-time argument. Its default value is defined in the Dockerfile.

  • ALLOW_PRIVATE_REPOS must be set at both build and run times to take effect. It is set to true by default.

  • DATABASE_URL must contain the database port, as it will be used at container startup to wait until the database is reachable. The format is documented here. For using MySQL in the docker image, use a mysql scheme url: -e DATABASE_URL="mysql://root:<secret>@db:3306/bors_ng" in conjunction with BORS_DATABASE=mysql

  • DATABASE_TIMEOUT may be set higher than the default of 15_000(ms). This may be necessary with repositories with a very large amount of members.

  • DATABASE_PREPARE_MODE can be set to to unnamed to disable prepared statements, which is necessary when using a transaction/statement pooler, like pgbouncer. It is set to named by default.

  • BORS_DATABASE can be set to mysql to switch the Docker container to MySQL

  • The database schema will be automatically created and migrated at container startup, unless the DATABASE_AUTO_MIGRATE env. var. is set to false. Make that change if the database state is managed externally, or if you are using a database that cannot safely handle concurrent schema changes (such as older MariaDB/MySQL versions).

  • Database migrations can be manually applied from a container using the migrate release command. Example: docker run borsng/bors-ng:latest /app/bors/bin/bors migrate. Unfortunately other mix tasks are not available, as they cannot be run from compiled releases.

  • The PORT environment variable is set to 4000 by default.

  • GITHUB_URL_ROOT_API and GITHUB_URL_ROOT_HTML should allow you to connect bors-ng to an instance of GitHub Enterprise. Note: I've never actually used GitHub Enterprise, so I'm kinda guessing about what you'd need here.

  • BORS_LOG_LEVEL allows you to set the log level at runtime for bors-ng. The allowed values are the usual Elixir Logger levels, e.g. info, debug, warn, etc. Defaults to info if not set.

    docker create --name bors --restart=unless-stopped \
        -e PUBLIC_HOST=app.bors.tech \
        -e SECRET_KEY_BASE=<secret> \
        -e GITHUB_CLIENT_ID=<secret> \
        -e GITHUB_CLIENT_SECRET=<secret> \
        -e GITHUB_INTEGRATION_ID=<secret> \
        -e GITHUB_INTEGRATION_PEM=<secret> \
        -e GITHUB_WEBHOOK_SECRET=<secret> \
        -e DATABASE_URL="postgresql://postgres:<secret>@db:5432/bors_ng" \
        -e DATABASE_USE_SSL=false \
        -e DATABASE_AUTO_MIGRATE=true \
        -e COMMAND_TRIGGER=bors \
        [-e BORS_LOG_LEVEL=<debug|info|warn|...>] \
        borsng/bors-ng
    docker start bors
    

Deploying on your own cluster

Your configuration can be done by modifying config/prod.secret.exs.

Optional step 3: make yourself an admin

bors-ng offers a number of special functions for "administrator" users, including diagnostics and the ability to open a repo dashboard without being a reviewer.

However, there's no UI for adding admins; you'll have to go into Postgres yourself to do it. There's two ways to do that:

You can do it from the iex prompt, like this:

shell$ iex -S mix # or `heroku run bash -c "POOL_SIZE=1 iex -S mix"`
iex> me = BorsNG.Database.Repo.get_by! BorsNG.Database.User, login: "<your login>"
iex> BorsNG.Database.Repo.update! BorsNG.Database.User.changeset(me, %{is_admin: true})

You can do it from a PostgreSQL prompt like this:

postgres=# \c bors_dev -- or bors_prod
bors_dev=# update users set is_admin = true where login = '<your login>';

Copyright license

bors-ng is licensed under the Apache license, version 2.0. It should be included with the source distribution in LICENSE-APACHE. If it is missing, it is at http://www.apache.org/licenses/LICENSE-2.0.

bors-ng's People

Contributors

ansonlc avatar bors-ng[bot] avatar bors[bot] avatar carlosjgp avatar champo avatar couchand avatar danielkza avatar dependabot-preview[bot] avatar dependabot-support avatar dependabot[bot] avatar dieend avatar dilumaluthge avatar diraol avatar francoisfreitag avatar gabriella439 avatar gmendonca avatar grahamc avatar indirect avatar jsoref avatar khodzha avatar kimsaehun avatar macarse avatar marksrobinson avatar noizwaves avatar notriddle avatar pksunkara avatar swist avatar tommilligan avatar umamaistempo avatar zli-simspace avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bors-ng's Issues

Implement a complete mock GitHub server

The test suite is not very comprehensive, largely because its handling of GitHub is just to have a "mock GitHub instance" that returns constants. This is (mostly) fixable, by having a GitHub mock that runs in a GenServer, so the test suite can program its state and then verify that the UUT leaves it in the correct state when it's done.

This issue is just to create the mock GitHub server. Migrating and creating tests that use it is a separate issue.

The whole "alert box" thing needs redone

Right now, the HTML code to do alert boxes is duplicated throughout several templates. It differs for some of them, because some alert boxes are shown by JavaScript (the "this list is out of date" alerts on the project show.iex pages), and some of them are shown by Elixir (anything involving put_flash is done this way).

The code for the put_flash triggered ones should be separated into its own IEX file. It should also be wired up to a JavaScript button so that the user can close them. The JavaScript-triggered ones may be able to reuse the same one that flash uses, and it can definitely be wired up so the user can dismiss it.

Add a way to block a PR if it has a certain label

There should be a new, optional, item in "bors.toml" for blocked label(s).

When a PR gets reviewed, the webhook controller sends a message to the batcher process. Inside that process's message handler, it should pull down the bors.toml file inside that pull request's branch (not the one in master, because: the one in master might not exist yet, we'd like to make it easy for the user to test out the label blocking feature, and we want to be consistent with the way it's already working). If the bors.toml is syntactically invalid or nonexistent, the pull request should be rejected. The pull request should also be rejected if any of the block labels are present (if the block labels key is missing, it's the same as if it was an empty list).

bors-ng itself will probably add this to its bors.toml once this feature is implemented:

block-labels = ["S-do-not-merge-yet"]

Add a "staging.tmp" error message for CircleCI

"staging.tmp" should not run in CI. If bors detects that it is running, it produces a warning with some information about how to fix it.

bors currently only does this for Travis CI and AppVeyor (added with PR #43). It should do it for CircleCI, too.

Required statuses on the direct Pull Request commit

Not all interesting "required status checks" work like a typical CI test runner. Some status checks are checking the format of the pull request itself, so they don't even make sense in the context of a stand-alone commit. It might just be something like the LGTM tool, which is based on the comments.

A separate section in the bors.toml should specify the pr-status = ["LGTM", "nubis/accepted"] type of format. The implementation will probably be similar to #24.

Bors just signals 'cancelled' when it can't merge

I had some additional options on and bors just sets its status to 'cancelled' leaving me confused for a moment (I luckily remembered that I had those options turned on), maybe it could write a comment and signal why or maybe just that it couldn't merge because of permission issues?

"Sync pull requests" race condition

The synchronization happens asynchronously in the background, and it notifies the user that it's done using Project.ping!. This would mostly work, except that if the task finishes before the page is done loading, it never learns that it's done.

Push commit statuses to GitHub

The tool should push a commit status for each Patch commit (so that it shows up in the Pull Request page), and it should push the status to the Batch commit (so that the user can turn on Status-Protected Branch mode in GitHub).

To make this work right, bors should have a page for each batch showing links to each of the underlying CI pages (as provided by their statuses), and they should be protected using the Integration callback mechanism (so that private repos aren't exposed to the whole world).

Crash in webhook after marking a PR as approved review

** (exit) an exception was raised:
** (ArgumentError) nil given for :pr_xref. Comparison with nil is forbidden as it is unsafe. Instead write a query with is_nil/1, for example: is_nil(s.pr_xref)
   (ecto) lib/ecto/query/builder/filter.ex:107: Ecto.Query.Builder.Filter.runtime!/6
   (ecto) lib/ecto/query/builder/filter.ex:98: Ecto.Query.Builder.Filter.runtime!/2
   (ecto) lib/ecto/repo/queryable.ex:304: Ecto.Repo.Queryable.query_for_get_by/3
   (ecto) lib/ecto/repo/queryable.ex:56: Ecto.Repo.Queryable.get_by!/5
   (aelita2) web/controllers/webhook_controller.ex:122: Aelita2.WebhookController.do_webhook/3
   (aelita2) web/controllers/webhook_controller.ex:23: Aelita2.WebhookController.webhook/2
   (aelita2) web/controllers/webhook_controller.ex:1: Aelita2.WebhookController.action/2
   (aelita2) web/controllers/webhook_controller.ex:1: Aelita2.WebhookController.phoenix_controller_pipeline/2

Note: It doesn't seem like this crash has actually affected anything.
The server has stayed up and everything is working as expected.

(Partially) ignore pull requests not made against master

Pull requests that are not made with the master branch (which, of course, may not literally be called "master") should not respond to "bors r+". They also shouldn't be shown in the review dashboard.

They can't be totally ignored, because GitHub allows a pull request to change header, so pull requests that started out being targetted at master but got changed should vanish, and pull requests that become targetted against master should appear.

The "this list is out-of-date" alert should only show up once

Steps to reproduce

Opens a "Pull requests" page in the bors dashboard, do not close it, and do not reload it. Open a pull request on that repo, then r+ it.

Expected behavior

One alert saying "this list is out-of-date" shows up.

Actual behavior

Every time an event like that happens, an event pops up. This results in there being multiple, identical copies of the same alert box.

Settings page should give suggestions for bors.toml

The settings page should pull the bors.toml from the master branch, and render it into a user-friendly GUI to edit it. My idea for the flow is something like this:

  • User opens the Settings tab in the bors dashboard. There is a section for reviewers, like we have now, and a section for build settings, which contains GUI representation of bors.toml.
  • The user edits the build settings from the GUI.
  • When the user attempts to apply the build settings, it creates a branch off of master with a now-modified bors.toml file, and opens a pull request in GitHub.
  • The user can now attempt to merge the bors.toml pull request into master.

Add a "site admin" flag

Users that are marked as "system administrators" should be able to see all repos, a list of all users, and, potentially, other relevant information like consistency checks and the ability to ban users or repos.

A "this list is out-of-date" page should exist for the dashboard home page

The dashboard home page, once you're logged in, allows you to see patches that are awaiting review. This is intended to be the main page that developers look at, since it shows the information they're most likely to want. However, it doesn't have the nice "stuff happened, you should refresh" thing that the project pages themselves have. This should be implemented.

It'll require going from "the project that just changed" to "the users who are interested." To avoid spurious notifications, it will also need to trigger only when a new patches is added or an existing patch is reviewed; it should not trigger when a patch goes from "in the backlog" to "running" to "merged," because that information is not shown on the front page.

Crash in batcher

[error] GenServer Aelita2.Batcher terminating
** (BadMapError) expected a map, got: [{"continuous-integration/codeship", 2}]
   (stdlib) :maps.is_key("continuous-integration/codeship", [{"continuous-integration/codeship", 2}])
   (aelita2) lib/aelita2/batcher.ex:216: anonymous fn/2 in Aelita2.Batcher.poll_running_batch/1
   (elixir) lib/enum.ex:814: anonymous fn/3 in Enum.filter/2
   (elixir) lib/enum.ex:1755: Enum."-reduce/3-lists^foldl/2-0-"/3
   (elixir) lib/enum.ex:814: Enum.filter/2
   (aelita2) lib/aelita2/batcher.ex:216: Aelita2.Batcher.poll_running_batch/1
   (elixir) lib/enum.ex:645: Enum."-each/2-lists^foreach/1-0-"/2
   (elixir) lib/enum.ex:645: Enum.each/2

After bors got its first PR which was ready to be merged (all statuses green), it started crashing with this message.
The bors.toml for this project looks like:

status = [
          "continuous-integration/codeship",
          "nubis/accepted"]

I can't get bors up again, it keeps crashing with this message on startup.

Two PRs on 'Running' (Retrying)

The currently 'Running' batch has two PRs in it, both of which have failed before.
Bors made a comment:

Build failed (retrying...)

continuous-integration/codeship

But nothing else is happening.
The PRs have since been fixed and are green, and I have commented bors r+ again.

What should trigger the retry?

bors has made no new commits to staging or staging.tmp and I see nothing in the logs related to this.

Add a way to cancel a pull request

This should probably be triggered by bors r-, and it should be automatically triggered if new commit(s) are pushed to a pull request after it has been queued up.

If a pull request is canceled while it is running, it should be cut out of the batch it is running in, and the batch should be restarted.

Merging a PR does not close it, or its associated issues

This is a bug in GitHub; it shouldn't even be possible to have a PR that's merged but not closed.

  • They've already been notified about it.
  • They've successfully reproduced it.
  • They're going to fix it at some point; they just haven't yet.

The Access calls for keywords expect the key to be an atom

Got a crash in Access:

** (ArgumentError) the Access calls for keywords expect the key to be an atom, got: 0
    (elixir) lib/access.ex:255: Access.fetch/2
    (elixir) lib/access.ex:269: Access.get/3
    (aelita2) lib/aelita2/batcher.ex:268: Aelita2.Batcher.bisect/1
    (aelita2) lib/aelita2/batcher.ex:256: Aelita2.Batcher.maybe_complete_batch/3
    (aelita2) lib/aelita2/batcher.ex:227: Aelita2.Batcher.maybe_complete_batch/1
    (elixir) lib/enum.ex:645: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:645: Enum.each/2
    (ecto) lib/ecto/adapters/sql.ex:508: anonymous fn/3 in Ecto.Adapters.SQL.do_transaction/3

Do you need any more data?
This was after three staging.tmp builds failed.

Include pull request body in the merge commit

The pull request body should go into the merge commit message. Currently, it looks like this:

Merge #XXX #YYY #ZZZ

XXX: title
YYY: title
ZZZ: title

It should look like this, instead

Merge #XXX #YYY #ZZZ

#XXX: title
body

#YYY: title
body

#ZZZ: title
body

Besides the advantage of keeping that information around, it also allows fixes the problem that "Fixes #III" in the pull request body doesn't work otherwise.

Error when adding additional reviewer

When adding an additional reviewer through the web interface I get a Internal server error.

This is the log from Heroku:

21:10:13.351 [error] #PID<0.520.0> running Aelita2.Endpoint terminated
Server: not-a-robot.herokuapp.com:80 (http)
Request: POST /repositories/8/reviewer
        (aelita2) web/controllers/project_controller.ex:1: Aelita2.ProjectController.action/2
** (exit) an exception was raised:
    ** (UndefinedFunctionError) function :ok.id/1 is undefined (module :ok is not available)
        :ok.id({:ok, %Aelita2.GitHub.User{avatar_url: "https://avatars.githubusercontent.com/u/123456?v=3", id: 123456, login: "someuser"}})
        (aelita2) web/controllers/project_controller.ex:128: Aelita2.ProjectController.add_reviewer/3
        (aelita2) web/controllers/project_controller.ex:1: Aelita2.ProjectController.phoenix_controller_pipeline/2
        (aelita2) lib/aelita2/endpoint.ex:1: Aelita2.Endpoint.instrument/4
        (aelita2) lib/phoenix/router.ex:261: Aelita2.Router.dispatch/2
        (aelita2) web/router.ex:1: Aelita2.Router.do_call/2
        (aelita2) lib/aelita2/endpoint.ex:1: Aelita2.Endpoint.phoenix_pipeline/1

Synchronize pull requests

Code needs to be implemented that fetches a list of open pull request from GitHub. Using this list, the code should do this:

  1. For every open pull request that GitHub returns, if there is no corresponding patch in bors's database, create the patch.
  2. For every patch that was not processed as part of step 1, mark it as closed.

This code should run when a repository is added, so that people can immediately start reviewing pull requests that existed before bors was added. In this case, step 2 shouldn't do anything.

A button to do this should also be added to the repository dashboard. This is a workaround in case a webhook is dropped. Unlike homu, this button should not also poll the batches, because we already do that on an automatic schedule.

The synchronization should ideally be done in a background process, because it's going to take awhile, and the UI can be kept up-to-date with a Phoenix channel. The initial implementation doesn't need to do this, but it should be structured in a way that doing so is feasible.

Note on force pushing

Add a note to the docs that bors uses force pushes when pushing to github.

I (and probably other people) have the master setup to disallow force pushing so we'll need to make an exception for bors.
While we are on the subject - why does bors need to force push?
That makes me a little uncomfortable.

Batcher sharding

Right now, there is a single Batcher process that loops through all the active batches to take care of activating the pending batches and polling the running ones.

Each project does need to have a single responsible process to avoid over-polling (or we need something like this "distributed cron" paper describes). So let's have a separate process for each project, and let's shard them across Erlang nodes.

Instance stuck on running after crash

After the previous crash, one PR is stuck on 'running' even though it has already successfully completed.
Is there a way to trigger bors-ng to look at this again?

Add the ability to try a PR without merging it

This should not go through the Batcher at all. There should be a separate table of trials that are running, because trials do not benefit from being batched or queued (since they're not merging, they can be run with maximum parallelization. It should be triggered by bors try. Any text after the word try should be copied into the commit message, so that scripts inside the repository can mine this for information on what should be tested; for example, once this is implemented, bors-ng's own Travis CI configuration will do this:

if echo "$TRAVIS_COMMIT_MESSAGE" | grep 'bors try: ' 2>&1 >/dev/null; do
    test_args=`echo "$TRAVIS_COMMIT_MESSAGE" | sed 's@^bors try: @@'`
else
    test_args=''
fi
mix test $test_args

Detect if staging.tmp is being built

staging.tmp is just intermediate work to build an octopus merge, and should not be built. If GitHub sends us status notifications for these temporary commits, we should push a message to the responsible pull request, so that the user knows about it.

We can also suggest specific fixes, if it's a known CI system. https://github.com/notriddle/test_repo shows an example with both Travis CI and AppVeyor.

Crash after reviewing

Request: POST /webhook/github
** (exit) an exception was raised:
** (CaseClauseError) no case clause matching: {{0, 7}, :nomatch}
   (aelita2) web/controllers/webhook_controller.ex:215: Aelita2.WebhookController.do_webhook_comment/2
   (aelita2) web/controllers/webhook_controller.ex:23: Aelita2.WebhookController.webhook/2
   (aelita2) web/controllers/webhook_controller.ex:1: Aelita2.WebhookController.action/2
   (aelita2) web/controllers/webhook_controller.ex:1: Aelita2.WebhookController.phoenix_controller_pipeline/2
   (aelita2) lib/aelita2/endpoint.ex:1: Aelita2.Endpoint.instrument/4
   (aelita2) lib/phoenix/router.ex:261: Aelita2.Router.dispatch/2
   (aelita2) web/router.ex:1: Aelita2.Router.do_call/2
   (aelita2) lib/aelita2/endpoint.ex:1: Aelita2.Endpoint.phoenix_pipeline/1

Tree closure

As long as the tree is closed, no new batches can start. Add a button in the dashboard page to close the tree. It changes to an open tree button when the tree is already closed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.