smarkets/marge-bot

A merge-bot for GitLab

marge-bot's Issues

Errors parsing version of current gitlab versions

Got an an error when parsing the version of newer gitlab versions. Looks like the "edition" part of the version is missing but expected.

24.7.2017 23:49:212017-07-24 21:49:21,800 DEBUG RESPONSE CODE: 200
24.7.2017 23:49:212017-07-24 21:49:21,800 DEBUG RESPONSE BODY: {'version': '9.4.0', 'revision': '9bbe2ac'}
24.7.2017 23:49:212017-07-24 21:49:21,800 ERROR Unexpected Exception
24.7.2017 23:49:21Traceback (most recent call last):
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/job.py", line 56, in execute
24.7.2017 23:49:21    approvals = merge_request.fetch_approvals()
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/merge_request.py", line 121, in fetch_approvals
24.7.2017 23:49:21    approvals.refetch_info()
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/approvals.py", line 10, in refetch_info
24.7.2017 23:49:21    if self._api.version().release >= (9, 2, 2):
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/gitlab.py", line 72, in version
24.7.2017 23:49:21    return Version.parse(response['version'])
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/gitlab.py", line 207, in parse
24.7.2017 23:49:21    release_string, edition = string.split('-', maxsplit=1)
24.7.2017 23:49:21ValueError: not enough values to unpack (expected 2, got 1)
24.7.2017 23:49:212017-07-24 21:49:21,800 DEBUG REQUEST: GET http://gitlab/api/v4/version {'PRIVATE-TOKEN': 'HW9c_BJgwgoK5cEmwsSr'} {'params': {}}
24.7.2017 23:49:212017-07-24 21:49:21,802 DEBUG Starting new HTTP connection (1): gitlab
24.7.2017 23:49:212017-07-24 21:49:21,810 DEBUG http://gitlab:80 "GET /api/v4/version HTTP/1.1" 200 None
24.7.2017 23:49:212017-07-24 21:49:21,811 DEBUG RESPONSE CODE: 200
24.7.2017 23:49:212017-07-24 21:49:21,811 DEBUG RESPONSE BODY: {'version': '9.4.0', 'revision': '9bbe2ac'}
24.7.2017 23:49:21Traceback (most recent call last):
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/job.py", line 56, in execute
24.7.2017 23:49:21    approvals = merge_request.fetch_approvals()
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/merge_request.py", line 121, in fetch_approvals
24.7.2017 23:49:21    approvals.refetch_info()
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/approvals.py", line 10, in refetch_info
24.7.2017 23:49:21    if self._api.version().release >= (9, 2, 2):
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/gitlab.py", line 72, in version
24.7.2017 23:49:21    return Version.parse(response['version'])
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/gitlab.py", line 207, in parse
24.7.2017 23:49:21    release_string, edition = string.split('-', maxsplit=1)
24.7.2017 23:49:21ValueError: not enough values to unpack (expected 2, got 1)
24.7.2017 23:49:21
24.7.2017 23:49:21During handling of the above exception, another exception occurred:
24.7.2017 23:49:21
24.7.2017 23:49:21Traceback (most recent call last):
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/bin/.marge.app-wrapped", line 4, in <module>
24.7.2017 23:49:21    marge.app.main()
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/app.py", line 114, in main
24.7.2017 23:49:21    marge_bot.start()
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/bot.py", line 50, in start
24.7.2017 23:49:21    self._run(repo_manager)
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/bot.py", line 92, in _run
24.7.2017 23:49:21    job.execute()
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/job.py", line 70, in execute
24.7.2017 23:49:21    merge_request.comment("I'm broken on the inside, please somebody fix me... :cry:")
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/merge_request.py", line 90, in comment
24.7.2017 23:49:21    if self._api.version().release >= (9, 2, 2):
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/gitlab.py", line 72, in version
24.7.2017 23:49:21    return Version.parse(response['version'])
24.7.2017 23:49:21  File "/nix/store/0yjvdhpw9cpmahw6ndfgby9n4pz8m79i-python3.6-marge-0.1.1/lib/python3.6/site-packages/marge/gitlab.py", line 207, in parse
24.7.2017 23:49:21    release_string, edition = string.split('-', maxsplit=1)
24.7.2017 23:49:21ValueError: not enough values to unpack (expected 2, got 1)

Versions
GitLab: 9.4.0
GitLab API: v4
marge-bot: 0.1.1

No `impersonate-approvers` on gitlab.com

Using impersonate-approvers requires admin permission... How do I grant admin permission? I reckon this is something that belongs to the GitLab installation, and therefore is not something mere mortals can do on gitlab.com, even in our own repositories. If it is so, it would be good to say mention it in the documentation, I didn't see that.

add --gitlab-url env equivalent

seems from documentation that --gitlab-url='http://your.gitlab.instance.com' can be specified only via commandline.

please add MARGE_GITLAB_URL env var support, so i could use this easily via docker-compose.yml

probably should unify configuration that all commandline options are working via env vars as well (and vice versa). maybe click can help in this.

Fix/investigate corner case of no-op merge request

This can happen when the commit messages of source and target differ, but content is identical; there is an example in our internal marge-test-repo. The weird thing we observed in that case is that rebasing into the target branch did not update the branch reference (i.e. unexpected git behavior which we should try to understand; possibly a bug more likely an UX issue or us missing something).

Marge should ensure reviewer and commit author are not the same.

For each rewritten commit, verify that author and reviewer are different people.

Handle GitLab setting merge request as WIP after pushing

Because of this GitLab feature, we can have the following interaction:

1. Someone creates a merge request containing a commit with WIP in its name. GitLab doesn't consider the Merge Request as WIP at this point.
2. Merge request is assigned to Marge; it checks that the merge requests is not WIP and proceeds.
   1 The branch is rebased and pushed again.
3. Because we have just pushed a commit that contains WIP (even if it was in one of the original commits), GitLab marks the merge request as WIP (it even reports "Marge Bot marked as Work In Progress").
4. After CI passes, she tries to merge, but GitLab now refuses to merge and we fail with "had some issue with gitlab"

We should test for WIP status before merging or, perhaps better, after the merge failed and we don't know why.

[Request] Cancel build if rebase is available

Add an option to automatically cancel the build and rebase again if a commit was added while the build is running

"Too Many Requests"

I got this log:

2018-08-31 16:23:55,992 INFO Waiting for CI to pass for MR !12
2018-08-31 18:11:34,929 ERROR Unexpected Exception
Traceback (most recent call last):
  File "/home/molcas-test/marge-bot/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/home/molcas-test/marge-bot/marge/single_merge_job.py", line 73, in update_merge_request_and_accept
    self.wait_for_ci_to_pass(merge_request, actual_sha)
  File "/home/molcas-test/marge-bot/marge/job.py", line 191, in wait_for_ci_to_pass
    ci_status = self.get_mr_ci_status(merge_request, commit_sha=commit_sha)
  File "/home/molcas-test/marge-bot/marge/job.py", line 129, in get_mr_ci_status
    pipelines = Pipeline.pipelines_by_branch(pid, ref, self._api)
  File "/home/molcas-test/marge-bot/marge/pipeline.py", line 29, in pipelines_by_branch
    params,
  File "/home/molcas-test/marge-bot/marge/gitlab.py", line 58, in call
    raise error(response.status_code, err_message)
marge.gitlab.UnexpectedError: (429, 'Too Many Requests')

as you can see, after waiting a couple of hours for the CI to finish (this is not a problem, the pipeline was running fine, it just takes a long time), marge got 429, 'Too Many Requests' from gitlab.com and crashed.

A better protection against this kind of error would be welcome, or maybe it's just a matter of increasing waiting_time_in_secs.

Space suggestion for marge bot username

Your suggestion to include space in bot name, breaks GitLab Jira integration.

[ Marge Bot|https://gitlab.example.net/marge-bot] mentioned this issue in a commit of libraries/imagetools:
'Merge branch 'imagetools-fixes' into 'master''

not sure what to do with this, remove the suggestion, report to gitlab-ce about the problem, or ...

Trigger pipelines for merge requests

Since GitLab does not support pipeline for merge requests yet (see https://gitlab.com/gitlab-org/gitlab-ce/issues/23902), can marge-bot take care of it?

I'd like her to:

1. Check if a pipeline has been created on the MR (or the branch it comes from)
2. Check if it is the correct pipeline (if it includes the right jobs, at least)
3. If either is no, trigger a pipeline for the MR and wait for completion

The reason is that for expensive tests one may not want to run them for every push to every branch, but they must be run on a MR if it's going to go into master. The CI configuration could be set up to run these tests only for, say, triggers and web, but can't enforce them to run on MRs. A possibility is running the test for some specially-named branches by setting a branch regexp for the CI, and require MRs to use these names, but that needs some discipline which is hard to find (although marge-bot could help here too).

Handle autosquash

Ticking autosquash seems to break marge at moment.

docker image date stamped 1970

so weird:

# docker images|grep marge-bot
smarketshq/marge-bot                                0.5.0                          cbc2be3ebd0c        47 years ago        504MB
smarketshq/marge-bot                                latest                         22433019da23        47 years ago        276MB

they seem ok in dockerhub:
https://hub.docker.com/r/smarketshq/marge-bot/tags/

Missing support for gitlab-CE

Looks like marge-bot is only compatible with gitlab-EE. It uses the approval feature that is not available in gitlab-CE. The endpoint /api/v4/projects/{id}/merge_requests/{id}/approvals is also not part of the Community Editions API.

25.7.2017 10:23:142017-07-25 08:23:14,400 DEBUG http://gitlab:80 "GET /api/v4/projects/1/merge_requests/1/approvals HTTP/1.1" 404 None
25.7.2017 10:23:142017-07-25 08:23:14,401 DEBUG RESPONSE CODE: 404
25.7.2017 10:23:142017-07-25 08:23:14,401 DEBUG RESPONSE BODY: {'error': '404 Not Found'}
25.7.2017 10:23:142017-07-25 08:23:14,402 ERROR Unexpected Exception
25.7.2017 10:23:14Traceback (most recent call last):
25.7.2017 10:23:14  File "/nix/store/02cbi06m7hk3qccalxrc71c2abiax6nz-python3.6-marge-0.1.2/lib/python3.6/site-packages/marge/job.py", line 56, in execute
25.7.2017 10:23:14    approvals = merge_request.fetch_approvals()
25.7.2017 10:23:14  File "/nix/store/02cbi06m7hk3qccalxrc71c2abiax6nz-python3.6-marge-0.1.2/lib/python3.6/site-packages/marge/merge_request.py", line 121, in fetch_approvals
25.7.2017 10:23:14    approvals.refetch_info()
25.7.2017 10:23:14  File "/nix/store/02cbi06m7hk3qccalxrc71c2abiax6nz-python3.6-marge-0.1.2/lib/python3.6/site-packages/marge/approvals.py", line 15, in refetch_info
25.7.2017 10:23:14    self._info = self._api.call(GET(approver_url))
25.7.2017 10:23:14  File "/nix/store/02cbi06m7hk3qccalxrc71c2abiax6nz-python3.6-marge-0.1.2/lib/python3.6/site-packages/marge/gitlab.py", line 55, in call
25.7.2017 10:23:14    raise error(response.status_code, err_message)
25.7.2017 10:23:14marge.gitlab.NotFound: (404, {'error': '404 Not Found'})

Supporting both versions with suitable workflows would be awesome!

Enabling automerge on gitlab side during marge processing causes exceptional behavior

Repro:

1. Assign marge
2. During marges processing enable automerge
3. Sometime later marge will error out: "I'm broken on the inside, please somebody fix me."

Everything worked but someone was impatient so marge complains

Merge instead of rebase

Would it be possible to add an option to use a git merge instead of a git rebase ?

Barriers to marge-bot working with GitHub?

Hi,

This project seems pretty interesting, and potentially quite useful. And I'm a big fan of GitLab. However, many organizations prefer GitHub to GitLab, which unfortunately means that, because marge-bot is built for GitLab, many organizations can't use this tool.

That makes me curious about what could be done to support either platform. I was hoping that those familiar with the project could outline what currently prevents marge-bot from working with GitHub, and perhaps we can then brainstorm some ideas for how we could get around some of these issues. (I'm imagining that most WebHook Events and APIs needed by marge-bot would exist on both platforms, so I'm hoping that maybe the idea of working on both platforms isn't too far out there...?)

Even if using marge-bot with GitHub came with some caveats that don't apply to GitLab use, having something could certainly be better than nothing for organizations that are currently "stuck" on GitHub. But if there are some legitimate deal-breakers that completely prevent marge-bot from working in any capacity with GitHub, just listing those would also be a satisfactory answer to this issue. Either way, I think discussing this is worthwhile :)

Thanks!

Edit: Of course, one big difference between the two is that GitLab has built-in CI. But I'm wondering how significant this difference is if we can assume that the GitHub repositories using this would have some CI service reporting status checks on each PR.

One should be notified immediately if a request is not mergeable

At the moment, if one assigns to marge a MR that cannot be merged, e.g., because it is marked as WIP or has unresolved conversations, marge will bail out and re-assign it to the owner but only when the time comes to process this merge request.

At busy periods, it may take a while until this happens. The user then needs to resolve the problem (e.g. remove the WIP tag), reassign and wait for long once again for the branch to be merged.

It would be much better if the feedback were almost immediately. We could do this, e.g. with a separate thread that polls continuously for assigned merge requests (those modified after the last check?) and unassigns marge from those in an unmergeable state.

Some issues starting up

Sorry to put some problems in the same issue, I can split it up if it's preferred.

python 3.5.2 complained about the trailing comma in job.py:

def update_from_target_branch_and_push(
        *,
        repo,
        source_branch,
        target_branch,
        source_repo_url=None,
        reviewers=None,
        tested_by=None,
        part_of=None,
        use_merge_strategy=False,
):

removing it worked.

Then there was a problem with (still in job.py) when using use-merge-strategy: true:

        fuse = repo.merge if use_merge_strategy else repo.rebase
        rewritten_sha = updated_sha = fuse(
            branch=source_branch,
            new_base=target_branch,
            source_repo_url=source_repo_url
        )

since repo.merge uses source_branch and target_branch instead of branch and new_base as argument names.

And finally (for now, because I haven't solved it):

2018-05-18 16:00:15,767 INFO Running git clone --origin=origin [email protected]:Jellby/test.git /tmp/tmpmj08l3pw/tmptm1hlsd2
2018-05-18 16:00:18,678 INFO Running git -C /tmp/tmpmj08l3pw/tmptm1hlsd2 config user.email [email protected]
2018-05-18 16:00:18,686 INFO Running git -C /tmp/tmpmj08l3pw/tmptm1hlsd2 config user.name Molcas
2018-05-18 16:00:18,693 INFO Processing !3 - 'dev/* branches and longer job'
2018-05-18 16:00:25,178 INFO Running git -C /tmp/tmpmj08l3pw/tmptm1hlsd2 fetch --prune origin
2018-05-18 16:00:26,163 WARNING git returned 128
2018-05-18 16:00:26,163 WARNING stdout: b''
2018-05-18 16:00:26,165 WARNING stderr: b'Warning: Identity file ssh-key not accessible: No such file or directory.\nPermission denied (publickey).\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n'
2018-05-18 16:00:26,165 INFO Running git -C /tmp/tmpmj08l3pw/tmptm1hlsd2 checkout master --
2018-05-18 16:00:26,175 INFO Running git -C /tmp/tmpmj08l3pw/tmptm1hlsd2 branch -D branch
2018-05-18 16:00:26,182 WARNING git returned 1
2018-05-18 16:00:26,183 WARNING stdout: b''
2018-05-18 16:00:26,183 WARNING stderr: b"error: branch 'branch' not found.\n"
2018-05-18 16:00:26,183 ERROR Unexpected Git error
Traceback (most recent call last):
  File "/home/ubuntu/marge-bot/marge/git.py", line 163, in git
    return _run(*command, env=env, check=True, timeout=timeout_seconds)
  File "/home/ubuntu/marge-bot/marge/git.py", line 189, in _run
    retcode, process.args, output=stdout, stderr=stderr,
subprocess.CalledProcessError: Command '[b'git', b'-C', b'/tmp/tmpmj08l3pw/tmptm1hlsd2', b'fetch', b'--prune', b'origin']' returned non-zero exit status 128

I don't know why it complains about the identity file, since it ran the git clone apparently successfully. Then git fails when it tries to remove the local branch branch, which I guess doesn't exist because it hasn't been set to track the remote one yet...

Better handling of missing approvals after rebase when not asked to reapprove

If the merge request has approvals and marge is not asked to reapprove, then after a rebase she won't be able to merge and will get confused / report the wrong error.

The right logic is to: refetch approvals immediately after the rebase, if there are missing approvals and was asked to reapprove, then reapprove, otherwise, fail with a proper error.

Automate docker image build in Gitthub CI

Rewrite commit hashes in commit messages

When a commit message contains a commit hash that we have rewritten, we should replace it with the rewritten commit hash.

Have option for Marge to automatically remove WIP status of branches to merge

By default, branches named "wip_" will get rejected by Marge as the merge request that is generated will start with "wip" and be marked in progress.

https://docs.gitlab.com/ce/user/project/merge_requests/work_in_progress_merge_requests.html

Typically developers will work on a development branch and label it as "wip_" to distinguish it from production branches, then decide to Marge it - in which case they will expect Marge to remove the WIP tag. Presently, their work flow would have to be:

1. Create development branch (wip...)
2. Submit MR
3. Watch MR get rejected unexpected by Marge as its WIP.
4. Rename their branch after figuring out the problem.
5. Resubmit

This is less than ideal and could be solved if Marge automatically removed the WIP tag. As some teams might actually want this protection in place, I would make it an optional setting of Marge so teams can decide the behaviour they want.

Give a better message when can't merge due to unresolved discussions

At the moment If a request is assigned to marge-bot, there are pending discussions, and GitLab is configured so that requests can be merged until discussions are resolved, she will bail out with:

I couldn't merge this branch: Gitlab refused to merge this request and I don't know why!

We should check if this is due to pending discussions and leave a more informative message instead.

Crash if no pipeline is run

I was testing what happens if no pipeline is run for the MR. Here's what I got (using merge strategy, if it matters):

2018-05-19 15:19:53,666 INFO Commit id to merge '8a8a087a54ea54e464af4dcf2bd457933b00dbe7' (into: '946a97b54cbcd4a47f21a1bed828e87c715a401b')
2018-05-19 15:19:59,405 INFO Waiting for CI to pass
2018-05-19 15:20:00,419 ERROR Unexpected Exception
Traceback (most recent call last):
  File "/home/ubuntu/marge-bot/marge/job.py", line 59, in execute
    self.update_merge_request_and_accept(approvals)
  File "/home/ubuntu/marge-bot/marge/job.py", line 152, in update_merge_request_and_accept
    self.wait_for_ci_to_pass(source_project.id, actual_sha, merge_request.source_branch)
  File "/home/ubuntu/marge-bot/marge/job.py", line 224, in wait_for_ci_to_pass
    assert current_pipeline.sha == commit_sha
AssertionError

I guess this is because there's no pipeline at all (configured with only: master, and master is the target branch, not the source). A graceful handling and a post in the MR would have been much better.

Handling merge requests from forks

A very significant flaw of the merge request model in GitLab is that it seems the only CI linked to the merge request is in the source project. This may be alright in very simple projects and configurations, but I don't think the CI from an external project should be trusted:

1. It could have no access to specific runners
2. Secret variables will not be available
3. Runner time limits could be different

These could cause CI jobs in a fork to fail, with almost no chance of fixing them. Perhaps more serious, I imagine it could be possible for a malicious user to set up a customized runner that will report success where a test should fail, in which case the CI would appear as passing, but it will be broken when run in the target project's runners.

I don't know if there's any way to run pipelines for merge requests on the target project. Apparently the SHA for the MR commit is created in the target project, and a pipeline can be run if a tag or branch is added to it, but it is not connected to the MR. See for example:

https://gitlab.com/Jellby/test/merge_requests/10 The CI on the source project fails (artificially forced to do so, but it could happen in a real case for legitimate reasons as seen above).

The commit appeared in the target repository: https://gitlab.com/Jellby/test/commit/1ca482baa35df4287afaf5313a2f64fc59b62ed5

I created a tag: https://gitlab.com/Jellby/test/commits/testingMR!10 and ran a pipeline: https://gitlab.com/Jellby/test/pipelines/22898164 on the target project, but the MR does not show its result, so the MR can't be accepted.

If someone can find a good solution, that would be great. Otherwise, I'd suggest at least an option to ignore MRs not originating from the same project (and it should be the default if the guarantee for a green master is important).

Re-approvals only applied after successful CI run

There is a comment related to this within marge/job.py, including for context:

# Re-approve the merge request, in case us pushing it has removed
# approvals. Note that there is a bit of a race; effectively
# approval can't be withdrawn after we've pushed (resetting
# approvals) and CI runs.

Occasionally CI may fail due to transient network issues that are unrelated to the change made. In this case, Marge will error out and not bother attempting to reapply any approvals. GitLab doesn't remove approvals on CI failure, so it doesn't quite make sense that this happens with Marge.

This also applies to any potential exception that might occur between the force push and applying approvals, we need to restart marge and then manually approve again.

I'm unaware as to whether there is a historical reason for why approvals are reapplied when they are, but could they no be applied immediately after the rebase?

Handle fixup commits

Marge should probably squash fixup commits automatically.

Is it possible to provide alternative of required approvals for Gitlab CE editions

I believe approval requirement for a merge request is available for Gitlab EE only. In CE edition, this option is not available. So, is there any other way of protecting unwanted merges by merge-bot?

Parallel Builds

In projects that have lots of merges and long build times, a synchronous build queue is not viable.

See https://gitlab.com/gitlab-org/gitlab-ce/issues/4176

Each merge request in the queue would need to be proactively rebased on the previous one before it is finished so that the builds could continue in parallel.

Downsides:

• The implementation would require rebasing the entire queue when a build fails
• The commit log and diff would contain unrelated changes (temporarily until master catches up)

Remove merge-commit-attempts when using the merge strategy

With the experimental support for merge-commits (see #72), spurious merge-commits may be left around. A possible scenario would be:

1. One assigns to marge
2. A merge commit is thus pushed to the branch
3. A flaky test fails
4. Another branch gets merged in
5. One reassigns to marge
6. A new merge commit is pushed, but the previous one is still there

This is a bit ugly, specially since the first commit will be marked as "Tested-by: marge" even though CI failed there. Ideally, the second time around, the failed merge-commit would be identified and removed.

iterating over projects takes over 30 seconds

as marge checks each project it is assigned one by one, and having over 50 projects assigned, the whole cycle (with no jobs found) takes ~2 minutes. how to solve this and make marge process assigned jobs sooner?

marge 0.6.1

# docker-compose logs |grep -E 'INFO Sleeping|INFO Finding out my current projects'
marge-bot_1  | 2018-03-26 20:06:22,903 INFO Finding out my current projects...
marge-bot_1  | 2018-03-26 20:08:14,924 INFO Sleeping for 0 seconds...
marge-bot_1  | 2018-03-26 20:08:14,924 INFO Finding out my current projects...
marge-bot_1  | 2018-03-26 20:10:07,329 INFO Sleeping for 0 seconds...
marge-bot_1  | 2018-03-26 20:10:07,330 INFO Finding out my current projects...

# docker-compose logs | sed -ne 's/.*INFO Fetching merge requests assigned to me in //p' | sort -u | wc -l
82

unexpected merge state locked

marge-bot_1  | 2018-02-05 12:39:47,026 INFO Commit id to merge 'bb052510c9a6357fe733b793f4c1f5b38a32e89e' (into: 'aa5d3b11e3768d34f74bde8de7095a629d76cba1')
marge-bot_1  | 2018-02-05 12:39:53,476 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:40:03,757 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:40:14,265 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:40:24,748 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:40:35,059 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:40:45,633 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:40:55,966 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:41:06,263 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:41:16,685 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:41:27,039 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:41:37,310 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:41:47,509 INFO Giving 10 more secs for !125 to be merged...
marge-bot_1  | 2018-02-05 12:41:57,879 ERROR Unexpected Exception
marge-bot_1  | Traceback (most recent call last):
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/job.py", line 55, in execute
marge-bot_1  |     self.rebase_and_accept(approvals)
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/job.py", line 198, in rebase_and_accept
marge-bot_1  |     self.wait_for_branch_to_be_merged()
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/job.py", line 238, in wait_for_branch_to_be_merged
marge-bot_1  |     assert merge_request.state in ('opened', 'reopened'), merge_request.state
marge-bot_1  | AssertionError: locked
marge-bot_1  | Traceback (most recent call last):
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/bin/.marge.app-wrapped", line 4, in <module>
marge-bot_1  |     marge.app.main()
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/app.py", line 221, in main
marge-bot_1  |     marge_bot.start()
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/bot.py", line 39, in start
marge-bot_1  |     self._run(repo_manager)
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/bot.py", line 108, in _run
marge-bot_1  |     merge_job.execute()
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/job.py", line 55, in execute
marge-bot_1  |     self.rebase_and_accept(approvals)
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/job.py", line 198, in rebase_and_accept
marge-bot_1  |     self.wait_for_branch_to_be_merged()
marge-bot_1  |   File "/nix/store/v4xzqxg6yvhvdqk6w453pig8v7r69v1c-python3.6-marge-0.5.1/lib/python3.6/site-packages/marge/job.py", line 238, in wait_for_branch_to_be_merged
marge-bot_1  |     assert merge_request.state in ('opened', 'reopened'), merge_request.state
marge-bot_1  | AssertionError: locked

here's how it looked at gitlab web:

TypeError: 'NoneType' object is not subscriptable

We're trying to get marge-bot up and running on our own gitlab instance, and might have found a bug?

> marge/project.py
>> return AccessLevel(self.info['permissions']['project_access']['access_level'])
>> TypeError: 'NoneType' object is not subscriptable

After looking into this a bit, it looks like gitlabs api will return permissions only on the /projects/:id request, but not for /projects.

Marge crashing with an UnicodeEncodeError

  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/job.py", line 55, in execute
    self.rebase_and_accept(approvals)
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/job.py", line 111, in rebase_and_accept
    tested_by=tested_by,
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/job.py", line 298, in push_rebased_and_rewritten_version
    start_commit='origin/' + target_branch,
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/git.py", line 52, in tag_with_trailer
    self.git('filter-branch', '--force', '--msg-filter', filter_script, commit_range)
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/git.py", line 138, in git
    return _run(*command, env=env, check=True, timeout=TIMEOUT_IN_SECS)
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/git.py", line 147, in _run
    with subprocess.Popen(args, env=env, stdout=PIPE, stderr=PIPE) as process:
  File "/nix/store/an47r56zf6xn6fp2kyfygdy8xbplzyc2-python3-3.6.1/lib/python3.6/subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "/nix/store/an47r56zf6xn6fp2kyfygdy8xbplzyc2-python3-3.6.1/lib/python3.6/subprocess.py", line 1260, in _execute_child
    restore_signals, start_new_session, preexec_fn)
UnicodeEncodeError: 'ascii' codec can't encode character '\xe3' in position 25: ordinal not in range(128)
Traceback (most recent call last):
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/bin/.marge.app-wrapped", line 4, in <module>
    marge.app.main()
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/app.py", line 151, in main
    marge_bot.start()
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/bot.py", line 36, in start
    self._run(repo_manager)
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/bot.py", line 90, in _run
    merge_job.execute()
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/job.py", line 55, in execute
    self.rebase_and_accept(approvals)
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/job.py", line 111, in rebase_and_accept
    tested_by=tested_by,
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/job.py", line 298, in push_rebased_and_rewritten_version
    start_commit='origin/' + target_branch,
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/git.py", line 52, in tag_with_trailer
    self.git('filter-branch', '--force', '--msg-filter', filter_script, commit_range)
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/git.py", line 138, in git
    return _run(*command, env=env, check=True, timeout=TIMEOUT_IN_SECS)
  File "/nix/store/q0m1zzzkxhliiyjhvhidisx9ja196v64-python3.6-marge-0.3.2/lib/python3.6/site-packages/marge/git.py", line 147, in _run
    with subprocess.Popen(args, env=env, stdout=PIPE, stderr=PIPE) as process:
  File "/nix/store/an47r56zf6xn6fp2kyfygdy8xbplzyc2-python3-3.6.1/lib/python3.6/subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "/nix/store/an47r56zf6xn6fp2kyfygdy8xbplzyc2-python3-3.6.1/lib/python3.6/subprocess.py", line 1260, in _execute_child
    restore_signals, start_new_session, preexec_fn)
UnicodeEncodeError: 'ascii' codec can't encode character '\xe3' in position 25: ordinal not in range(128)

There are two bugs here:

• we should always force utf-8 encoding and decoding, locale be damned (the problem here almost certainly is that the LANG/LC_ALL are not .utf-8)
• marge-bot should unassign hereself on failure

Handle CI status 'skipped'

Marge-bot will currently wait up until the ci-timeout before reporting that the CI is taking too long if the status is 'skipped'. This status will occur if the top commit message of the MR contains the magic [ci skip] tag.

This status should either be treated in the same way as 'success', since the user is explicitly stating that the CI needn't run for the particular change. Or instead reported as an error earlier, with a clearer message. This could perhaps be configurable.

Marge fails to fetch projects, 0.7.0 regression

Gitlab instance: Gitlab CE 11.1 gitlab.gnome.org

Using the docker image:

When running the 0.7.0 tag and try to merge an MR from a fork, we get the following error.
The MR in question is here, it happened with other MRs too. This specific MR has comments from months ago when we were hitting this gitlab bug that prevented from using marge, You can ignore those and only focus on the ones from this day.

2018-08-13 04:19:11,202 INFO Processing !4 - 'Add a non-working test, from a branch'
2018-08-13 04:19:13,219 INFO Ensuring MR !4 is mergeable
2018-08-13 04:19:15,781 ERROR Unexpected Exception
Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 50, in update_merge_request_and_accept
    source_project, source_repo_url, _ = self.fetch_source_project(merge_request)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 200, in fetch_source_project
    remote_url=remote_url,
TypeError: fetch() got an unexpected keyword argument 'remote'
2018-08-13 04:19:17,796 INFO Unassigning from MR !4
Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/bin/.marge.app-wrapped", line 4, in <module>
    marge.app.main()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/app.py", line 261, in main
    marge_bot.start()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 41, in start
    self._run(repo_manager)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 59, in _run
    projects,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 97, in _process_projects
    self._process_merge_requests(repo_manager, project, merge_requests)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 164, in _process_merge_requests
    merge_job.execute()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 50, in update_merge_request_and_accept
    source_project, source_repo_url, _ = self.fetch_source_project(merge_request)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 200, in fetch_source_project
    remote_url=remote_url,
TypeError: fetch() got an unexpected keyword argument 'remote'

Then I setup an instance from the 0.6.1 tag and it succeded.

2018-08-13 04:29:31,163 INFO Fetching merge requests assigned to me in alatiera/Bors-test...
2018-08-13 04:29:33,107 INFO Got 1 requests to merge; will try to merge the oldest
2018-08-13 04:29:33,107 INFO Running git clone --origin=origin [email protected]:alatiera/Bors-test.git /tmpubhyfb3i/tmpggc__srt
2018-08-13 04:29:36,851 INFO Running git -C /tmpubhyfb3i/tmpggc__srt config user.email [email protected]
2018-08-13 04:29:36,856 INFO Running git -C /tmpubhyfb3i/tmpggc__srt config user.name 'Merge Bot, Bors Wannabe'
2018-08-13 04:29:36,861 INFO Processing !4 - 'Add a non-working test, from a branch'
2018-08-13 04:29:40,206 INFO Running git -C /tmpubhyfb3i/tmpggc__srt fetch --prune origin
2018-08-13 04:29:43,478 INFO Running git -C /tmpubhyfb3i/tmpggc__srt remote rm source
2018-08-13 04:29:43,483 WARNING git returned 128
2018-08-13 04:29:43,483 WARNING stdout: b''
2018-08-13 04:29:43,483 WARNING stderr: b'fatal: No such remote: source\n'
2018-08-13 04:29:43,483 INFO Running git -C /tmpubhyfb3i/tmpggc__srt remote add source [email protected]:federico/Bors-test.git
2018-08-13 04:29:43,492 INFO Running git -C /tmpubhyfb3i/tmpggc__srt fetch --prune source
2018-08-13 04:29:47,240 INFO Running git -C /tmpubhyfb3i/tmpggc__srt checkout -B test-non-working source/test-non-working --
2018-08-13 04:29:47,247 INFO Running git -C /tmpubhyfb3i/tmpggc__srt rebase origin/master
2018-08-13 04:29:47,336 INFO Running git -C /tmpubhyfb3i/tmpggc__srt rev-parse HEAD
2018-08-13 04:29:47,341 INFO Running git -C /tmpubhyfb3i/tmpggc__srt rev-parse origin/master
2018-08-13 04:29:47,345 INFO Running git -C /tmpubhyfb3i/tmpggc__srt checkout test-non-working --
2018-08-13 04:29:47,355 INFO Running git -C /tmpubhyfb3i/tmpggc__srt diff-index --quiet HEAD
2018-08-13 04:29:47,360 INFO Running git -C /tmpubhyfb3i/tmpggc__srt ls-files --others
2018-08-13 04:29:47,365 INFO Running git -C /tmpubhyfb3i/tmpggc__srt config --get remote.source.url
2018-08-13 04:29:47,370 INFO Running git -C /tmpubhyfb3i/tmpggc__srt push --force source test-non-working
2018-08-13 04:29:50,533 INFO Running git -C /tmpubhyfb3i/tmpggc__srt checkout master --
2018-08-13 04:29:50,539 INFO Running git -C /tmpubhyfb3i/tmpggc__srt branch -D test-non-working
2018-08-13 04:29:50,545 INFO Commit id to merge '18a991a77917afc735b088c9ad434e3db355cde9' (into: '79375273a12686d15e230fa39bec69107966c83c')
2018-08-13 04:30:05,322 INFO Successfully merged !4.

Handle protected branches

If a branch has been protected in GitLab, then Marge cannot force push to it (which is necessary if Marge has added trailers, or rebased the MR). However, there's no handling for this case and so Marge simply blows up with the "broken on the inside" message.

Make git pull timeout configurable

in marge/git.py, you're hardcoding the git clone timeout to 120 seconds. While this may be a sensible default, it absolutely doesn't work for large repositories which take longer to clone.

I'd rather propose this to be configurable by either an environment variable (MARGE_GIT_TIMEOUT?) or a command option (or both?)

Don't assume a branch called `master` exists when removing the merged branch

It does not always merge request should be merged into master. E.g., in some git workflows, MR should be merged into develop branch. I propose to add configuration parameter --branch-name-merge-into <string name> for such situations.

Rebase a branch without merge to keep it up to date

Sometimes you have a branch which lasts on a long term development (sereral weeks or months).
The merge operation can be painfull after waiting so long and one is not interested in regularly rebasing himself that branch.

Could Marge do it for me please ?

merge failure because base branch not updated

marge-bot_1  | 2017-08-17 11:47:36,764 INFO Processing !39 - 'Add 360 video support, DVID-103'
marge-bot_1  | 2017-08-17 11:47:36,944 INFO Running git -C /tmpa7qybnrd/tmphn_oz1jn fetch origin
marge-bot_1  | 2017-08-17 11:47:37,667 INFO Running git -C /tmpa7qybnrd/tmphn_oz1jn checkout -B DVID-103_add_360_video_support origin/DVID-103_add_360_video_support --
marge-bot_1  | 2017-08-17 11:47:37,687 INFO Running git -C /tmpa7qybnrd/tmphn_oz1jn rebase origin/master
marge-bot_1  | 2017-08-17 11:47:37,777 WARNING git returned 128
marge-bot_1  | 2017-08-17 11:47:37,777 WARNING stdout: b'First, rewinding head to replay your work on top of it...\nApplying: change1\nApplying: change2\nApplying: change3\nUsing index info to reconstruct a base tree...\nM\tfile1.php\nM\tfile2.php\nFalling back to patching base and 3-way merge...\nAuto-merging file2.php\nCONFLICT (content): Merge conflict in file2.php\nAuto-merging file1.php\nCONFLICT (content): Merge conflict in file1.php\nPatch failed at 0003 change1\nThe copy of the patch that failed is found in: .git/rebase-apply/patch\n\nWhen you have resolved this problem, run "git rebase --continue".\nIf you prefer to skip this patch, run "git rebase --skip" instead.\nTo check out the original branch and stop rebasing, run "git rebase --abort".\n\n'
marge-bot_1  | 2017-08-17 11:47:37,777 WARNING stderr: b'error: Failed to merge in the changes.\n'
marge-bot_1  | 2017-08-17 11:47:37,778 WARNING rebase failed, doing an --abort
marge-bot_1  | 2017-08-17 11:47:37,778 INFO Running git -C /tmpa7qybnrd/tmphn_oz1jn rebase --abort
marge-bot_1  | 2017-08-17 11:47:37,823 INFO Running git -C /tmpa7qybnrd/tmphn_oz1jn checkout master --
marge-bot_1  | 2017-08-17 11:47:37,833 INFO Running git -C /tmpa7qybnrd/tmphn_oz1jn branch -D DVID-103_add_360_video_support
marge-bot_1  | 2017-08-17 11:47:37,837 WARNING I couldn't merge this branch: got conflicts while rebasing, your problem now...

when i check workspace of marge bot, it's at 4d81b82, but real origin/master is at 8fba43a
and therefore the merge conflicts because base branch is out of date.

also seems that origin master is fetched and up to date. perhaps merge against origin/master not master?

bash-4.4# GIT_DIR=/tmpa7qybnrd/tmphn_oz1jn/.git /nix/store/m9wmmxajfxpw06sn0wv3gmvda2zlgmv2-git-2.13.3/bin/git show -s HEAD
commit 4d81b823fb7038b90a36bd2a8bd9da378a0e111a (HEAD -> master)


bash-4.4# GIT_DIR=/tmpa7qybnrd/tmphn_oz1jn/.git /nix/store/m9wmmxajfxpw06sn0wv3gmvda2zlgmv2-git-2.13.3/bin/git show -s master
commit 4d81b823fb7038b90a36bd2a8bd9da378a0e111a (HEAD -> master)

bash-4.4# GIT_DIR=/tmpa7qybnrd/tmphn_oz1jn/.git /nix/store/m9wmmxajfxpw06sn0wv3gmvda2zlgmv2-git-2.13.3/bin/git show -s origin/master
commit 8fba43a37831666af74a6b5de71cd6089ccf71c9 (origin/master, origin/HEAD)
Merge: 4e24499 dcb781f
bash-4.4#

stale ssh private key files

the current version (59ace57) leaves temporary ssh key files laying around.

first problem is that the temporary file is created even if --ssh-key-file commandline is used.

and second problem that the temporary files are never removed. thus root filesystem of docker container is filled with ssh private key files (probably system temp dir for non-docker runs).

bash-4.4# ls -l /
-rw-------   1 root root 3243 Aug  7 09:07 ssh-key
-rw-------   1 root root    0 Aug  7 10:36 ssh-key-rgss44m3
-rw-------   1 root root    0 Aug  7 10:35 ssh-key-tmagyr7i

ssh-key is mounted via docker volume and used as commandline --ssh-key-file=/ssh-key, the other two came from processing (two?) merge requests.

Provide a Dockerfile to simplify deploying

The Dockerfile should provide at least:

• Code and python dependencies
• A version of git that supports GIT_SSH_COMMAND
• Some hooks to provide credentials (environment variables?)
• Ensure the certificate of gitlab host is added to ~/.ssh/known_host/ before starting

merge failure (405 method not allowed on merge)

marge got:

Reporter   
I couldn't merge this branch: had some issue with gitlab, check my logs...

i checked the logs and it has:

marge-bot_1  | 2017-08-09 06:29:43,992 ERROR Unanticipated ApiError from Gitlab on merge attempt
marge-bot_1  | Traceback (most recent call last):
marge-bot_1  |   File "/nix/store/9vhc0sbcxii1s05sz755ws8gr46iynqw-python3.6-marge-0.2.0/lib/python3.6/site-packages/marge/job.py", line 136, in rebase_and_accept
marge-bot_1  |     merge_request.accept(remove_branch=True, sha=actual_sha)
marge-bot_1  |   File "/nix/store/9vhc0sbcxii1s05sz755ws8gr46iynqw-python3.6-marge-0.2.0/lib/python3.6/site-packages/marge/merge_request.py", line 104, in accept
marge-bot_1  |     sha=sha or self.sha,  # if provided, ensures what is merged is what we want (or fails)
marge-bot_1  |   File "/nix/store/9vhc0sbcxii1s05sz755ws8gr46iynqw-python3.6-marge-0.2.0/lib/python3.6/site-packages/marge/gitlab.py", line 55, in call
marge-bot_1  |     raise error(response.status_code, err_message)
marge-bot_1  | marge.gitlab.MethodNotAllowed: (405, {'message': '405 Method Not Allowed'})
marge-bot_1  | 2017-08-09 06:29:44,545 WARNING I couldn't merge this branch: had some issue with gitlab, check my logs...

questions:

1. what could be cause, is it temporary error?
2. how to make marge retry?

i changed marge user access from reporter to developer, i used reporter before because such privilege worked in earlier project (marge itself is admin), but i don't see how to make marge retry. i even force pushed last commit removing the tested-by lines and restarted the bot docker container.

Configurable ci backend?

Would that be possible to have a different ci then gitlab-ci

Marge should not merge MRs with failed CI unless explicity stated by a config.

So here is the scenario, we setup marge with default config and made a failing MR to test. We assigned marge to the MR but instead of complaining it did go ahead and merge it since we hadn't yet turned on this giltlab flag for the project.

I think Marge should prevent stupid mistakes like this and have it require a flag be set if you really want her to merge failing CI MR.

Leave a message on the MR when not merging due to an embargo

The complication is that we need to keep track which MRs were already notified about the embargo

Syntax error for using bare asterisk as function args

As I known, in Python, you can use this syntax to force caller of function to use keyword args after *:

def func(a, b, *, c=None, d=None):
    ....

Which means, caller have to call func with:

result = func(a, b, c="c-value", d="d-value")

Other than:

result = func(a, b, "c-value", "d-value")

However, in marge-bot source code, I found such examples:

class Bot(object):                                                              
    def __init__(self, *, api, config):                                          
        self._api = api                                                         
        self._config = config
        ....
...

class MergeJob(object):

    def __init__(self, *, api, user, project, merge_request, repo, options):
        self._api = api
        self._user = user
        self._project = project
        ....

And I got SyntaxError for those lines in both Python2 and Python3.

Just wondering what am doing wrong, and why didn't other people encount this error?

Marge times out waiting for the MR to merge instead of detecting CI failure.

When "Only allow merge requests to be merged if the pipeline succeeds" feature is not enabled Marge seems to timeout waiting for the merge to happen instead of detecting the CI failure. And instead of reporting the expected "CI failed" comment, she says:
I couldn't merge this branch: It is taking too long to see the request marked as merged!

cc: @federicomenaquintero

Add reference to original merge request in commit message

Marge could put a link to original merge request in the commit message, presumably for every commit in a MR.