carlos-jenkins/python-github-webhooks

Simple Python WSGI application to handle Github webhooks

python-github-webhooks's Issues

Fails under WAMP configuration

Shebang is not supported in windows. :/
Running this in a WAMP environment, it fails.

not work with secret

File "/app/webhooks.py", line 82, in index
    mac = hmac.new(str(secret), msg=request.data, digestmod='sha1')
  File "/usr/local/lib/python2.7/hmac.py", line 136, in new
    return HMAC(key, msg, digestmod)
  File "/usr/local/lib/python2.7/hmac.py", line 52, in __init__
    self.outer = self.digest_cons()
  File "/usr/local/lib/python2.7/hmac.py", line 50, in <lambda>
    self.digest_cons = lambda d='': digestmod.new(d)
AttributeError: 'str' object has no attribute 'new'

Test the web app before deploying?

Is there a recommended way to test the web app defined in the script webhooks.py locally? Sorry if this is obvious, but I'm new to webhooks and web apps in general.

More logs is needed

At this time, when there is an issue with GitHub integration (like test returning 301 code), there is absolutely no logs or whatever.

We need both access and error logs to get a chance to understand where the issue can be.

500 Server Error

Hi,

I have my webhook server up and running (apache calling python-github-webhooks via WSGI), however, I see the following error in my logs whenever GitHub tries to POST something to this endpoint:

[Wed Jan 25 12:31:26.499245 2017] [:info] [pid 62282] [client 10.240.0.187:40078] mod_wsgi (pid=62282, process='', application='blahblah.c9users.io|/webhooks'): Loading WSGI script '/home/ubuntu/python-github-webhooks/webhooks.py'.
[Wed Jan 25 12:31:26.661105 2017] [:error] [pid 62282] [client 10.240.0.187:40078] [2017-01-25 12:31:26,659] ERROR in app: Exception on / [POST]
[Wed Jan 25 12:31:26.661239 2017] [:error] [pid 62282] [client 10.240.0.187:40078] Traceback (most recent call last):
[Wed Jan 25 12:31:26.661279 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
[Wed Jan 25 12:31:26.661286 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     response = self.full_dispatch_request()
[Wed Jan 25 12:31:26.661291 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request
[Wed Jan 25 12:31:26.661295 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     rv = self.handle_user_exception(e)
[Wed Jan 25 12:31:26.661300 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception
[Wed Jan 25 12:31:26.661305 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     reraise(exc_type, exc_value, tb)
[Wed Jan 25 12:31:26.661309 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request
[Wed Jan 25 12:31:26.661314 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     rv = self.dispatch_request()
[Wed Jan 25 12:31:26.661318 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request
[Wed Jan 25 12:31:26.661323 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     return self.view_functions[rule.endpoint](**req.view_args)
[Wed Jan 25 12:31:26.661326 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/home/ubuntu/python-github-webhooks/webhooks.py", line 52, in index
[Wed Jan 25 12:31:26.661329 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     config = loads(cfg.read())
[Wed Jan 25 12:31:26.661332 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
[Wed Jan 25 12:31:26.661335 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     return _default_decoder.decode(s)
[Wed Jan 25 12:31:26.661338 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
[Wed Jan 25 12:31:26.661341 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
[Wed Jan 25 12:31:26.661344 2017] [:error] [pid 62282] [client 10.240.0.187:40078]   File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
[Wed Jan 25 12:31:26.661346 2017] [:error] [pid 62282] [client 10.240.0.187:40078]     obj, end = self.scan_once(s, idx)
[Wed Jan 25 12:31:26.661355 2017] [:error] [pid 62282] [client 10.240.0.187:40078] ValueError: Expecting , delimiter: line 5 column 5 (char 129)

I tried configuring the webhooks on the GitHub side, as having the Content Type header application/json and also application/x-www-form-urlencoded but for both, I'm getting the same error, what could be wrong?

Many thanks in advance,
VG

Docker build

I need to extend your image and so i want to base my Dockerfile on yours. I get this error:

Sending build context to Docker daemon   2.56kB
Step 1/5 : FROM carlos-jenkins/python-github-webhooks
pull access denied for carlos-jenkins/python-github-webhooks, repository does not exist or may require 'docker login'

Can you please make a docker hub integration that will pull the master releases and build images? It's very easy, just follow these instructions: https://docs.docker.com/docker-hub/builds/

hooks on docker

In my hooks, I've had to use git (among other tools) as a specific user (whose uid is 999).

Since git was not provided by the carlos-jenkins/python-github-webhooks docker image (and I did not want to hack into it) I derived a customized image based on yours, with this Dockerfile.extended:

FROM carlos-jenkins/python-github-webhooks

MAINTAINER "François Van Der Biest" <[email protected]>

# add packages required to run your hooks, eg:
RUN apk update && apk add bash git openssh-client

# create user which will run hooks (group ping has gid=999 in base image)
RUN adduser -S -G ping -s /bin/bash -u 999 sftp

# required here to populate root's known_hosts so that git pull command 
# does not interactively ask to check RSA key fingerprint:
RUN mkdir -p /root/.ssh && \
    chmod 700 /root/.ssh && \
    ssh-keyscan github.com >> /root/.ssh/known_hosts

then: docker build -t fvanderbiest/python-github-webhooks -f Dockerfile.extended .

Finally, I set the setuid bit on my hook, and gave it to user with uid 999:
chmod u+s push-myrepo-mybranch

It works great !
This is not really an issue, but I thought it might be useful to others...

hmac.compare_digest does not exist in Python < 2.7.7

One issue I ran into with this webhooks script is that hmac.compare_digest is new in 2.7.7, and for legacy reasons I'm stuck on 2.6.

I have a few different ways to solve this problem in a general way, and I wanted to get your feedback before I submit a pull request.

Option 1: fall back to a simple string comparison if hmac.compare_digest is unavailable. This is functionally equivalent, but simply fails to provide protection against a timing attack. (It's not clear to me that a timing attack is even a concern, given the inherit network latency involved, so this may be perfectly safe.)

Option 2: have an option, disabled by default, to allow the fallback. This is what I have implemented locally. Code snippet below.

Option 3: have a fallback to a python-based version of compare_digest. This would be slower than the C implementation in the standard library, but would provide the same protection. It would also require adding a python-fallback to the repo, as I don't think there's one available in a pip package anywhere.

The code for option 2 would look something like:

        # HMAC requires the key to be bytes, but data is string
        mac = hmac.new(str(secret), msg=request.data, digestmod=sha1)

        # Python prior to 2.7.7 does not have hmac.compare_digest
        try:
            if not hmac.compare_digest(str(mac.hexdigest()), str(signature)):
                abort(403)
        except AttributeError as e:
            # What compare_digest provides is protection against timing attacks, but if
            # the user is okay with not having this protection, let's let them allow a
            # trivial fallback
            if not config.get('allow_insecure_compare_digest', False):
                logging.error('Could not call hmac.compare_digest; are you running python < 2.7.7?', exc_info=True)
                abort(500)

            if not str(mac.hexdigest()) == str(signature):
                abort(403)

Let me know your thoughts, and I can prepare a pull request.

Major issue: not closing temporary file

I finally noticed that you missed close() after an uptime of one week:

ERROR:uwsgi_file__***:Exception on / [POST]  
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app  
    response = self.full_dispatch_request()
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request  
    rv = self.handle_user_exception(e)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception  
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request  
    rv = self.dispatch_request()
  File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request  
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/var/www/cgi-bin/***/webhooks/github.py", line 52, in index  
    with open(join(path, 'github/config.json'), 'r') as cfg:

Ouch.

This was determined by this:

# sh -c 'readlink /proc/30756/fd/*'
/tmp/tmpjCxknv (deleted)
/tmp/tmp5p0JBV (deleted)
/tmp/tmp7W680K (deleted)
/tmp/tmpLp_FMI (deleted)