sigmavirus24/github3.py

Hi, I'm a library for interacting with GItHub's REST API in a convenient and ergonomic way. I work on Python 3.6+.

github3.py is a comprehensive, actively developed, and extraordinarily stable wrapper around the GitHub API (v3).

Note: This library currently works with Python 3.7+ or pypy3. For older versions, please use version 1.3.0.

Installation

$ pip install github3.py

Dependencies

• requests
• uritemplate
• python-dateutil
• PyJWT

Contributing

Please read the CONTRIBUTING document.

Testing

You can run pip install -e .[dev] to install the following before testing or simply make test-deps. It is suggested you do this in a virtual environment. These need to be installed for the tests to run.

• betamax
• coverage by Ned Batchelder

Build status

You can find build statuses for different environments.

• Github

License

Modified BSD license

Examples

See the docs for more examples.

Testing

Install the dependencies from requirements.txt e.g.:

make tests

Author

Ian Stapleton Cordasco (sigmavirus24)

Contact Options

• Feel free to use the github3.py tag on Stack Overflow for any questions you may have.
• If you dislike Stack Overflow, it is preferred that you use Github discussions (https://github.com/sigmavirus24/github3.py/discussions).
• You may also contact (via email) the author directly with questions/suggestions/comments or if you wish to include sensitive data.

github3.py's Issues

TypeError exception when trying to create an existing repo `Org.create_repo`

When my script tries to create a repo that already exists, github responds with a status code > 400. But github3.py reports the error as a type error.

---------------------------------------------------------------------------
GitHubError                               Traceback (most recent call last)
/usr/local/lib/python2.7/dist-packages/IPython/utils/py3compat.pyc in execfile(fname, *where)
    176             else:
    177                 filename = fname
--> 178             __builtin__.execfile(filename, *where)

/home/seth/code/GITenburg/GITenburg.py in <module>()
    178     #update_catalog()
    179     catalog = load_catalog()
--> 180     do_stuff(catalog)

/home/seth/code/GITenburg/GITenburg.py in do_stuff(catalog)
    171         #create_metadata_yaml(book, folder)
    172         #make_local_repo(folder)
--> 173         repo = create_github_repo(book.title)
    174         git_add_remote_origin(repo.git_url, folder)
    175         git_push_origin_master(folder)

/home/seth/code/GITenburg/GITenburg.py in create_github_repo(title)
    125     team = org.list_teams()[0] # only one team in the github repo
    126 
--> 127     repo = org.create_repo( title, description= u'A Project Gutenberg book, now on Github.  Se
e [GITenberg](http=//GITenberg.github.com/) for more information', homepage=u'http=//GITenberg.github.
com/', private=False, has_issues=True, has_wiki=False, has_downloads=True, team_id=int(team.id))
    128 
    129     print repo.html_url

/usr/local/lib/python2.7/dist-packages/github3/models.pyc in auth_wrapper(self, *args, **kwargs)
    142 
    143             if auth:
--> 144                 return func(self, *args, **kwargs)
    145             else:
    146                 raise GitHubError(type('Faux Request', (object, ),

/usr/local/lib/python2.7/dist-packages/github3/orgs.py in create_repo(self, name, description, homepag
e, private, has_issues, has_wiki, has_downloads, team_id)
    251         if team_id > 0:
    252             data.update({'team_id': team_id})
--> 253         json = self._json(self._post(url, dumps(data)), 201)
    254         return Repository(json, self) if json else None
    255 

/usr/local/lib/python2.7/dist-packages/github3/models.pyc in _json(self, request, status_code)
     66             ret = request.json
     67         if request.status_code >= 400:
---> 68             raise GitHubError(request)
     69         return ret
     70 

<type 'str'>: (<type 'exceptions.TypeError'>, TypeError('sequence item 0: expected string, dict found'
,))
```1

Switch to a different unittest framework

Several people have indicated that using garybernhardt/expecter has made the tests confusing or somewhat inconsistent in their structure (tests/test_api vs every other set of tests). I believe @dahlia on twitter suggested a move to py.test. The standard unittest would be module except that for python 2.6 we would always require unittest2.

So the following are our current options and they're capable of being expanded by new suggestsions:

• stick with expecter+mock
• move to strictly unittest+mock (+unittest2 on python 2.6)
• move to py.test+mock

No matter what, mock will have to stay unless someone can find a better mocking library. (It's even in the stdlib as of python 3.3.0 ;)).

Octocat

Cover api.github.com/octocat

`AttributeError: NoneType has no attribute "get"` when commits don't have a committer/author

Pull request forthcoming.

Repository.refresh() is documented but not present.

Is it deprecated?

Upgrade dependency of package requests

setup.py says it depends on requests==1.1.0, but the latest version of requests is 1.2.0

Do we need to hardcode the version number? Or just requests>=1.1.0?

Debian packaging

I've added debian packaging files in the 'debian' branch in my clone. If you're interested, please pull them into a branch in your repo. Don't merge into master as Debian devs prefer debian/* files to be in a separate branch.

Incorporate notifications API when released

github/developer.github.com#153

list_followers no longer exists in GitHub

list_followers() in the following code snippet in http://github3py.readthedocs.org/en/latest/ no longer exists.

print(kennethreitz.name)
print(kennethreitz.login)
print(kennethreitz.followers)

gh.list_followers('kennethreitz')

More thoroughly mock the tests

@zyga found an issue with Repository.create_comment due to position being given a default argument of 1.

We can't test directly against the API, but I need to go through with the documentation and more thoroughly write more for the tests to catch these bugs.

Fix usage of expect.raises

I'm not sure why, but I was under the impression that using expect.raises for multiple calls that should raise errors would go through each of them. Instead it stops after the first one. So I have several places where I do:

with expect.raises(github3.GitHubError):
    # multiple calls that should all raise GitHubError

This actually will not test that the calls after the first raise GitHubError.

Distinguish stargazers and subscribers

http://developer.github.com/v3/repos/watching/
http://developer.github.com/changes/

github3.py does not install on Heroku

Issue type: Bug

---

Traceback:

None

---

Description:
When specifying github3.py as a requirement for a Heroku application, slug compilation fails. This started happening in version 0.3

---

Generated with github3.py using the report_issue script

Refactor

The library is pretty damn stable (in my humble opinion) and I'm annoyed personally by having some of the huge files that we do, e.g., github3/repos.py. I'm going to start splitting them off into sub-modules, e.g., github3/repos/{repo,repocommit,...}.py.

Thanks to code coverage and the unittests I can be certain I won't break anything with this refactor. ;)

On a separate note, I'm wondering if there is anything besides what issues are open that people might be interested in seeing included. While refactoring, I think I'm going to take care of #63 and #69 myself.

I think subclassing JSONEncoder is over-kill provided we have the {to,from}_json methods.

I'll probably echo this to the mailing list that I've all but neglected, in case there's anyone there with a different opinion. (Assuming there's anyone there at all :))

Make ``refresh`` and other methods conditional-friendly

This entails:

• Storing the Last-Modified and ETag headers on the object.
  • Make this transparent to the user? For example, add attributes for each of those or store them in a headers dictionary that is a regular attribute?
• Allowing for an optional parameter, e.g., repo.refresh(conditional=True) or repo.refresh(if_modified=True). (I personally like the latter, it is less ambiguous.)

Caching support!

Split off from #75, github3.py will be adding support for caching!

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/450183-caching-support?utm_campaign=plugin&utm_content=tracker%2F183477&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F183477&utm_medium=issues&utm_source=github).

Calls to self._post without JSON encoding the data

There are several calls to self._post where the data is not being JSON encoded. This fails with the GitHub3 API as that only accepts JSON-encoded data.

E.g.repos.py line 1154

Since POST has to JSON encode data (and the code always sends a application/json header), it would be better to move the dumps() into _post, so that each individual method doesn't have to worry about it.

Release 0.1b

I have all modules above 90% coverage. The entire project is at 96% coverage. I'm going to try to get these numbers higher, but I want tto release this weekend.

An "github3.models.GitHubError: 422 Validation Failed" will occured if milestone is specified by "create_issue”

I use version 0.4.
I write following code.

gh= login(user, password)
repo = gh.repository('masahitojp', 'mysite')

title = "foo"
body="bar"

# create_issue with milestone
repo.create_issue(title=title , body=body, milestone=2.1,labels=["2.1"])

# It OKs, if you have no milestone. 
# repo.create_issue(title=title , body=body, labels=["2.1"])

but It will become an error if milestone is specified.
Why?

Traceback (most recent call last):
  File "cccc.py", line 20, in <module>
    repo.create_issue(title=title , body=body, milestone=2.1,labels=["2.1"])
  File "c:\python27\lib\site-packages\github3.py-0.4-py2.7.egg\github3\decorators.py", line 31, in auth_wrapper
    return func(self, *args, **kwargs)
  File "c:\python27\lib\site-packages\github3.py-0.4-py2.7.egg\github3\repos.py", line 397, in create_issue
    json = self._json(self._post(url, data=dumps(issue)), 201)
  File "c:\python27\lib\site-packages\github3.py-0.4-py2.7.egg\github3\models.py", line 82, in _json
    raise GitHubError(response)
github3.models.GitHubError: 422 Validation Failed

Construct _api attribute on our own

We use the _api attribute to make constructing urls for objects. After a discussion today with GitHub support, it seems as if they'll be using the url parameter in the json objects for the same url with parameters in the future. This change was recently made (possibly accidentally) breaking what functionality I had relied on (although they fixed it once I reported it).

From now on, to keep with that functionality, I'll have to construct these urls myself (the form I was intending on relying upon).

Another option I have is to just strip off the parameters and save those elsewhere in case the API starts to require that.

Make models hashable

What about making the core models hashable? That'd allow to use them in sets or
as dictionary keys, which can be handy for some use cases.

It shouldn't be much effort, it occurs to me that we could use the entities'
class name (__qualname__ was introduced in Python 3) and its unique
identifier for constructing a hash. For example, in an issue we could add:

    def __hash__(self):
        return "{0}.{1}".format(self.__class__.__name__, self.id)

It's just an idea, let me know what you think. I'd be more than happy to help
out with this.

iter_repos is not correct

It needs to be split into iter_user_repos and iter_repos. Currently it accepts invalid parameters while accepting a user's login which gives the false sense that you can retrieve another user's private repositories.

http://developer.github.com/v3/repos/#list-user-repositories

Login via oauth token

It would be nice if we could pass an oauth token instead of username/password to github3.

github3/gists.py is missing 2 iter_ functions

I think all list_* functions should have iter_* counterparts, even if they don't require the paginated github API. For instance, in gists.py, I would add iter_files and iter_forks as below. If you agree, I'll create a pull request.

diff --git a/github3/gists.py b/github3/gists.py
index 412c01b..21b37ae 100644
--- a/github3/gists.py
+++ b/github3/gists.py
@@ -201,10 +201,19 @@ class Gist(GitHubCore):
         stored in this gist."""
         return self._files

+    def iter_files(self):
+        """List of :class:`GistFile <GistFile>` objects representing the files
+        stored in this gist."""
+        return iter(self._files)
+
     def list_forks(self):
         """List of :class:`Gist <Gist>`\ s representing forks of this gist."""
         return self._forks

+    def iter_forks(self):
+        """List of :class:`Gist <Gist>`\ s representing forks of this gist."""
+        return iter(self._forks)
+
     def refresh(self):
         """Updates this gist by getting the information from the API again.

Incomplete repos when '/user/repos' url is used!

Any repos derived from getting a list of repos via the /user/repos url, and then turning the resulting json list into repo objects will be incomplete relative to fetching this information via the url given by repo.url (/repos/:user/:name).

I came across this when I used 'list_repos', and noticed that none of the repo objects had 'parent' or 'source' attributes.

Maybe repo objects can be made to fetch more json data via the self._api url, and update themselves?
Maybe then missing data attributes could be properties that execute this updater method?

My own quick-fix is to replace the last 'return' line of list_repos in github.py with:

return [Repository(self._json(self._get(repo['url']), 200), self) for repo in json]

Add __eq__ methods to classes for ease of comparison

The problem here is determining which characteristic should define equality. One could have one object that is a Repository and later (elsewhere) make a separate one and update that one to change the name. Technically, it's the same repository, but there's no way to be entirely certain without checking the id. And if we check that alone, then it might lead to confusion where <Repository [sigmavirus24/github3.py]> == <Repository [sigmavirus24/some_other_name]>.

I've been debating this internally since I started the project. I figure it can't hurt to open it up to discussion now.

Upload wheels to PyPI too

I think this PEP was just accepted and that it's possible to do so. I remember someone asking Kenneth (Reitz) about the availability of one for requests 1.2.0 on twitter.

New Ratelimit is Killing The Tests

Looks like our only option will be to rewrite the tests to use mock. Before the ratelimit was fine (5000/hr unauthenticated) but now it's too low.

I guess this will make getting 100% easier.

Feature request: Support for new status API

It would be great if you could add support for the new status API:

http://developer.github.com/v3/repos/statuses/

We could start using this straight away for our bot.

Test report_issue.py again

Issue type: feature request

---

Traceback:

None

---

Description:
Actually make sure the fields are filled in

---

Generated with github3.py using the report_issue script

Contents decode fails on non-ascii files

I'm using the Contents class in various deployment scripts to grab certain SQL files from our repos and run them in MySQL. Our SQL files are stored in various encodings depending on the content, but most commonly UTF-8 or ISO-8859-1. By default, the Python decode() function uses the ascii encoding to decode the string, which is what github3.py uses (see https://github.com/sigmavirus24/github3.py/blob/develop/github3/repos.py#L1356). Unfortunately, this fails with a UnicodeDecodeError on our UTF-8 encoded scripts.

I'm doing something similar to https://github.com/WheresWardy/github3.py/blob/unicode-fix/github3/repos.py#L1358-1364 in order to fix this, but there may be a better solution you'd like to integrate.

missing dumps in authorize

Hi,
I was following the online docs to grab a token for myself when I ran into some trouble.

Github will respond with a 400 Problem parsing JSON
github3.models.GitHubError: 400 Problems parsing JSON

If the line is changed from:

                json = self._json(ses.post(url, data=data), 201)

to

                json = self._json(ses.post(url, data=dumps(data)), 201)

works fine.

Serialization of objects

Add functions to dump and read JSON about the object.

to_json would be a function only for use on an object. from_json would be a classmethod but I am beginning to think it is entirely unnecessary.

We can change the default behavior of a class like Repository to accept a GitHub object instead of a session variable as an optional parameter. This will allow session sharing and a clean way of instantiating a class from pure JSON.

@doda comments? concerns? opinions?

Test report_issue.py

Issue type: {0}

---

Traceback:

{1}

---

Description:
{2}

---

Generated with github3.py using the report_issue script

Document how to write tests

AttributeError: 'NoneType' object has no attribute 'encode'

File "/home/yac/.local/lib64/python2.7/site-packages/github3/models.py", line 321, in __init__
    self.name = acct.get('name', '').encode('utf-8')

These account attributes are optional and so may return None and then this code blows up on missing None.encode()

Generalize coverage_all target

Perhaps there should be a configuration file for what targets to use when testing with authentication? As of now, I'm the only person who can run the authenticated tests since I'm the only one with access to all of the repos and areas that require authentication. This would allow people to set up their own test environment to run coverage_all/coverage_auth.

list_authorizations / iter_authotizations cause GitHubError: 404 Not Found

Traceback (most recent call last):
  File "/usr/local/bin/git-hub", line 490, in <module>
    main()
  File "/usr/local/bin/git-hub", line 91, in main
    commands[command](opts)
  File "/usr/local/bin/git-hub", line 487, in apps
    print list(gh.iter_authorizations())
  File "/usr/local/lib/python2.7/dist-packages/github3/models.py", line 157, in _iter
    json = self._json(response, 200)
  File "/usr/local/lib/python2.7/dist-packages/github3/models.py", line 86, in _json
    raise GitHubError(request)
github3.models.GitHubError: 404 Not Found

Use tox

Design Tests for Repo methods

I need to figure out a way to design tests for the Repository methods:

• create_commit
• create_pull
• create_pull_from_issue
• create_tree
• merge

The first one will likely be the most difficult. I had the idea that a new repository could be created, then a new branch would be made. We could then create a commit on that branch and use that to test merge. Then we could fork, create an issue, create a commit and create a pull from the issue. Then use that to test merging via a pull request. Then finally create a tree and commit and create a plain pull. Then delete all the things. This will not be easy in the slightest and likely not possible like this at all.

I might have to pull some other people into this to help me design tests for these, but I want to hold off for now.

Subclass `json.JSONEncoder` for easier serialization

We could subclass the standard library's JSONEncoder class to provide an easy way to serialize objects to JSON. An example of the usage would look like this:

from github3 import GitHubEncoder, user
from json import dump

u = user('sigmavirus24')
with open('fixtures/sigmavirus24', 'wb+') as fd:
    dump(u, fd, cls=GitHubEncoder)

This is far more elegant (in my opinion) than using to_json which lazily just stores a copy of the object received from the API. This means that while we parse the object we could use pop on the dictionary instead of get and reconstruct it on the fly with the custom class. I'd also rather have a better name than GitHubEncoder.

Include tests in distribution

This is for inclusion in Debian and other package repositories. Besides it is just good practice, even if others don't do it.

Tests don't run on clean checkout

STR:

• Clone repo

    git checkout develop

    python setup.py test

    python  run_tests.py

    Traceback (most recent call last):
      File "run_tests.py", line 28, in <module>
        suite = unittest.defaultTestLoader.loadTestsFromNames(names)
     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
       suites = [self.loadTestsFromName(name, module) for name in names]
     File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/loader.py", line 100, in loadTestsFromName
       parent, obj = obj, getattr(obj, part)
    AttributeError: 'module' object has no attribute 'test_api'

Deprecate __str__ for Contents

NOTE: This won't happen in 0.7 but will happen in 0.8 to give people time to hear about it.

Everything we receive from requests is in the form of bytes (even, technically, on python 2). This is beneficial in the sense that all encodings fit into bytes strings, but causes issues when trying to return bytes from __str__, which I should have thought about before implementing. With that said, and given that this breaks python 3.x, I'm going to deprecate this functionality.

For the record, I also considered using bytes.encode on python 3.x only, but the problem then occurs when there are encodings which can not fit into UTF-8 and the fact that we can not predict what the encoding will be.

Pagination of certain functions

So currently, when someone uses a method like list_issues() they only get
the first "page" of issues as returned by the API (provided the parameters
they decide to use). The next "page" can be accessed by appending page=2
or by accessing the link headers of the response object and using rel_next
but not through the list_issues() method.

The first option that came to mind was accepting another parameter to
list_issues(): page. page would accept any of the following:
rel_next, rel_prev, rel_first, rel_last, and <int>. The
problem then occurs, of course when someone does something like

from github3 import repository

repo = repository(owner, repo_name)
issues = repo.list_issues(page='rel_next')

There is no current set of link headers for this so rel_next would not
produce obvious results. To try and guess what the user meant would be
ridiculous, so the obvious solution would be to just return the first page. If
they give us an integer, then clearly they just want that one page and we add
that as a parameter (easy).

The next option that came to mind was to change list_issues() to list
all issues on a repository and to introduce a function iter_issues()
which yields each page as a list of Issues. The problem with this then
occurs that you can not request a specific page unless we use yet another
function.

So right now I think the first option would be the least confusing of the two
options but with an extra acceptable value for page -- "all". Any and all
input is very welcome.

Incorporate new ways to list issues

github/developer.github.com@22a13da

Correct params in commit commenting

github/developer.github.com@3a1fbd5

Add links to GitHub's API documentation where possible

re: #62

Repository.create_download does not work

Installation fails without requests installed