tarmstrong/git2json

Convert git logs to JSON for easy analysis

git2json's Issues

Its not possible to pass other arguments to the log

I don't always want the json log for the whole tree or for a particular date but between 2 sha1's or 2 tags

git2json --git-dir ....   V1_0_TAG...HEAD
git2json --git-dir ....   V1_0_TAG...V1_1_TAG

or sometimes you might just want to log 1 specific directory

git2json --git-dir ....    src/bin/gtest
git2json --git-dir ....    .

Could this be added? maybe

git2json  --revisions=V1_0_TAG...HEAD
git2json  --srcdir=.

I realize you needing to control the arguments to git log, but perhaps allow passing additional arguments

git2json --args="V1_0_TAG...HEAD ."

bash shorthands like `~` don't work with --git-dir argument

I was playing around with http://nbviewer.ipython.org/urls/raw.github.com/tarmstrong/code-analysis/master/IPythonReviewTime.ipynb and noticed two issues:

• The --git-dir expects the .git dir, but the error messages suggest it is looking for one directory up. See #5.
• It does not support paths with ~

Hidden files (those starting with a '.') parsed incorrectly

For example:

[
    {
        "author": {
            "date": 1383882701, 
            "email": "[email protected]", 
            "name": "Tavish Armstrong", 
            "timezone": "-0500"
        }, 
        "changes": [
            [
                0, 
                1, 
                "travis.yml"
            ]
        ],

The problem is that the period is being sliced from the beginning of .travis.yml.

Since not working?

git2json --since 2018-01-01
usage: git2json [-h] [--git-dir GIT_DIR]
git2json: error: unrecognized arguments: --since 2018-01-01

Also other date formats and "this year" (which git log supports) don't work. -h also doesn't mention since. Installed today via pip, maybe that version is outdated?

git2json crashes in rare case parse_message_line returns None

OS - Linux crunchbang 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux
Python - 2.7.2

I went to take git2json for a spin on a repository with a long commit log to see how it would work.

I chose the Rails repository, which contains the following commit:

commit e04818d1b8de44d47003031791f142182320ed89
tree 3f56c6c53f914a2ed882e32a8a3438be619f8f17
parent 6cbec586731b7c5f293f2a7b6c7330c7181acdfe
author Jeremy Kemper <[email protected]> 1348081584 -0700
committer Jeremy Kemper <[email protected]> 1348081584 -0700

    Revert "Make sure :environment task is executed before db:schema:load or db:structure:load
"

    Breaks db:setup because it tries to load the environment before creating the database.

    This reverts commit 5ca11fefce6d83f5db399aa4412f1f1a0d42b2e6.

2   2   activerecord/lib/active_record/railties/databases.rake

I was unable to determine why, but, for some reason, there is a carriage return character between load and ".

So calling splitlines() on

'''    Revert "Make sure :environment task is executed before db:schema:load or db:structure:load
"
    Yip yip yip
''' 

we get:

['    Revert "Make sure :environment task is executed before db:schema:load or db:structure:load', '"', '    Yip yip yip']

But if you pass parse_message_line a string like '"', it will return None, and join cannot handle lists with None in them.

>>> l = ['fee', 'foo', 'fi', None] 
>>> "\n".join(l)
TypeError: sequence item 3: expected string, NoneType found

[enhancement] Use pygit2 over scraping git command output

Python bindings for Git: https://www.pygit2.org/

opening a repository
parsing ids
revwalk in C | in python
commits
commit author
tree-to-tree diffs

TypeError: sequence item 1: expected str instance, NoneType found

$ git clone https://github.com/fabioparra/TypeScript.git; cd TypeScript
$ git checkout bc691ef78ec
$ ipython3
from git2json.parser import parse_commits
from git2json import run_git_log
list(parse_commits(run_git_log('.git')))

TypeError                                 Traceback (most recent call last)
<ipython-input-4-a4fe5c202781> in <module>()
----> 1 list(parse_commits(run_git_log('.git')))

~/.virtualenvs/bayesian/lib/python3.6/site-packages/git2json/parser.py in parse_commits(data)
     47         full_commit = rc.groups()[0]
     48         parts = RE_COMMIT.match(full_commit).groupdict()
---> 49         parsed_commit = parse_commit(parts)
     50         yield parsed_commit
     51 

~/.virtualenvs/bayesian/lib/python3.6/site-packages/git2json/parser.py in parse_commit(parts)
     71         parse_message_line(msgline)
     72         for msgline in
---> 73         parts['message'].splitlines()
     74     )
     75     commit['changes'] = [

TypeError: sequence item 1: expected str instance, NoneType found

Problem with lines.iter() on python 3 on Windows

@stonebig wrote in issue #1 :

oups ! looking further, I see my windows python 3 suffering on lines.iter()
it seems you may have to write

    if sys.version_info < (3, 0):
        line = lines.next()
    else:
        line = next(lines)

Better error messages when invalid git-dir specified

If I run git2json in my home directory, this happens:

tavish@temeraire:~ $ git2json
fatal: Not a git repository (or any parent up to mount parent /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
[null]

If I run it with --git-dir specified, but not to the .git folder, I get this:

tavish@temeraire:~ $ git2json --git-dir=code/git2json
fatal: Not a git repository: 'code/git2json'
[null]

These should 1) print a good error message 2) have a non-zero exit status 3) not output anything to stdout.

Originally reported in #2