Giter Site home page Giter Site logo

vifactor / repostat Goto Github PK

View Code? Open in Web Editor NEW
99.0 5.0 12.0 897 KB

Inspired by gitstats project: git repository desktop analyzer

License: GNU General Public License v3.0

Python 74.56% CSS 8.23% JavaScript 8.43% HTML 8.78%
pygit2 gitstats git pandas html-report analytics statistics repository

repostat's Introduction

Python3-compatible Git repository analyser and HTML-report generator with nvd3 -driven interactive metrics visualisations.

Initially, a fork of gitstats tool.


Check how a repostat's report looks like by going to:

https://repostat.imfast.io/

Installation

Starting from v2.0.0, repostat is installable from PyPi under the name repostat-app. Installation should be as simple as:

pip3 install repostat-app

Newest and older versions

  • To install a development version with newest changes from repostat's github repository, the following command may be executed:

    sudo pip3 install git+https://github.com/vifactor/repostat

    This command installs repostat from HEAD of master branch.

  • To install repostat at specific tag or branch, use the following syntax

    sudo pip3 install git+https://github.com/vifactor/repostat@<branch|tag>

NOTE: Versions prior to v2.0.0 have additional system-dependencies, e.g. gnuplot.

OS-specific requirements

Linux installation

Repostat for Ubuntu 20.04

python3-pip must be in the system and then installation via pip works fine.

Mac OS (Catalina) installation

Repostat for Mac OS

Prior to installing repostat one needs to make sure to have right version of libgit2 in the system. This can be achieved

$ brew update
$ brew install libgit2

Then, install repostat via:

$ pip3 install repostat-app

NOTE:

  1. Homebrew-way to install packages is slow and may break system dependencies.
  2. repostat's CI for OSX builds libgit2 from source.

Windows installation

Repostat for Windows

Once there is python v3.6+ in the system path, repostat can be installed via:

python -m pip install repostat-app

NOTE: On Windows 10+, symlink to general.html is not generated, when repostat launched by an unprivileged user.


Usage

repostat [--help] [--version] [--config_file CONFIG_FILE_PATH]
                 git_repository_path report_output_path

Run repostat --help for details.

Configuration file

A report can be customized using a JSON settings file. The file is passed using the --config-file option as follows:

repostat --config-file <path_to_config.json> <repo_path> <out_path>

Configuration file might contain following fields (all are optional):

{
    "max_domains": 10,
    "max_authors": 7,
    "max_plot_authors_count": 10,
    "max_authors_of_months": 6,
    "authors_top": 5,
    "colormap": "classic",
    "max_recent_tags": -1,
    "orphaned_extension_count": 2,
    "time_sampling": "W"
}

Detailed information about role of the fields is below.

Authors page configuration

These values are usually adjusted to accommodate projects with various number of contributors and activity levels, to avoid showing too much or too little information.

  • max_domains: number of e-mail domains to show in author stats
  • max_authors: number of authors in the "top authors" table (other authors are listed without detailed stats)
  • max_plot_authors_count: number of authors to include in plots in "Authors"-page (rest of the authors will be grouped as "Others").
  • max_authors_of_months: number of months for which "author of the month" should be displayed
  • authors_top: number of authors to show for each month/year in the author of month/year list
  • orphaned_extension_count: max file extension count to be considered as orphaned and displayed in report in the corresponding category (default: 0, i.e. all extensions are displayed)

Colorscheme configuration

The colors of the thread "heat maps" tables in the activity page can be customized using the "colormap" option. The allowed values are:

History plots sampling

is controlled by "time_sampling" field in configuration file and defines how timeseries , e.g. number of files over a repository history, are sampled. By default, weekly-sampling is used. For old repositories one might want to increase that value to month or even quarter. Accepted values for "time_sampling" are the Pandas' Offset aliases

Tags rendering

Some git repositories contain thousands of tags most of which are not worth to check. Since v.1.3.0 there is a possibility to limit the number of tags displayed in "Tags" tab of the HTML report or even hide the tab.

The feature is controlled by "max_recent_tags" field

If JSON file has following content { [...], "max_recent_tags": 8 }, the report will contain the 8 most recent tags in "Tags" page. Setting the field max_recent_tags to zero will not render "Tags" page at all. If no such field is provided in JSON settings, the report will contain a "Tags" page with all tags in the analysed repository.

Additional features

Mailmap

Starting from v1.1.2+ repostat supports git mailmap. Two things are required in order to make the feature working:

  • have pygit2 v.0.28+ installed
  • create and fill .mailmap file (e.g. in the root of your repository)

Relocatable reports

By default, images, css- and js-files required for html report rendering do not get copied to a report directory. Html pages contain absolute paths to assets located in repostat's package installation directory.

Starting from v.1.0.x, the --copy-assets command-line option forces program to copy assets to generated report and embed relative paths in the generated html-files.

How to contribute

Bug reports and feature requests as well as pull requests are welcome. Please, check the "Issues" on github to find something you would like to work on.

repostat's People

Contributors

ari3s avatar beastd avatar beregabor avatar chilikink avatar digeomel avatar doudou avatar flatcap avatar hoxu avatar janihur avatar janpieper avatar jonespm avatar karl82 avatar mattparlane avatar mikelolasagasti avatar moy avatar neilfloow avatar penberg avatar philantrop avatar pulkomandy avatar sjoubert avatar skuschel avatar stezz avatar svenvh avatar tnielsen avatar tobix avatar tomk32 avatar tony2guo avatar vifactor avatar xsgordon avatar zsx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

repostat's Issues

Failed tests

unit2 test reports the following failures (Python 2.7.15 was used):

======================================================================
FAIL: test_changes_history (test.TestPygitMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jberan/git/repostat/test.py", line 579, in test_changes_history
    t, expected_record, self.gs.changes_history[t]))
AssertionError: 1186502755: {u'files': 1, u'lines': 818, u'del': 0, u'ins': 0} vs. {u'files': 2, u'lines': 818, u'del': 141, u'ins': 141}

======================================================================
FAIL: test_rev_list (test.TestPygitMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jberan/git/repostat/test.py", line 603, in test_rev_list
    self.assertListEqual(expected_data, actual_data)
AssertionError: Lists differ: [(1539032340L, '1e80d967ac61c7... != [(1539032340, '1e80d967ac61c7f...

First differing element 181:
(1261815612L, 'b1dd28ed04df03bc06206bededc46b70cc57afa8')
(1261644398, 'a61bae76cbd8c8e76bc486110d21240fd203bf1c')

Diff is 50476 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------
Ran 15 tests in 14.476s

FAILED (failures=4)

Performance of pygit2 is poor

Repository.diff works 5-10 times slower than parsing of a text output of a git log command.

Although, v.0.27 works slightly better in comparison to v.0.24, still its speed is much worse than git subprocess invokation.

Contribution by author plot

Constribution as a percentage of lines written by author and got from 'git blame'
Remove the info from table and plot pie-chart diagram

Failure when submodule is added to the project

Traceback (most recent call last):
File "/home/viktor/PycharmProjects/repostat/gitstats", line 745, in
g.run(sys.argv[1:])
File "/home/viktor/PycharmProjects/repostat/gitstats", line 729, in run
report.create(data, outputpath)
File "/home/viktor/PycharmProjects/repostat/gitstats", line 407, in create
about_html = self.render_about_page()
File "/home/viktor/PycharmProjects/repostat/gitstats", line 631, in render_about_page
"contributors": [author.decode('utf-8') for author in get_project_contributors()]
File "/home/viktor/PycharmProjects/repostat/gitstats", line 93, in get_project_contributors
contributors_list = fetch_contributors_from(repo, head_commit)
File "/home/viktor/PycharmProjects/repostat/tools/versioning.py", line 13, in fetch_contributors_from
blob_blame = repo.blame(p.delta.new_file.path)
File "/home/viktor/PycharmProjects/repostat/venv/local/lib/python2.7/site-packages/pygit2/repository.py", line 480, in blame
check_error(err)
File "/home/viktor/PycharmProjects/repostat/venv/local/lib/python2.7/site-packages/pygit2/errors.py", line 49, in check_error
raise ValueError(message)
ValueError: object at path 'git-pandas' is not of the asked-for type 3

Upload on PyPi

This became possible after #5 is closed and repostat became a pure Python

CSV Output feature

Implement new CSV Output feature. CSV files can used to load data any data analyse or visualizer system.

new cli option: output valid values: html, csv
default is 'html'

with html value generate the html riport as the current main feature.
with csv value generate some csv output: authors, commits details... and so on.

Project name and repo name have to be in csv files.

Non-local browse is broken

With #65 two use cases seem broken now:

  1. The generated report can't be served by an HTML server anymore since the repostat installation dir is likely not in correct folders allowed by the server. For example, aggregating the report as Jenkins artifacts now only serves the html. Jenkins refuses to provided the css and js from the repostat installation dir.
  2. The generated report is not relocatable anymore. For example, generating reports on a build machine then aggregating multiple reports on a single host dedicated to stats requires repostat to be installed on the end machine (and in the same place as the build hosts)

I don't know what were the reason for removing the assets from the report (reducing space usage I guess, but the assets size is not that big), but having standalone relocatable reports should be a valid use case in my opinion.

Catch uncaught exception

File "./gitstats", line 517, in create
tags_html = self.render_tags_page(data)
File "./gitstats", line 798, in render_tags_page
self.authors_by_commits = getkeyssortedbyvalues(data.tags[tag]['authors'])
KeyError: 'authors'

Fix debian lintian complains while depackaging

Now running lintian repostat_0.95-1_amd64.changes ...
E: repostat changes: changed-by-address-malformed Viktor Kopp vifactor@Lenovo-IdeaPad-Y580
W: repostat source: package-needs-versioned-debhelper-build-depends 10
W: repostat source: no-debian-copyright
W: repostat source: ancient-standards-version 3.9.6 (released 2014-09-17) (current is 4.1.4)
E: repostat: debian-changelog-file-contains-invalid-email-address vifactor@Lenovo-IdeaPad-Y580
E: repostat: no-copyright-file
E: repostat: extended-description-is-empty
W: repostat: script-with-language-extension usr/bin/export_repos.py
W: repostat: script-with-language-extension usr/bin/gitstats.py
W: repostat: binary-without-manpage usr/bin/export_repos.py
W: repostat: binary-without-manpage usr/bin/gitstats.py

Restore cache functionality

Store fetched data in a easily readable format and next time when repo is about to be analyzed fetch only updated git objects.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.