The fetchBot wastes a lot of resources on crawling giant old not-so-popular repos

[lourot]

The fetchBot wastes a lot of resources on crawling giant old not-so-popular repos

[brillout]

We can skip the "commit days" idea for now, it's not that big of an improvement. This means we can skip fetching the commits of other users.

For the time badge we only need the first and last user commit. We don't need the user commits in the middle.

For the user contribs graph we only need the user commits, not all commits. If the user has too many commits then maybe we could linearly extrapolate between the first user commit and the last user commit.

Also if we skip the commit days idea then we can use endpoints like stats/contributors that give coarser info. (This one doesn't seem to have an option to filter by author though, so it's not that helpful for us. But maybe there are some GraphQL endpoints that would help.)

From my side, the only feature that would sadden me if we don't implement it, is the user contribs graph. The rest I can live fine without.

But anyways, we can always work on something else. There are many things we can do without extra data. (E.g. hall of fame, medals summary like on military vests)

So to sum up: Can we get the date of the first user commit and the date of the last user commit? (I checked on api v3 and it seems that we cannot invert the sorting order. But I vaguely remember that it's possible on the graphql api. Didn't check that out yet.)

[lourot]

@brillout FYI here is how a "commit list file" looks like when it's truncated: https://s3.amazonaws.com/ghuser/data/repoCommits/torvalds/linux.json

{
  "contributors": {
    "torvalds": {
      "2018-09-16": 2,
      "2018-09-15": 7,
...
    }
  },
  "last_fetched_commit": {
    "sha": "7876320f88802b22d4e2daf7eb027dd14175a0f8",
    "date": "2018-09-16T18:52:37Z"
  },
  "ghuser_truncated": true
}