GSoC idea: Reporting of CHAOSS Metrics

[chaoss]

GSoC idea: Reporting of CHAOSS Metrics

[germonprez]

@jsdabrowski @apoorvkhare07 @f2014169 you can see the message at:

https://lists.linuxfoundation.org/pipermail/oss-health-metrics/2018-February/000517.html

[jgbarah]

@apoorvkhare07 said:

I am still not clear about how many repositories to consider ? How many repositories should be stored in the indexes ? Can it be all the repositories of a particular organisation/user ?

You should consider any number of repositories: those that are referenced in the enriched index.

In other words: for these microtasks, you may assume that you already have an enriched index available for analysis. Your script should work with any properly formed index (for the relevant kind, eg, git, github, etc.). Of course, for testing, you need to produce your own index, with as many repos as you may want. The more testing you do the better, so try to ensure your script works with indexes with extreme cases, such as a single repository, several repositories of the same organization, several repos of several organizations, etc.

[jgbarah]

@aswanipranjal said in chaoss/grimoirelab-tutorial#19:

I had some questions about the project Reporting of CHAOSS metrics:
To extend the functionality of Manuscripts, we will need to add code to calculate the Metrics listed down at https://github.com/chaoss/metrics, right? If so, how do you suggest that the user demographics be collected and monitored?

We may need too add code, or to reuse some code and write the rest from scratch. We welcome your ideas and proposals on how to tackle that, based on your understanding of the code.

With respect to specific metrics, please have in mind that not all of them can be retrieved automatically from software development repositories. Some of them could be, for example, collected manually, and written into some properly formatted YAML file, which Manuscripts would read to produce the relevant metrics. But again, we're open to your ideas.

In the 3rd Microtask for the project, it is said that: Produce a listing of repositories, as a table and as CSV file, with the number of commits authored, issues opened, and pull requests opened, during the last three months, ordered by the total number (commits plus issues plus pull requests). Does that mean all the public repositories created in the last 3 months? Or does that mean that all the repositories should be listed according to the number of commits authored + pull requests made + issues opened made in the last 3 months? If the answer is the latter one: then how many repositories should be listed out?

It is the latter, and the number is all of them. Please remember that the idea is that this should run on GrimoireLab indexes, which means the real answer is "all repositories in the indexes considered". In other words, the code should just look at the indexes, find all the repositories in them, and produce numbers with their activity during the last three months (including those with zero activity).

Is it more clear now?

[germonprez]

@aswanipranjal, I would focus on a clean completion of the microtask first and foremost. I'll see what others say but I can't guarantee feedback on a proposal. I also believe that if we make the offer to one, we will need to make the offer to all -- which could be very tricky from a timing perspective.

[jgbarah]

@aswanipranjal I agree with @germonprez in his previous comment. This said, I'm going to try to comment on any specific question about the proposal, as long as it is public. This will ensure that all applicants have the same information. In particular, I understand that the part more difficult for the application may be the description of what do you want to do, and the approach for that, so I will try to be specially supportive about that.

I suggest that we use this issue for all of this, or if needed, a separate one for specific stuff (but please, mention this issue in it, so that we can have a pointer).

[jgbarah]

@aswanipranjal thanks for your comment! Yes, it is as you say. In fact, it will not only something that will help us to decide, but also you to know about the project, and about your own skills in this area.

[apoorvkhare07]

@jgbarah How to update you about the repository ?
I have almost done the first microtask.

[jgbarah]

@apoorvkhare07 we're about to publish a message in the mailing list about that. Please, stay tuned!

[apoorvkhare07]

@f2014169 Not yet.

[f2014169]

@gopal86 please visit https://lists.linuxfoundation.org/pipermail/oss-health-metrics/2018-February/000517.html

[jgbarah]

@kmn5409 I'm not sure what do you mean. If that is if you can run Elasticsearch from a bash (or a powershell) in Windows, I assume you can, although I'm not familiar with Windows, sorry. If that is something else, please let me know.

However, for clarifying, almost all the development for GrimoireLab is done in Linux-based systems. That means that using Linux for development is certainly more friendly, and we can provide much more support.

[kmn5409]

@jgbarah Thank you for the response I tried using the Bash on Windows 10 but I ran into a number of difficulties so I instead decided to have a dual OS (Ubuntu and Windows) so I'm happy to know you all would be able to provide help with Ubuntu

[jgbarah]

@kmn5409 said:

I have read the Micro-task information and I'm at the first one "Produce a listing of the number of new committers per month" what Github repository is being specified here?

As @aswanipranjal commented, any repository could be analyzed. Have the link to the repository as a parameter for the script.

[apoorvkhare07]

@kmn5409 Yes while doing the microtask you'll use your own github token to produce the desired output (csv or the plot) and while pushing it on github you can remove it.

[aswanipranjal]

@kmn5409, yes, while solving you'd have to use your own personal token. You should remove the token from the notebook when pushing the notebook to GitHub. As stated above:

Please remember removing passwords and/or auth tokens that could be in them, before uploading

[jgbarah]

Thanks @apoorvkhare07, @aswanipranjal! Yes, that's it. In fact, @kmn5409 the recommendation about removing the token is a very good one for whatever credentials you have in your files: remember to remove them before uploading to any public site.

[Deepayan-Ghosh]

@jgbarah, @apoorvkhare07 Yes it was a version problem. Switching to the latest versions solved the problem. Thanks for all the help.

[valeriocos]

@kmn5409, I'm not able to replicate your issue. Do you have the latest version of kidash, ELK?
Is the version of your ElasticSearch aligned with the one of Kibana?

[apsknight]

Hey @jgbarah

I'd like to work on this project for Google Summer of Code 2018.
Please help me getting started.

[jgbarah]

Hi, @apsknight have a look at the description of this issue, and let me know if you need further clarifications.

[vishnoitanuj]

Hi @jgbarah
I had worked on projects related metrics and was trying to gain an insight at GrimoireLab. I am not much aware of the Elasticsearch databases. So can you suggest some good resources for its training, so that i can move forward with the idea.
Also i am facing issues with understanding Mordred.

[jgbarah]

Hi @vishnoitanuj. For Elasticsearch, I've added in the description of this issue two refernces, to the most usual Python packages to access it. I would recoment going through them, and maybe some tutorial for starters on Elasticsearch.

WRT Mordred, please let me know what issues you're facing, I will try to produce documentation to help you address them, if I can.

[jsdabrowski]

Hi @jgbarah , is it suitable for you if I create my private repo with the work and share it with you and your colleagues once I finished? Sorry if the question is trivial, but I was a bit confused by saying "open a GitHub repository, and upload to it:"

Best,
Jacek

[gopal86]

Hello @jgbarah , i am quite interested in contributing in this project . Can you guide me through ?

[jgbarah]

@jsdabrowski: the idea is that you open a public repository, and work in it. We will publish soon how to inform us about the url of the repository.

[jgbarah]

@gopal86, did you have a look at the microtasks and guidelines in the description of this issue?

[sreetamparida]

I need to ask that to work on this project for Google summer of code, I need to compete those microtask provided??

[aswanipranjal]

Hey @sreetamparida, these tasks are for you to understand the libraries and how to analyse repositories using GrimoireLab tools and ELasticsearch. You can also complete these tasks to show that you are capable and comfortable in working with the above listed technologies.
As written above:

Of course, there is no need to do all the microtasks, you only need to show that your skills are in good standing for working in this project.

[jgbarah]

@sreetamparida we need to evaluate candidates. If we have people fulfilling the microtasks, that will help us a lot to know about their skills.

[f2014169]

@jgbarah Was the message published? , because I joined the mailing list recently.

[jsdabrowski]

@jgbarah , could you please include in the mail some information related to a project proposal (what do you expect from it, and what should be included) ?

[Deepayan-Ghosh]

@jgbarah I am interested in this project. I have installed the necessary tools and have gone through the tutorials and also ran some code. However I had one question in my mind, which may be trivial. The question is:
Are both raw and enriched indices the same for elasticsearch? That is, whether both of them appear as just indexes for elasticsearch, and whether the enriched index is something that is made by ELK exclusively just to facilitate making dashboards?
Isn't there any programmatic way of creating and uploading the enriched index to elasticsearch, instead of running the p2o.py script?

Thank you

[gopal86]

hi @jgbarah , @germonprez . I have completed the microtask-1 . can you guide as to what to do next ?

[f2014169]

@jgbarah do you have any specific format for proposal??

[jgbarah]

@jgbarah do you have any specific format for proposal??

Not for now. But let me know if the elements that should be included in it are not clear.

[jgbarah]

Are both raw and enriched indices the same for elasticsearch? That is, whether both of them appear as just indexes for elasticsearch, and whether the enriched index is something that is made by ELK exclusively just to facilitate making dashboards?

The raw index is intended to store data as similar as possible to the original data source. Raw indexes are usually rich JSON documents, with all the details obtained from the service API or similar.

The enriched index is produced from the raw index, for being visualized with Kibiter/Kibana. It is usually a flat JSON document (because nested documents have some trouble with Kibana), which is in part a summary of the documents in the raw index (because only those properties relevant for visualizing are included), and the result of some computing for some parameters (eg, time to close a ticket, which is not directly available from the API, but has to be computed from the time the ticket was opened, to the time it was closed).

Isn't there any programmatic way of creating and uploading the enriched index to elasticsearch, instead of running the p2o.py script?

Yes. p2o.py is just a Python script with uses the grimoire_elk module to fetch data (using Perceval), enrich it, and upload it to Elasticsearch.

[jsdabrowski]

@jgbarah what would you like to be Included in the proposal? Would you like a candidate to propose how to approach the project? If so, then more detail are needed. Otherwise, cover letter is enough?

[kmn5409]

Just wondering would you be able to use Linux Bash Shell on Windows 10 for the things that will be installed for example Elasticsearch. Or would you use something like a virtual machine like Oracle VM?

[jgbarah]

@jsdabrowski the proposal could be as much detailed as you feel convenient. If you have doubts on the scope, or on what is asked for, that's my fault: let me know what you're missing, and we'll complete the description of this issue as much as needed. If you're doubting between different approaches, that's no problem: either explain the alternatives you see, or just select one and expose why you think it is better. That does not mean we're going to decide to go that way, but it is going to tell us about how you deal with selecting choices...

But again, if you feel you need more info, let us know which info you are missing, please.

[apoorvkhare07]

@jgbarah I am still not clear about how many repositories to consider ?
How many repositories should be stored in the indexes ?
Can it be all the repositories of a particular organisation/user ?

[kmn5409]

@jgbarah I have read the Micro-task information and I'm at the first one "Produce a listing of the number of new committers per month" what Github repository is being specified here?

[aswanipranjal]

@jgbarah, @germonprez, @jcabot: Where should we submit the proposal for a review? Here or in the mailing list?

[aswanipranjal]

@kmn5409: Any repository can be analysed for that Microtask.

[germonprez]

@aswanipranjal you will submit the proposal to the GSoC site.

[aswanipranjal]

@germonprez, what i meant was: Can we get the proposal reviewed by any one of you before we submit it to the GSoC website? So that we can get some pointers as to where the proposal is weak and what you expect it should cover.

[kmn5409]

I have a question about Microtask #2 is it that in solving it you would use your own token from github? And when posting what you did you would instruct to put your own personal token?

[Deepayan-Ghosh]

@jgbarah Hello, I am trying to use kibana to visualise data. I have installed kibana and when I go to http://localhost:5601, kibana shows up. However, when I try to visualise I am told to create an index pattern, which I do. I also see a warning on the left side stating that there is no default index pattern. So I set the new index pattern as default. However, as soon as I click on visualise tab on the menu, it disappears and I am shown the same screen and warning about creating a new index pattern. Any help on this?
I followed this: https://grimoirelab.gitbooks.io/tutorial/grimoireelk/a-simple-dashboard.html and downloaded the git-dashboard.json file, moved it to /tmp and tried to visualise it, when this error occured.

[apoorvkhare07]

@Deepayan-Ghosh It seems to be due to the couchbase template installed in elasticsearch, which might be interfering with the kibana config stored in the .kibana elasticsearch index. The 'mark as default' button in kibana would not work and the kibana config stored in elasticsearch will appear to be empty.
Uninstalling the couchbase template or modifying it so it only matches my couchbase indices (e.g. changing "template": "", to "template": "cb-",) in couchbase_template.json might resolve this issue.

you need to apply the changes by using:
curl -XPUT http://localhost:9200/_template/couchbase -d @/usr/share/elasticsearch/plugins/transport-couchbase/couchbase_template.json
and finally,
You need to delete the .kibana index (or remove the section added by Couchbase template):
curl -XDELETE http://localhost:9200/.kibana

[jgbarah]

@apoorvkhare07, @Deepayan-Ghosh: Which version of Kibana are you using? Indeed, that could be the problem.

[apoorvkhare07]

@jgbarah can I get documentation for the Manuscripts projects somewhere ?
and can you tell more about the project's aim and demands ?

[valeriocos]

@apoorvkhare07 wrt the documentation you can find something at: https://grimoirelab.gitbooks.io/tutorial/content/manuscripts/first-report.html.
Hope this helps

[Deepayan-Ghosh]

@jgbarah I was following this tutorial: https://grimoirelab.gitbooks.io/tutorial/mordred/a-grimoirelab-dashboard-in-one-step.html . Downloaded the mordred-grimoirelab.cfg file, menu.yaml, and project.json files. I also cloned the directory containing the panels but when I run mordred -c mordred-grimoirelab.cfg I get the following error:

Kibiter/Kibana: version found is 6.2.2
Kibiter/Kibana: configured!
Panels, visualizations: uploading...
Panels, visualziations: uploaded!
Kibiter/Kibana: uploading dashboard menu...
Kibiter/Kibana: uploaded dashboard menu!
Elasticsearch aliases: creating...
URL= http://127.0.0.1:9200/_aliases
2018-03-18 03:42:51,223 - mordred.task_manager - ERROR - Exception in Task Manager 400 Client Error: Bad Request for url: http://127.0.0.1:9200/_aliases
Exception in thread git:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_manager.py", line 92, in run
    task.execute()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_enrich.py", line 215, in execute
    self.__enrich_items()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_enrich.py", line 140, in __enrich_items
    task_aliases.execute()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_panels.py", line 417, in execute
    self.__create_aliases()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_panels.py", line 409, in __create_aliases
    self.__create_alias(es_enrich_url, index_enrich, alias)
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_panels.py", line 380, in __create_alias
    res.raise_for_status()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://127.0.0.1:9200/_aliases

Why am I getting such error? When I visit http://127.0.0.1:9200/_aliases in my browser I get the following json:

{
   "idx_comb":{
      "aliases":{

      }
   },
   "git_en_repo-plugins":{
      "aliases":{

      }
   },
   "github_toga_raw":{
      "aliases":{

      }
   },
   "github_grimoirelab-perceval_raw":{
      "aliases":{

      }
   },
   "git_grimoirelab-elk":{
      "aliases":{

      }
   },
   "github_raw_repo-plugins":{
      "aliases":{

      }
   },
   "idx_comb_raw":{
      "aliases":{

      }
   },
   "git_grimoirelab-perceval_raw":{
      "aliases":{

      }
   },
   "github_grimoirelab-elk_raw":{
      "aliases":{

      }
   },
   "github_voc_raw":{
      "aliases":{

      }
   },
   "git_raw":{
      "aliases":{

      }
   },
   ".kibana":{
      "aliases":{

      }
   },
   "git_voc":{
      "aliases":{

      }
   },
   "git_grimoirelab-perceval":{
      "aliases":{

      }
   },
   .................................................................
}

Also if I change post to get at https://github.com/chaoss/grimoirelab-mordred/blob/master/mordred/task_panels.py#L361 then I do not get the error.

[Deepayan-Ghosh]

@jgbarah This is related to the proposal. The goal of the project is to add support to manuscripts to produce reports of any repositories based on the metrics in https://github.com/chaoss/metrics. For example, now the pdf report that is generated has sections on: "Overview","Activity","Community" and "Process", however we want to add support for the metrics provided in https://github.com/chaoss/metrics like Diversity-Inclusion, Risk, etc and for each of these categories, analyse the indices, obtain relevant information and produce representations of metrics like "Contributor demographics" for Diversity-Inclusion category, and so on. Is my understanding correct??

[kmn5409]

I'm not sure but when I try to create the Kibana dashboard I'm getting the error
$ kidash --elastic_url-enrich http://localhost:9200 --import /tmp/git-dashboard.json 2018-03-19 09:29:59,907 Panel not found (not in directory, no panels module): /tmp/git-dashboard.json 2018-03-19 09:29:59,907 Wrong file format (can't find 'dashboard' field): /tmp/git-dashboard.json
Now I might be putting the tmp folder in the wrong place, where is it supposed to go exactly, I have placed it in a few places in my (perceval) virtual environment. Also I tried looking online and I saw that I could just have the .json file in the same folder. I used this command
$ kidash --elastic_url-enrich http://localhost:9200 \ --import git-dashboard.json usage: usage: kidash [options] kidash: error: unrecognized arguments: --import git-dashboard.json
But then I get that error.
The directory I'm in looks like
$ ls git-dashboard.json perceval_github_1.py tmp token.txt

[valeriocos]

Hi @kmn5409,
A similar problem has been reported and successfully addressed in #71 (look for /tmp/git-dashboard.json in the comments).
Let me know if this helps you to solve your problem.

[kmn5409]

@valeriocos Yeah so this was what I had used to try and fix my problem but I still kept getting other issues

[valeriocos]

@kmn5409, can you tell more about the other issues (or the other issues are the ones above?)

[kmn5409]

So @valeriocos I'm trying to follow the tutorial for https://grimoirelab.gitbooks.io/tutorial/grimoireelk/a-simple-dashboard.html and when I try to create the dashboard I get the issue
$ kidash --elastic_url-enrich http://localhost:9200 --import /tmp/git-dashboard.json 2018-03-19 09:29:59,907 Panel not found (not in directory, no panels module): /tmp/git-dashboard.json 2018-03-19 09:29:59,907 Wrong file format (can't find 'dashboard' field): /tmp/git-dashboard.json

[jgbarah]

@Deepayan-Ghosh said:

This is related to the proposal. The goal of the project is to add support to manuscripts to produce reports of any repositories based on the metrics in https://github.com/chaoss/metrics. For example, now the pdf report that is generated has sections on: "Overview","Activity","Community" and "Process", however we want to add support for the metrics provided in https://github.com/chaoss/metrics like Diversity-Inclusion, Risk, etc and for each of these categories, analyse the indices, obtain relevant information and produce representations of metrics like "Contributor demographics" for Diversity-Inclusion category, and so on. Is my understanding correct??

In short, yes. But details could be different. For example, we could use a combination of scripts and Python notebooks for producing the result. But first step will be doing that with the current Manuscripts machinery, the rest will depend on progress and ideas.

[jgbarah]

@kmn5409 said:

So @valeriocos I'm trying to follow the tutorial for https://grimoirelab.gitbooks.io/tutorial/grimoireelk/a-simple-dashboard.html and when I try to create the dashboard I get the issue
$ kidash --elastic_url-enrich http://localhost:9200 --import /tmp/git-dashboard.json 2018-03-19 09:29:59,907 Panel not found (not in directory, no panels module): /tmp/git-dashboard.json 2018-03-19 09:29:59,907 Wrong file format (can't find 'dashboard' field): /tmp/git-dashboard.json

Which file are you using for /tmp/git-dashboard.json? Becase it seems the file is not in the right format.

[jgbarah]

@Deepayan-Ghosh said:

I was following this tutorial: https://grimoirelab.gitbooks.io/tutorial/mordred/a-grimoirelab-dashboard-in-one-step.html . Downloaded the mordred-grimoirelab.cfg file, menu.yaml, and project.json files. I also cloned the directory containing the panels but when I run mordred -c mordred-grimoirelab.cfg I get the following error:
...

The only thing I see different from my setup is that you're using Kibana 6.2.x, and the latest release I've tried is Kibana 6.1.x. Would you mind trying with Kibana (and Elasticsearch) 6.1.x? Meanwhile, I will try with 6.2.x, to see if we have some trouble with it.

Also if I change post to get at https://github.com/chaoss/grimoirelab-mordred/blob/master/mordred/task_panels.py#L361 then I do not get the error.

Maybe this means something changed in ES 6.1x to 6.2.x... Not sure about that. I will check. BTW, please ensure that you have the latest GrimoireLab pip packages, since those available one week ago were known not to work with ES 6.x. (If you don't mind, tell me which version of the grimoire-mordred package are you using).

[valeriocos]

@kmn5409 I'm not able to replicate your error. Below, the steps I did:

• Start your ES (let's suppose that your ES is running on http://localhost:9200)
• Run these commands where your p2o.py is located (in my case /home/GrimoireELK/utils/p2o.py):
  p2o.py --enrich --index git_raw --index-enrich git -e http://localhost:9200 --no_inc --debug git https://github.com/grimoirelab/perceval.git
p2o.py --enrich --index git_raw --index-enrich git -e http://localhost:9200 --no_inc --debug git https://github.com/grimoirelab/GrimoireELK.git
• Check that there is something in your indexes:
  http://localhost:9200/git_raw?pretty=true
http://localhost:9200/git?pretty=true
• Click on https://grimoirelab.gitbooks.io/tutorial/grimoireelk/dashboards/git-dashboard.json, right click, save as git-dashboard.json wherever you want (for instance /tmp/)
• Run this command where kidash is located (in my case /home/grimoirelab-kidash/bin/kidash):
  kidash --elastic_url-enrich http://localhost:9200 --import /tmp/git-dashboard.json

Let me know if this helps to solve your issue.

[Deepayan-Ghosh]

@jgbarah The same modred config file works and does not give any error, when I used Kibana (and elasticsearch) 6.1.0 and 6.1.3 and 6.1.4 versions. However there is an error with the default index pattern that I previously mentioned here: #70 (comment) and hence I moved to using lastest 6.2.2 versions.

On the other hand, I had mordred version 0.1.15 and I upgraded and reinstalled using pip to get version 0.1.16. However, I see in the github repo that version.py of mordred was just updated to 0.1.19. Trying to use the mordred config file using mordred 0.1.16 still gives error:

Kibiter/Kibana: version found is 6.2.2
Kibiter/Kibana: configured!
Dashboard panels, visualizations: uploading...
Dashboard panels, visualizations: uploaded!
Dashboard menu: uploading...
Dashboard menu: uploaded!
Collection for git: starting...
Collection for git: finished after 00:00:04 hours
Enrichment for git: starting...
Elasticsearch aliases for git: creating...
2018-03-21 03:19:26,568 - mordred.task_manager - ERROR - Exception in Task Manager 400 Client Error: Bad Request for url: http://127.0.0.1:9200/_aliases
Exception in thread git:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/deepayan/Grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_manager.py", line 92, in run
    task.execute()
  File "/home/deepayan/Grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_enrich.py", line 253, in execute
    self.__enrich_items()
  File "/home/deepayan/Grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_enrich.py", line 145, in __enrich_items
    task_aliases.execute()
  File "/home/deepayan/Grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_panels.py", line 400, in execute
    self.__create_aliases()
  File "/home/deepayan/Grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_panels.py", line 392, in __create_aliases
    self.__create_alias(es_enrich_url, index_enrich, alias)
  File "/home/deepayan/Grimoirelab/grimoirelab/lib/python3.6/site-packages/mordred/task_panels.py", line 363, in __create_alias
    res.raise_for_status()
  File "/home/deepayan/Grimoirelab/grimoirelab/lib/python3.6/site-packages/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://127.0.0.1:9200/_aliases

Also I successfully created report using manuscripts for git as data_source. However, if I use github_issues as data_source then I get this error, which on comparison with git.py in manuscripts I see that https://github.com/chaoss/grimoirelab-manuscripts/blob/master/manuscripts/metrics/github_issues.py#L36 does not have anything mentioned in the list:

Traceback (most recent call last):
  File "/home/deepayan/grimoirelab/grimoirelab/bin/manuscripts", line 130, in <module>
    report.create()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/manuscripts/report.py", line 711, in create
    self.create_data_figs()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/manuscripts/report.py", line 603, in create_data_figs
    self.sections()[section]()
  File "/home/deepayan/grimoirelab/grimoirelab/lib/python3.6/site-packages/manuscripts/report.py", line 286, in sec_overview
    author = self.config['overview']['author_metrics'][0]
IndexError: list index out of range

[kmn5409]

@valeriocos I don't know if I'm saving the git-dashboard.json file in the wrong place, when you say the /tmp/ folder where is that exactly. I get the same issue as before when I run the command
kidash --elastic_url-enrich http://localhost:9200 --import /tmp/git-dashboard.json 2018-03-21 08:22:32,426 Panel not found (not in directory, no panels module): /tmp/git-dashboard.json 2018-03-21 08:22:32,426 Wrong file format (can't find 'dashboard' field): /tmp/git-dashboard.json
After following your steps of using p2o.py and then ensuring that something is in the indexes, those work fine

[kmn5409]

Also I would like to know about how Microtask #5, so when it comes to providing things like documentation, it says that we can do it on any of the GrimoireLab tools so would it also include repositories like chaoss/grimoirelab-perceval and even the repository we're in now chaoss/grimoirelab?

[valeriocos]

@kmn5409 , open a shell and write cd /tmp/ (you will be in the /tmp/ folder). In a similar way you could store the git-dashboard.json in another directory (for instance with cd ~/, you will be in your home directory).
Can you try to use the .json file in git-dashboard.json.tar.gz instead of the one you are using?

[kmn5409]

Okay thank you for the responses @valeriocos so I saved it in the tmp folder now and when I run the command I now have this error
kidash --elastic_url-enrich http://localhost:9200 --import /tmp/git-dashboard.json Traceback (most recent call last): File "/home/keanu/venvs/perceval/bin/kidash", line 77, in <module> ARGS.data_sources, ARGS.add_vis_studies) File "/home/keanu/venvs/perceval/lib/python3.5/site-packages/kidash/kidash.py", line 703, in import_dashboard feed_dashboard(dashboard, elastic_url, es_index, data_sources, add_vis_studies) File "/home/keanu/venvs/perceval/lib/python3.5/site-packages/kidash/kidash.py", line 720, in feed_dashboard dashboard['dashboard']['value'], data_sources, add_vis_studies) File "/home/keanu/venvs/perceval/lib/python3.5/site-packages/kidash/kidash.py", line 241, in import_item_json res.raise_for_status() File "/home/keanu/venvs/perceval/lib/python3.5/site-packages/requests/models.py", line 935, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://127.0.0.1:9200/.kibana/doc/dashboard:Git

[valeriocos]

You're welcome @kmn5409 , do you have Kibana installed?

[kmn5409]

To the best of my knowledge yes, is this how you check? @valeriocos

[gopal86]

I was running this command as given in the tutorial kidash --elastic_url-enrich http://localhost:9200 --import /tmp/git-dashboard.json. I am getting the following error :-

Traceback (most recent call last): File "/home/vallabh/venvs/grimoirelab/bin/kidash", line 33, in <module> from kidash.kidash import import_dashboard, export_dashboard, list_dashboards File "/home/vallabh/venvs/grimoirelab/bin/kidash.py", line 33, in <module> from grimoire_elk.panels import import_dashboard, export_dashboard, list_dashboards ImportError: No module named 'grimoire_elk.panels'

Can anyone help me out ?

[valeriocos]

Hi @gopal86, have you imported/installed the sigils/panels package?

[gopal86]

yes @valeriocos , i had installed it . it also shows Successfully installed kidash-0.4.3 .

[valeriocos]

It's possible that you are using an old version of grimoire elk. Please @gopal86, try with the latest version.

[kmn5409]

@valeriocos I saw that it might have been the memory on my computer was filled up so I had to make room on it, I assume that was the issue because when I tried to upload things to elasticsearch it was not working, so I think I got it out, and also to answer your question I use docker to get my instance of kibana and elasticsearch.

[LShivaPrabhakar]

Hey @jgbarah

I'd like to work on this project for Google Summer of Code 2018.
Please help me getting started.

[germonprez]

@LShivaPrabhakar

Thanks for interest. We do have tasks that we ask potential students to complete. They can be found here:

https://lists.linuxfoundation.org/pipermail/oss-health-metrics/2018-February/000517.html

You should also note that the student application deadline is: March 27 16:00 UTC.

[jgbarah]

GSoC is well over, so I'm closing this issue.

[urstrulykkr]

Hi@jgbarah
I would like to take up idea #2 as my GSoC-19 project in the link provided
https://wiki.linuxfoundation.org/chaoss/gsoc-ideas
Also, looking to patch few bugs being reported in the issue section of the grimoirelab repo
https://github.com/chaoss/grimoirelab/issues

what would you say ?

[urstrulykkr]

@jgbarah needed your guidance regarding GSoC-19 project contribution.

[jgbarah]

@urstrulykkr sorry for being so late with this. In case you're still interested: this issue was for GSoC'18. For '19, we're still in the stage of proposing our project for participate in GSoC'19. We will open issues with the new tasks in due time. Thanks a lot for your interest!

[urstrulykkr]

Hi Jesus, I have a question to ask,should I wait for the GSoC'19 tasks (micro tasks ) and issues or continue solving the previous year issues. I think issues related to a project would be good. Secondly, what would you recommend me to do, so as to get selected into GSoC'19, to contribute for CHAOSS and learn a lot of stuff with you guys. Best, K. Kaushik Reddy.

…

On Wed, Jan 30, 2019 at 2:44 AM Jesus M. Gonzalez-Barahona < ***@***.***> wrote: @urstrulykkr <https://github.com/urstrulykkr> sorry for being so late with this. In case you're still interested: this issue was for GSoC'18. For '19, we're still in the stage of proposing our project for participate in GSoC'19. We will open issues with the new tasks in due time. Thanks a lot for your interest! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AohxGGtdjU-zIQOczp2ldnlQ5tTaE_Ffks5vILnKgaJpZM4SI2AM> .