Discussion: Where the data should come from?

[gabriel19913]

Discussion: Where the data should come from?

[cuducos]

If we can do that, why not just contribute to Brasil.IO which is open source, open data and hosting tons of datasets already? I think making them better is a win-win : )

[gabriel19913]

Hi @cuducos that would be nice too! I just don't know if this projetct should walk in parallel with brasil.io or if they should be independent from each other.
I think the community could contribute getting the data for brasil.io, but maybe the other contributors think is better to have the data in another place.
I really don't what is the best choice.

[cuducos]

I just don't know if this projetct should walk in parallel with brasil.io or if they should be independent from each other.

My opinion on that is that preferably they should walk in parallel. Open-source in general (at least on the level we are in Open Knowledge Brasil and in Brasil.IO) usually suffer from a lack of support and hands-on to keep things up and running. Thus, bringing data collections, cleaning, and treatment to the scope of this project mean two things:

• increasing its complexity (or, in other words, risking decreasing it maintainability)
• capturing some volunteer (or even paid) work hours that would considerably help another open-source project (namely, Brasil.IO)

I think both consequences are not what Open Knowledge and Brasil.IO aspire for.

Surely I hear you: depending on an external project might make things move slower here — and I do agree with that. But I wouldn't assume the two consequences below before hearing from @turicas, for example, that Brasil.IO, for whatever reason, cannot accommodate contributions in a timely manner.

TLDR Getting back to the opening statement, data comes from brasil.io website and they are not updated frequently, I believe we could use this stamina to help them update the data we need more often instead of increasing the complexity of this project : )

[gabriel19913]

I got it @cuducos, thanks for your considerations.
Thinking now maybe I was little bit harsh with this statement.
It wasn't my intention, I know how difficult is to work with open data specially when it depends on the goodwill of the public organizations. Besides that I imagine how hard is to get data for brasil.io specially now with the efforts aimed at covid's data.
I think you made really good points and we can help to update the data that is necessary for this project.

[hilam]

@cuducos

If we can do that, why not just contribute to Brasil.IO which is open source, open data and hosting tons of datasets already? I think making them better is a win-win : )

Because I agree with you, I tried, really, but the onboarding process is very difficult.

In several failed attempts, during this week, I fell into a hell of unresolved dependencies. And the issue has six days without any response. turicas/brasil.io#388

[cuducos]

turicas/brasil.io#388

Just a side note on Brasil.IO: this is not the repository I was suggesting to work on. This is the source code of their website, a Django instance with the dashboard, API and so on. Each dataset has its own repository with the code to collect and clean data.

If we go, for example, to their Eleições Brasil datasets page, there is the URL for the repository that collects and cleans these datasets: github.com/turicas/eleicoes-brasil.

[hilam]

Excellent and necessary guidance, thank you.