reproducible-agile/reproducible-agile.github.io

Website for Initiative "Reproducible Research" @ AGILE conference

Reproducible Computational Geosciences

Go to workshop website

Site management

The design is based on Hyde by Mark Otto. The instructions assume you manage your Ruby installation with rvm.

# install required software (if you're using a different system, consider deleting Gemfile.lock for a working configuration)
# rvm install ruby-2.7 --with-openssl-dir=$HOME/.rvm/usr (on Ubuntu 22.04, see https://stackoverflow.com/questions/72179373/cant-install-ruby-via-rvm-error-running-rvm-make-j4-on-ubuntu-22-04)
# bundle install

rvm use ruby-2.7

bundle exec jekyll serve # build the site

Ribbon

A generic "Fork me" ribbon has been added and can be configured (text, link) in the file _config.yml. The color is configured in public/css/agile.css. The ribbon appears on all pages via _layouts/default.html, and stylesheets are included in _includes/head_default.html if enabled.

Exclude pages from menu

If you do not want a page to appear in the left hand side menu, include the parameter exclude_from_nav: true in the page's frontmatter.

Privacy

External CDN links were replaced with local copies as follows:

• https://cdnjs.cloudflare.com/ajax/libs/anchor-js/3.2.1/anchor.min.js saved to public/js/
• https://cdnjs.cloudflare.com/ajax/libs/github-fork-ribbon-css/0.2.0/gh-fork-ribbon.min.css saved to public/css/ (same for IE9 variant .ie.min.css)

Google Fonts stored locally using https://google-webfonts-helper.herokuapp.com. Font Awesome manually downloaded from https://fontawesome.com/v4.7.0/.

reproducible-agile.github.io's Issues

registration for Agile workshop - Stefan Steiniger

1 - preferred hands-on session (R or Python):
=> Python
2 - a short description of experience in R and/or Python
=> I did a bit scripting in Python with Yupiter and QGIS; not much though
3 - a summary of computational work, if available with references to published papers, data or code,
=> My main work has been around java-GIS programming mostly with/for OpenJUMP. However, I haven't touched any code now since 2 years. See:

• Steiniger et al. (2008) on urban pattern recognition based on building structures : a wee bit old now and original work was done in Matlab and Jump in 2006: https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-9671.2008.01085.x
• Steiniger and Hunter (2013) - an animal movement analysis toolbox programmed in 2009-10
  https://onlinelibrary.wiley.com/doi/full/10.1002/wsb.168
• Poorazizi et al. (2015): a social media analysis platform written by a doctoral student (I didn't code anything):
  http://www.mdpi.com/2220-9964/4/3/1389/html
• and lately we have worked on a set of urban sustainability indicators in R, but here as well I also did not write a line of code on this. A manuscript has been submitted to Cities and code is here: https://github.com/CEDEUS/city-indicators-calculation-scripts

4 - plans for future computer-based research.
=> Currently I mainly working with my team on an urban accessibility/ analysis platform see Steiniger et al. 2013 (https://www.sciencedirect.com/science/article/pii/S2214140517308691) and on urban sustainability indicators.

Jeremy Azzopardi

1. preferred hands-on session:
   Python

2. a short description of experience in R and/or Python:

**Experience in using python as a tool for data management (mostly automating repetitive tasks) as well as some experience as part of my master's degree (using python to process some results). Example: Iterating through hundreds of data packages, finding all relevant shapefiles and access databases, converting to OGC GML and csv and renaming + repackaging relevant folders, etc. some experience in relevant packages such as numpy, use of spyder for geodata , etc **

3. a summary of computational work, if available with references to published papers, data or code, and
   plans for future computer-based research:
   None really, my plan is to take some courses in data analytics which would eventually help towards a phd. My interest in this workshop is partly due to my job as datasamordnare at the Swedish National Data Service

AGILE Workshop Registration - Stefan Wiemann

• preferred hands-on session (R or Python)

  • R

• a short description of experience in R and/or Python

  • R: statistical analysis in projects in the field of statistical air quality modeling and the analysis of hydro-meteorological extreme events; combination with OpenCPU platform for service-based R processing
  • Python: primary use as scripting language in combination with ArcGIS or FME

• a summary of computational work, if available with references to published papers, data or code

  • experienced with Java, Web languages (HTML, JavaScript, PHP), R, Python
  • current GitHub repositories (also referenced in a number of articles):
    • https://github.com/GeoinformationSystems/xtruso_R (EXTRUSO R package)
    • https://github.com/GeoinformationSystems/xtruso_web (EXTRUSO information portal)
    • https://github.com/GeoinformationSystems/SpatialDataFusion (PhD work)

• plans for future computer-based research

  • mainly application-based programming in projects
  • further development of PhD work on spatial data fusion

Registration Lars Harrie, Lund

1. Prefer Python
2. Do only some Python coding myself, but I supervise PostDocs and PhD students that do much coding. In some projects we use the Jupyter environment for sharing code with the future aim to make the code writing more transparent.
3. A project where part of the code is in Python available on Github is published in:
   https://www.tandfonline.com/doi/full/10.1080/13658816.2018.1441416
   Another article that deals with data integration is found in:
   https://www.tandfonline.com/doi/pdf/10.1080/10095020.2015.1071065
   At the Agile conference I will present a short paper that describes how we deal with part of the computations in ICOS CP (https://www.icos-cp.eu/) that utilizes Python Jupyter notebook.
4. Is interested to learn and improve structuring of code and annotate data for improving the transparency in the studies made by our group in Lund.

Workshop report

Contents

• summary of call
• titles and short summaries of presented submissions
• summary of discussion
• committee members
• list of rejected submissions (links to PRs)

AGILE 2020 workshops

Proposals

The current suggestion is two have two workshops: one about the guidelines, and one more practical. Interested people can attend both, but they can choose better.

Contributions to the proposals are welcome:

• Workshop I: https://docs.google.com/document/d/1OiL9nZqi2vLzOUDRennrWet0IGYojzmnw8P9WXCHCj0/edit
• Workshop II: https://docs.google.com/document/d/1TZ0BAgeivMpQRlbndniPft-kVlSg762qyXWk_vewyOc/edit

Tasks - proposal

• reach out to the organisers and ask if they prefer two proposals, or just one and they make clear in descriptions etc. that people can attend/register for only parts.
• everybody checks the proposals
• submit, see https://agile-online.org/conference-2020/call-for-workshops-2020
• add proposal PDFs to website repo

Tasks - after acceptance

• create workshop website
• build (extend) organising team
• material: I suggest to create one OSF project per workshop, as component of the main project.
  • take a good look at https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007433

AGILE Workshop Registration - Simon Oulevay

Apologies for submitting my registration after the due date. I did not know I was coming to AGILE for sure until today.

1. Preferred hands-on session (R or Python)
   Python
2. A short description of experience in R and/or Python
   I have some Python scripting experience but am not proficient. However, I am a programmer and know several other dynamic languages (JavaScript, Ruby), and will brush up on my skills with a few tutorials before the workshop.
3. A summary of computational work, if available with references to published papers, data or code
   No computational/scientific work to speak of. I am a programmer mostly working with JavaScript, Java, Ruby, HTML, CSS.
4. Plans for future computer-based research
   Nothing specific right now, but my institute is working in the area of GIS and machine learning where the learned skills will surely prove useful.

(I am a bit of an intruder in that I am a mostly a web programmer with little to no experience in computational work or scientific publications. However, I have an interest in GIS, literate programming, specifically Jupyter notebooks, and reproducibility. Hence why I would like to participate in the workshop if you have any room left.)

Review guidelines

Do we need review guidelines? I think the reviewers might appreciate a little input, and it should help using similar standards across al reviews.

Draft for review guidelines

• Reviewers are encouraged to provide a public pull request review, but intermediate interactions with the submitter are of course also welcome, either by commenting on a pull request or using the chat. Please try to limit use of other/private publication channels (email, Skype) to cases where confidentiality is required.
• Reviewers should not concern themselves with the format of the submission.
• Review criteria Please use these criteria to structure the review and use keywords "high", "medium", "low", "not applicable".
  • Relevance and contribution to the workshop call and the topic of reproducible research
  • Quality of the submission (language, organization of content, completeness, clarity)
  • Reproducibility (if submission contains computation)
  • Originality (unless reports on pre-published work)
  • Significance of results (where appropriate)
  • Scientific rigour (references etc.)
• Be professional and fair.

Resources

• http://www.cgl.uwaterloo.ca/smann/Research/review-conference.txt
• http://www.dfi.org/pdf/Reviewer%20Guidelines.pdf
• http://fusion2016.org/Review_guidelines
• http://aom.org/annualmeeting/reviewerguidelines/

Fix schedule

The conference organizers advise to put deadlines rather "too early", because of the high number of accepted proposals and Wageningen's hotels being potentially filled quickly. With this in mind I created a timeline draft:

• submission deadline: March 19
• assignment of reviewers: March 21
• reviews due: April 2
  • confirmed reviewers for this period
    • @schibalski
    • @JudithVerstegen
    • @vwmaus
    • @ploewe
    • @mrtnhmtz
    • @foost
    • @edzer
• notification of acceptance: April 4
• agenda published: April 10
• workshop: May 9
• publication of report: May 31

@edzer @MarkusKonk I am on holidays until March 21. If you'd be available to assign the submissions to the reviewers, then we can also push that date to a couple of weeks earlier.

Alternatively, we could go way earlier and put the deadline to Feb 12 and assign reviewers on Feb 13/14.

Registration for Levente Juhász

Experience level: My language of choice is python and I'm proficient in R and JavaScript. I'm also confident in using other languages as needed. I have published bits of code and datasets here and there for my research but it was always ad-hoc. I prefer Linux (Ubuntu and Red Hat mainly).

Expectations: I want to participate in the workshop because I want to learn more about RR. I've been wanting to make my work more reproducible for some time now but never really took the first step. I am excited to learn more about theory and best practices and eventually see how they work in real life. I'm hoping that this workshop will push me towards making my work more reproducible.

Registration for Amelie Haas

Experience level: I am familiar with R and Python, but don't have any (own) experiences with RR yet. My background is geoinformatics / GI technologies / remote sensing / statistics. I'm using Windows OS.

Expectations: I am looking forward to gain an insight into RR concepts and practices. As I am at the very beginning of my PhD, this will be valuable knowledge for my future work.

AGILE2018-Tiinas-new-issue

tefiurtegduy

Peter Kedron

Experience level: (e.g. programming language(s) and proficiency, published reproducible articles, published datasets, experience in data analysis, used operating system, taught courses using OER, ...)

Python, R, Matlab

Expectations: I want to participate in the workshop because ...
Very interested in this topic.

Software prerequisites

We can use this issue to notify the participants (@-mentioning) about preparations they should take before coming to Lund.

Review process

@MarkusKonk @edzer Feedback welcome!

Draft for review process

This workshop runs a public peer review. The review process described here is therefore transparent for submitters and tries to be both brief and explicit.

1. Submitters create a pull request (PR) with their submission. By creating the PR the submitter accepts that their work will be published under a Creative Commons Attribution 4.0 International License.
2. The organization committee (@o2r-project/agile-2017-org-committee) checks the form of the submission and assigns the PR to at least one member of @o2r-project/agile-2017-reviewers to conduct the main review following the guidelines. Reviewers are welcome to follow new PRs and volunteer for a submission.
3. After the review is completed, the reviewer creates a the review recommendation comment on the PR mentioning the organizing committee (@o2r-project/agile-2017-org-committee) and including one of the following statements: Review recommendation: Accept for presentation, Review recommendation: Accept for presentation after revision, Review recommendation: Reject
4. All reviewers are welcome to give a short comment on a submission.
5. In case of Accept for presentation after revision, the PR is kept open until the recommendation is updated to either Accept for presentation, or Reject.
6. Once the review recommendations for all submissions are either acceptance or rejection, the @o2r-project/agile-2017-org-committee holds a meeting public Gitter chat and discusses all recommendation considering the overall programme and bounding conditions (numbers of possible slots, diversity of topics). The committee then make a decision comment for each submission, trying to follow the reviewers' recommendation: Decision: Accept for presentation, Decision: Reject
7. The decision comment invites all review committee members (@o2r-project/agile-2017-reviewers) to briefly check the review arguments and cast a vote using "thumb up/down" reactions on the comment. There must be a simple majority and the review committee is the tie breaker.
8. In case of acceptance of the submission, the PR is merged by @o2r-project/agile-2017-org-committee. In case of rejection, the pull request is closed without merge (a list of accepted and rejected submissions will be included in the workshop report.

Registration for Stefano De Sabbata

Experience level: I use R (incl RMarkdown) and Python (incl Jupyter notebooks) on a regular bases, but I have used other languages as well. I use git and I have experience with data analysis. I have published data and code for some of my articles, but none that could be considered fully reproducible. I have published a reproducible lecture (and practical session) on reproducible research in R, and I hope to expand that to the rest of the module I teach on programming in R for our MSc in GIScience. I use win, linux and mac on a regular basis.

Expectations: I want to participate in the workshop because I want to learn about best practices in reproducible research. I am very interested in how that is integrated in the research workflow and discuss how that can work in the reality of everyday research (rather than in the slightly idealised scenarios commonly found in books).

Registration for Carsten Keßler

Experience level: I use Python and R on a regular basis (but not as often as I would like to) and have experience in a couple of other languages. I have occasionally published code and/or data for my articles, but not to a level that I would call reproducible. Mac user, but can survive on Linux and Windows.

Expectations: I want to participate in the workshop because I want to make my work more reproducible and follow common conventions as closely as possible.

Registration for Pengxiang Zhao

Experience level: I am proficient in Python and use it a lot now. I am also familiar with a couple of other programming languages (e.g. C#, matlab, R). I have no experience in publishing datasets. I have rich experience in spatiotemporal data (especially GPS trajectory data) analysis and mining. I mainly use Windows OS.

Expectations: I want to participate in the workshop because I would like to know more about reproducible research. I also want to make my research (i.e. data, code, publications) more reproducible, but I don't know where I can get started. I hope I can learn some experience and skills about reproducible research.

Registration for Martin Tomko

Experience level: Python, R, played with Knitr and interested in similar for Python, published datasets, with experience in data analysis, used MacOSX, Linux, Win, taught courses in geospatial analysis with Python

Expectations: I want to participate in the workshop because I want to learn about best practices. I know about the theory of reproducible articles and literate programming ( in R primarily), but I would like to explore this in Python ,and see also how the data+code +article can be published, executed ( as per Binder), dependencies managed for longevity of the contribution. Ultimately, interactive papers would be the goal as I see it (such as exporables).

2nd workshop 2018 - proposal

The call for workshops is out. Deadline is 30 November 2017.

2017 proposal

Ideas for topics

• re-do http://o2r.info/agile-2017/ as it happened (introduce the topic to newcomers, share experiences)
• present current paper (in writing), discuss results
• focus on suggestions for AGILE (conference, member labs)
• re-do the original workshop as intended (call for papers, public review, present reproducible works)
• provide hands-on instructions for RR

---

Please feed free to add comments and more ideas here, I'll integrate them in this comment asap.

Registration for Junaid Abdul Jabbar

Experience level: I have worked mostly on python. I am also familiar with R and Java. I have not published any dataset yet. I mainly use windows OS. I have taught a bachelor's level course on GIS Development with Python using IDLE.

Expectations: Main purpose of my participation is to learn the principle and steps for making my work reproducible so that anyone going through it can have a clear picture and produce same results. It has been a challenge for me when I have been going through some research works during my Master's studies.

AGILE - Francisco Ramos

1. preferred hands-on session (R or Python),
   R

2. a short description of experience in R and/or Python

Basic statements and R for leaflet

3. a summary of computational work, if available with references to published papers, data or code

Computer Scientist with programming skills, I have coded in Java, Javascript, Swift, C++, etc.
http://franciscoramos.name

4. plans for future computer-based research.

Data and Statistics Analysis with R and GPU Programming

Rob Lemmens

Experience level: (e.g. programming language(s) and proficiency, published reproducible articles, published datasets, experience in data analysis, used operating system, taught courses using OER, ...)

Not so much data analysis recently. More focus on developing light-weight ontology/concept maps for education , GI workflows and domain research, such as in citizen science

Expectations: I want to participate in the workshop because ...

I would like to see how my work on conceptual modelling / workflows can be utilised in RR. And of course want to make my own work reproducible and learn how to do that better.

3rd workshop 2019 - proposal

Call: https://agile-online.org/conference-2019/call-for-workshops-2019

Deadline: 30 November 2018

Proposal document draft: https://docs.google.com/document/d/1ns7ZNfsd-tjuFGX__nuzrovdC9Q_laQdbbnjDhUtZaA/edit?usp=sharing

For ideas see #8 and https://peerj.com/preprints/27216v1/#feedback-1207