remi-gau/ecobidas

Working repository to turn the COBIDAS guidelines to report methods and results in neuroimaging into a user friendly checklist

COBIDAS guidelines checklist

• COBIDAS guidelines checklist
  • Prototypes
  • Want to know more ?
  • How to reach us
  • How to contribute
  • Contributors ✨

The main aim of this project is to improve reporting methods and results in neuroimaging (f/MRI, i/EEG, MEG, PET...) in order to increase transparency and reproducibility.

We want to do this by developing a checklist based on best practices guidelines that is both easy and practical to use, and that provides a computer readable output that can be used to automatically generate parts of the methods section.

Prototypes

Those apps and incoming ones are based on the following best practices guidelines:

• Best Practices in Data Analysis and Sharing in Neuroimaging using MRI
• Best Practices in Data Analysis and Sharing in Neuroimaging using MEEG
• Guideline for Reporting Standards of Eye-tracking Research in Decision Sciences
• Guidelines for the content and format of PET brain data in publications and archives: A consensus paper

web app	reference	spreadsheet	Github repository
COBIDAS MRI	link
neurovault	link	spreadsheet
pet	link	spreadsheet	github
eyetracking	link	spreadsheet	github
artemis	link	spreadsheet	github
reexecution	link	spreadsheet	github

Want to know more ?

You want to know more about:

• the motivations behind the project?
• the different goals of the project?
• how it is implemented?

Head over to our documentation

How to reach us

If you want to be kept posted about the progress of the project, you can join our google group.

For more frequent updates and behind the scenes, come and join us on the cobidas_checklist channel on the brainhack mattermost. Join our channel

Otherwise you can open a new issue on the repository itself if there is something you would like to discuss directly here.

How to contribute

We are looking for people to give us feedback or help us move forward.

Join our working group at the International Neuroinformatics Coordinating Facility

To learn more about how to contribute see here.

Contributors ✨

Thanks goes to these wonderful people (emoji key).

For a more detailed description of the contributions, see here.

Remi Gau 💻 🎨 🖋 🤔 🚇 🚧 📆 🔧 📢	cassgvp 💻 🎨 🖋 🤔 🚇 📢	joyswe 💻 🎨 🖋 🤔 🚇	Federico Adolfi 💻 🎨 🖋 🤔 🚇	Sanu Ann Abraham 💻 🎨 🚇	Thomas Nichols 🎨 🖋 🤔	Anisha Keshavan 💻 🎨 🚇	Satrajit Ghosh 💻 🎨 🚇
Tim van Mourik 💻 🎨 🚇	m-miedema 💻 🎨 🖋 🤔 🚇	David Moreau 🖋 🤔	Zsuzsika Sjoerds 🖋 🤔	angietep 💻 🎨 🖋 🤔 🚇	Martina G. Vilas 💻 🎨 🖋 🤔 🚇	Kristina Wiebels 🖋 🤔	Johannes Algermissen 🖋 🤔
Dorien Huijser 🖋 🤔	Wouter Weeda 🖋 🤔	jasminetan6032 💻	MarCordero 💻 🤔	Cyril Pernet 🤔 🖋	lspieser 🖋

This project follows the all-contributors specification. Contributions of any kind welcome!

ecobidas's Issues

Adding contributors

In this issue we use the all contributor bot to add contributors and specify what they did

incorporate intracranial ephys to MEEG

Define use case for fMRI pre-registration

Identify correspondance to brain map metadata

It could be interesting to see if it is possible to extract the metadata from the brainmap database to speed up creating a COBIDAS database?

encode branching logic within the spreahsheet

General comments

to what extent should analysis options be pre-defined based on methods selection?

Eg. task-specific or resting-state data, method of source-modelling

at which point should outcomes be specified? eg. might be more user-friendly to pre-select outcomes and then track methods backwards

include search functionality? might help for controversial/unclear categorizations

which parts of checklist should contribute to Methods output? (eg. don't need to describe whether or not units were put on figures)

related to the above: should some sections of checklist be optional? (people might be more likely to use a shorter checklist; 'obvious' criteria such as units for figures might seem like a waste of time)

for example, rather than having required fields could "suggest" fields highlight the most important fields to complete |  

convert MEEG COBIDAS into spreadsheet

Use BIDS terminology for the whole project

This project will use the BIDS terminology so it is important to make sure that the spreadsheets used to generate the checklist use the right terminology.

• MRI
• MEEG
• PET

Use Carp 2012 to create list of items

Without worrying too much about branching logic at the moment.

With only one anatomical and one fMRI task.

• cross-refs items from Carp 2012
• itemize them as much as possible
• note percentage of study reporting this item
• create a simplified csv file
• create scripts to convert neurovault metadata csv into a schema
• create scripts to convert Carp 2012 csv into a schema
• create one question for each item

Find a way to arrange to have several fMRI / MEG /EEG tasks

Identify correspondance with BIDS

If we want to speed up how quickly user can go through the checklist, we can do this by leveraging the cases where users have organized their data using a BIDS to automatically get as much metadata from the BIDS from possible to fill in the checklist.

This requires knowing if each COBIDAS item has a counterpart in the BIDS and where it is located.

Reciprocally we might realize that some COBIDAS items could easily be added as new field in the BIDS world.

get software list through the RRID database

Possibility to import a json file and continue filling the checklist

This feature will be needed if some users cannot finish going through the checklist in one go and want to resume wokring on it from where they left off.

This could also be useful for users who run very similar studies and want to start from a given template.

In both cases users could save their work by exporting the JSON files corresponding to all the items of the checklist they have provided an answer to. They could then restart from where they left by importing the previously saved JSON file.

comment

Make items more explicit

• add an example for those that are too obscure
• redirect to a page with more explanation

fuse spreadsheet from SIPS 2019 with those of OHBM hackathon 2019

render schema-ui locally and adapt to our needs

Quick instructions to render the schema thing locally:

• Clone the schema ui repo (https://github.com/ReproNim/schema-ui)
• Install npm (https://www.npmjs.com/get-npm)
• Step inside the schema-ui directory and run npm install
• Run npm run dev
• Open your browser and go to localhost:8080

update README

• link to google group
• link to google drive
• update of hackathon results
• breakdown in several markdown docs
• link to contributors, front-end rendering...

identify gaps or missing information

ideally for a COBIDAS 2.0

compare COBIDAS to Carp 2012

creating a mini-checklist: compare cobidas checklist to content of Carp 2012 to identify what minimum should be reported to do better than 50% in the litterature

create choice lists - MEEG

Tidy and incorporate google drive

Bring docs, presentations, etc. into repo.

add contributor info and update info on how to contribute

Add contributor.md so we know who we all are and what we're doing!

Create one question for each item

incorporate meta-analysis information

(work by Cristina)

• comments
• meta-analysis use case into main MRI spreadsheet

Implement "mandatory item"

• make items unskippable

Using the PET guidelines seems like an easy way to do this as it already possesses a column of "required / not required".

This needs to be implemented in the python scripts used to convert the csv into the activities.

Requires figuring out how this is encoded in the UI part of the schema.

create choice lists - MEEG

Use the NARPS results to test the checklist

The NARPS project released a table of how its 70 teams analysed the data by having each team use the COBIDAS guidelines to report what they did.

This could be a good way to test the checklist.

A copy of this table is in the reference folder of our google drive: https://docs.google.com/spreadsheets/d/1mNQdwVgkcAPws67CmnAxCWbsdYWm8sWpkJYy1aPcUWg/edit?usp=sharing

TO DO

• make sure all the items used in NARPS are in the list of the included items in the checklist.

Implement the "slider" input type for the app

• could be used for things that are on a limited range: statistical power, alpha level...

Add a "help" section for each item

The user can get more info on an item by clicking on a "help" button.

This could possibly include links to other resources to better explain the item.

create choice lists - MRI

(e.g list of type of most commonly used data interpolations)

Implement drop-down menus for softwares

Check Voice study language questions in Demographics

identify and define use cases

• personal use checklist to make sure you got everything
• write methods
• review a paper

consider using personas and pathways

• https://mozillascience.github.io/working-open-workshop/personas_pathways/

front-end

• check ReproNim standardization option
• turn it into a docker image
• play with to display a proof of concept website
• host on website

OHBM abstract and zoom meeting

Hey all.

1. I am thinking of sending an abstract for this work for the next OHBM. Do you think it is a good idea? I might need some people willing to have a look at it. I am planning to make a pretty inclusive list of authors (pretty much based on the current list of contributors) on that so let me know if you do not want to be on this.

2. I have been working a bit on the next upgrade for the app but we should definitely have a zoom meeting on all of this. So here are some possible dates: https://doodle.com/poll/eqxsa9mizifinbhb

Let me know what you think

@fedeadolfi @kwiebels @m-miedema @zsjoerds @TimVanMourik @JamesEBartlett @Joyswe @angietep @cassgvp @eduardklap

Allow re-using of some parts that have been already answered

It is possible that some users will want to specify that they have used exactly the same parameters for certain parts of their experiment or analysis (e.g same MRI sequence and preprocessing for 2 different fMRI tasks).

Having the possibility to mention this by simply clicking one button would be a big gain of time.

Defining use case

Users could use this checklist with different goals in mind:

• as sanity check: during write to make sure that they are methods is thorough
• for pre-registration or when planning a study
• when evaluating a study
  • during peer-review
  • when performing a meta-analysis

Later on:

• to write the methods section automatically

Create choice list in PET spreadsheet

Clean OSF wiki

• reuse readme

Implement text box appearing when "Other" has been selected

Check Voice pilot study demographics questions

Refactor spreadsheets

This will need to happen later once the MEEG version has been OHBM approved.

• identify which section can are almost identical (like the participants section)
  • behavior
  • sample
• turn those sections into an independent spreadsheet
  • behavior
  • sample

Add a code of conduct and how to contribute guidelines

• code of conduct
• contributing guidelines
• governance

atomize items

link to a URI (unifirm ressource identifier)? Interlex? NIDM?

Refactor conversion python scripts

Refactor

• define_response_choice
• get_item_info

---

• automate getting column headers (name and column number)
• skip header file automatically
• transform main script into a function
• create a script to loop through all the CSVs
• create classes

Identify items that can easily be incorporated into BIDS specs

Identify response type for each question - MRI

boolean, numeric, choice list...

Identify items corresponding to neurovault metadata

Identify correspondance with NIDM results

Similar to issue #45 but with NIDM results, we can speed up filling in the checklist by reading directly the content of an NIDM results.

Create boilerplate text for methods section automatic generation.

We want the web-app to automatically generate a method section.

We therefore need to have some template or boilerplate method section with empty placeholders to be filled in by the different options chosen by users when they fill the list.

This requires writing those boilerplate texts for each activity of the checklist.

• This might need identifying which set of items can be grouped together to be put in the same sentence in the methods section.

A possible simple boilerplate to create is the M/EEG equivalent used by the report module pyBIDS to describe the content of BIDS data set. pyBIDS is very MRI focused but this could be a simple addition to the COBIDAS project that could be reinjected into the pyBIDS package.

• M/EEG descriptor just like the report module in pyBIDS

Create tests for python conversion scripts and add them to CI