bids-standard/bep021

Organizing and coordinating the BIDS Extension Proposal 21: Common Electrophysiology Derivatives

bep021's Issues

Defining derivatives in a chain

NOTE This stems from the bids derivatives workshop Copenhagen 2023

---

How to define derivatives in a "chain"?

Usually this is done via the Sources metadata. However, it might be nice to document the processing chain more explicitly, for example in the entities:

1. <source_entities>_desc-downsampled_<suffix>.<ext>
2. <source_entities>_desc-downsampled+filtered_<suffix>.<ext>

--> problem: this would result in very long filenames fairly quickly.

Alternatively, one could have much shorter labels for the preprocessing steps:

1. <source_entities>_desc-ds+filt_<suffix>.<ext>

--> problem: short labels like "ds" may be too general (i.e., take up a lot of "namespace")

Another alternative would be to have "generic" desc labels with an "inbuilt" index (label1, label2, etc.):

1. <source_entities>_desc-preproc1_<suffix>.<ext>
2. <source_entities>_desc-preproc2_<suffix>.<ext>
3. <source_entities>_desc-preproc3_<suffix>.<ext>

paired with a: <source_entities>_<suffix>.json

that is organized as:

{
    "PreprocessingChain": {
        "preproc1": "downsampling",
        "preproc2": "filtering",
        "preproc3": "..."
    }
}

--> Important note: order in a JSON object should have a meaning ... if it doesn't (to be clarified), we might need to use a JSON array.

More complete example:

{
    "PreprocessingChain": {
        "preproc1": "downsampling",
        "preproc2": "filtering",
        "preproc3": "..."
    },
    "downsampling": {
        "description": "lorem ipsum",
        "Sources": "bids::<source_entities>_<suffix>.<ext>",
        "Anti-aliasing-filter": "<link to key-value pair in SoftwareFilters>",
        "Method": "downsampling method (e.g., decimation --> taking every nth sample; or something else)",
    	"SamplingRate": 300
    },
    "filtering": {
        ... 
    }   
}

BEP 021: Electrophysiology Derivatives (EEG, MEG, iEEG)

Dear all,

As many of you are perhaps aware, we have been working hard on standardizing data archival and storage of electrophysiology data through extensions to the Brain Imaging Data Structure (BIDS) for EEG, iEEG and MEG.

While the focus so far was on raw files, the next step is to standardize the storage of so-called derivative or processed data. In this regard, we would like to survey the electrophysiology community on its present needs and to address them in a data-driven manner. We invite researchers in the community to take 5 minutes of their time to answer this poll: https://forms.gle/k8qpW1ddyk4Hqh5q8. Your feedback and suggestions are very important to us.

See the draft for the derivatives extension here: https://docs.google.com/document/d/1PmcVs7vg7Th-cGC-UrX8rAhKUHIzOI-uIOh69_mvdlw

Best,
BIDS derivative team for electrophysiology

Discuss BEP021 derivatives for electrophys

This continues the issue started here bids-standard/bids-specification#733. This BEP021 is a better place to continue the discussion, as we can also use projects and share files here which would not fit under bids-specification directly.

review of example that Arno shared

On https://drive.google.com/drive/folders/1UepEeZ80wpPxU-OhmiR9oGphomwRR2oX @arnodelorme shared an example derivative EEG dataset. This issue serves to share comments on that draft dataset.

time series convergenve

see bids-standard/bids-specification#713 maybe we need to converge on one way to specify time stamped events

New entity to denote temporal resolution or frequency

Would a tres or freq entity be useful for this BEP? I came across an fMRI workflow (rapidtide) where time series are generated with different frequencies, and @effigies pointed out that BEP021 is probably the BEP with the greatest use-case for a corresponding entity.

Annotations of bad segments of MEG/EEG/iEEG data

The BIDS specification for EEG/MEG/iEEG data allows for the specification of "bad channels", but how about bad segments? Bad segments are segments were all channels are bad, for example due to some issue with the recording hardware, or two raw files were concatenated, causing a "jump" in the signal. This is valuable information to have alongside the raw data, because most processing algorithms will need to skip these sections in order to function properly. Is there a way to encode this information in the BIDS standard?

Changes to BEP021 "moderator" list

Hi all!

This is to announce that I'd like to step down as an "official moderator" of the BIDS extension proposal 021 (ephys derivatives). This is due to some changes in my professional life that will happen in the next two months and that force me to set some priorities in where and how to contribute to BIDS.

We have a long list of moderators on this BEP, see: https://github.com/bids-standard/bids-website/blob/4416d7fd4700cba629533b8d70643ca18b8d02ea/_data/beps.yml#L121-L130

... so perhaps my stepping down can result in a little less "diffusion of responsibility".

I will still contribute (both as a "normal contributor", and in my role as a BIDS maintainer) where I can and where my input would be helpful.

Cheers!

Data format(s)

Hi!

I'll share here some of the discussions we have on the google doc related to BEP021, so we can keep track and they don't get forgotten once we resolve the comments.

One of the first things we need to agree is what data formats should we use to store the resulting matrices of data preprocessing.

This is what we currently have:

.mat
PROS:
-Open specification
-Well supported I/O in both Matlab and Python
CONS:
-Proprietary format
-Allows for highly complex data structures that might need further documentation v7.3 is which is based on HDF5 format (not proprietary) is not supported in Python

.npy
PROS:
-Open specification
-Well supported I/O in Python and C++
-Allows only n-dimensional arrays, limited complexity and thus not easily abused
CONS:
-Experimental support for Matlab

.txt
PROS:
-Simple and easy I/O
CONS:
-Large memory footprint, inaccurate numeric representation

.h5
-See blog post for detailed discussion

@ChristophePhillips commented: Any chance of using the NIfTI format? It was devised for images but can easily any type of 2D/3D/4D signals... and it's typically well interfaced.

@arnodelorme commented: Consider adding .set EEGLAB format and .vhdr Brain Vision Exchange Format which both support data epochs definition and are also both included in BIDS raw EEG data definition.