juaml / nimgen Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU Affero General Public License v3.0
License: GNU Affero General Public License v3.0
When I have a bit of time, readme should be updated (after merging the significant changes that come with the new command line tool and the pipelining module)
integrate D-CCA from this paper for comparison of domain general and domain specific gene expression gradients.
There are no tests as of yet. I want to add at least a few basic unit tests. Not sure how easy this will be, in particular the AHBA infrastructure, but at least basic functions should be tested. I will try for now to bring coverage to something like 60 to 70% and add some basic CI infrastructure using pytest, tox and github actions.
At the moment it does not work as well as I'd like, but takes time I dont have currently.
Later. Better.
No response
No response
For now, we facilitate use of Pearson and Spearman correlation for the mass-univariate correlation analysis between markers and gene expression. We also want to include TIC and MIC from the paper here:
https://academic.oup.com/gigascience/article/7/4/giy032/4958979?login=false
make a new correlation function that can be dropped in
No response
No response
The parcellation is smashed, and the smashed parcellation is applied to both marker and gene expression data in the surrogate analysis for empirical p-value calculation.
The smashed parcellation should only be applied to the marker, not the gene expression data, which should use the real parcellation.
Run the pipeline.
No response
No response
There are a a few things that need adjusting for the output format:
The lists provided by these fields
correlation_method:
- spearman
alpha:
- 0.05
are run in parallel. Overall, they share a lot of the same data fetching and processing steps, so will likely be better/more efficient to simply run them in sequence.
Each step can read in the configuration file as an argument and then run the pipeline in a for loop over the product of these lists.
No response
No response
I should probably provide some templates for issues and some instructions on how people can contribute to the project.
Currently most parameters for null map generation are hard coded to some implicit defaults in smash.py
. I would like to add some fields to the configuration file to allow for customisation of this process using the neuromaps API (within some constraints regarding volumetric data and MNI space.)
Add fields to configuration file and improve smash.py
No response
No response
Outputs based on all genes should not be saved in marker specific directory, since they typically take a lot of disk space and this way we save duplicates in different marker specific directories which is inefficient.
Save it in a top level/general directory for all genes.
No response
No response
I want to add type hints everywhere as soon as I have the time for it.
Once I merge the command line tool, I will need to create some examples how to run a pipeline using a yaml file. I also need to create some examples on how to directly use some of the functions from the toolbox.
At the moment, nimgen consists mainly of two modules: "expressions" (for loading and processing gene expression data and performing statistical analyses with some parcellated marker) and "smash" (for creation of surrogate nii parcellation files used to perform n correlation analyses with a "smashed" brain and obtain empirical p-values). However, both of these modules take care of quite a bit of "pipelining" i.e. saving outputs etc, whereas other pipelining tasks are left to the user.
I think I would like to cleanly separate the purely analysis based modules from the pipelining. The idea is to have an additional "pipeline" module where individual jobs are defined based on the scheduling system (i.e. HTCondor/juseless, slurm/jureca, multiprocessing/pc etc.) in individual submodules.
Ideally I would then like to be able to configure pipelines in a yaml file (i.e. inputs, outputs, analysis options, etc.) and run it using nimgen as a command line tool, i.e.:
nimgen run config.yaml
or something along those lines. Nimgen can create submit files and submit them then, and perhaps prompt me if I am sure I want to submit this with x amount of jobs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.