shads196770/neurofinder

Benchmarking platform and competition for source extraction from imaging data

NeuroFinder

Benchmarking platform and challenge for source extraction from imaging data.

Develop algorithms interactively in the browser using Jupyter notebooks — or download the data locally and develop in your favorite computing environment. Then have your algorithms automatically deployed and tested in the cloud.

explore the training data in interactive notebooks

1. Go to neurofinder
2. Click on the notebook link
3. Explore the tutorial notebooks to learn about data formats and run example algorithms
4. Run the custom algorithm notebooks to see how to write an algorithm.

All notebooks also available at this repo

download the training data for local exploration

1. Visit the local section of this repo for an overview
2. Browse the list of datasets for direct download
3. Download one or more of them
4. Use the example scripts to load and explore the data in your language of choice

submit an algorithm

1. Sign up for an account on github (if you don't already have one)
2. Fork this repository
3. Create a branch
4. Add a folder inside neurofinder/submissions with the structure described below
5. Submit your branch as a pull request and wait for your algorithm to be validated and run!

Submission structure:

neurofinder/submissions/user-name-alogirthm-name/info.json
neurofinder/submissions/user-name-alogirthm-name/run/run.py
neurofinder/submissions/user-name-alogirthm-name/run/__init__.py

The file info.json should contain the following fields

{
    "algorithm": "name of your algorithm",
    "description": "description of your algorithm"
}

The file run.py should contain a function run that accepts as input an Images object and an info dictionary, and returns a SourceModel (these classes are from Thunder's Source Extraction API). See the existing folder neurofinder/submissions/example-user-example-algorithm/ for an example submission.

Jobs will be automatically run every few days on a dynamically-deployed Spark cluster, so be patient with your submissions. You will be notified of job status via comments on your pull request.

data sets

Data sets for evaluating algorithms have been generously provided by the following individuals and labs:

• Simon Peron & Karel Svoboda / Janelia Research Campus
• Adam Packer, Lloyd Russell & Michael Häusser / UCL
• Jeff Zaremba, Patrick Kaifosh & Attila Losonczy / Columbia
• Nicholas Sofroniew & Karel Svoboda / Janelia Research Campus
• Philipp Bethge and Fritjof Helmchen / University of Zurich (in preparation)

All data hosted on Amazon S3. Training data availiable through the CodeNeuro data portal, and also for direct download via the links in this list.

environment

All jobs will be run on an Amazon EC2 cluster in a standardized environment. Our notebooks service uses Docker containers to deploy an interactive version of this same environment running in Jupyter notebooks. This is useful for testing and developing algorithms, but is currently limited to only one node.

The environment has following specs and included libraries:

• Python v2.7.6
• Spark v1.3.0
• Numpy v1.9.2
• Scipy v0.15.1
• Scikit Learn v0.16.1
• Scikit Image v0.10.1
• Matplotlib v1.4.3

as well as several additional libraries included with Anaconda. You can develop and test your code in a full cluster deployment by following these instructions to launch a cluster on EC2.

about the job runner

This repo includes a suite for validating and executing pull requests, storing the status of pull requests in a Mongo database, and outputting the results to S3. To run its unit tests:

• Install the requirements with pip install -r /path/to/neurofinder/requirements.txt
• Call py.test inside the base neurofinder directory