kevin931/cytofdr

A Python package for dimension reduction and evaluation workflows with CyTOF data.

Kevin Wang

He/Him/His. Currently working on school, writing quirky software, and most likely watching YouTube :)

What I like:

• 📊📉Statistics, machine learning, and data science
• 💻🖱Digital humanities
• 📚📔Poetry and fiction
• 😍Monchhichis, 🏞 Nature, 🥈Fitness, 🎧Audio

Projects and Publications:

CytofDR

• Wang et al. (2023) in Nature Communications
• A generalizable dimension reduction (DR) framework
• Run and evaluate DR all in one place
• For CyTOF and beyond with customizability and extensibility

PyCytoData

• Yang et al. (2023) in Genome Biology
• An elegant data analysis tool for CyTOF
• A mega package for CyTOF workflows, including CytofDR and cytomulate
• Introducting PyCytoData Alliance and PyCytoData Alliance Plus

Cytomulate

• Yang et al. (2023) in Genome Biology
• An accurate and efficient simulation tool for CyTOF
• My contributions: The maintenance guy who fixes stuff, mops up bits and pieces of code, and writes documentations.

Poetic

• Find out how poetic YOU are via a statistical model!
• I'm working on this (as time allows, that is.) I promise!
• Stable releases are now out! Check it out!

Goals and motivations:

• Stay healthy and fit!
• Research principles: Adaptability, Interpretability, and Usability!
• Contribute to open source!

Follow me:

• Instagram: kevinwang_peace
• Twitter: kevinwang_peace
• Github: kevin931

cytofdr's Issues

[BUG] Python 3.7 Test Failing in CI

What issues are you experiencing?

Our CI tests are failing for the Python 3.7 workflow. Here are the errors:

==================================== ERRORS ====================================
______________________ ERROR collecting tests/test_dr.py _______________________
ImportError while importing test module '/home/runner/work/CytofDR/CytofDR/tests/test_dr.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.7.[17](https://github.com/kevin931/CytofDR/actions/runs/8368312426/job/22912219782#step:5:18)/x64/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_dr.py:1: in <module>
    from CytofDR import dr
CytofDR/__init__.py:3: in <module>
    from CytofDR import dr, evaluation
CytofDR/dr.py:7: in <module>
    import umap
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/umap/__init__.py:36: in <module>
    from importlib.metadata import version, PackageNotFoundError
E   ModuleNotFoundError: No module named 'importlib.metadata'
__________________ ERROR collecting tests/test_evaluation.py ___________________
ImportError while importing test module '/home/runner/work/CytofDR/CytofDR/tests/test_evaluation.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_evaluation.py:1: in <module>
    from CytofDR import evaluation
CytofDR/__init__.py:3: in <module>
    from CytofDR import dr, evaluation
CytofDR/dr.py:7: in <module>
    import umap
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/umap/__init__.py:36: in <module>
    from importlib.metadata import version, PackageNotFoundError
E   ModuleNotFoundError: No module named 'importlib.metadata'
=============================== warnings summary ===============================
../../../../../opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/learn_io/generator_io.py:[26](https://github.com/kevin931/CytofDR/actions/runs/8368312426/job/22912219782#step:5:27)
  /opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/learn_io/generator_io.py:26: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    from collections import Container

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.7.17-final-0 -----------
Coverage XML written to file coverage.xml

=========================== short test summary info ============================
ERROR tests/test_dr.py
ERROR tests/test_evaluation.py
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
======================== 1 warning, 2 errors in 13.[38](https://github.com/kevin931/CytofDR/actions/runs/8368312426/job/22912219782#step:5:39)s =========================

To Reproduce

See our CI reports here: https://github.com/kevin931/CytofDR/actions/runs/8368312426/job/22912219782.

Additional context

This may be related to #6 here as well. We will need to the update workflows anyways. The problem is that we cannot drop Python 3.7 support because of SAUCIE and Grandprix. While they are optional dependencies, SAUCIE's overall good performance is very related to our paper. We need to maintain compatibility for as long as possible.

[BUG] Run SAUCIE Twice!

What issues are you experiencing?

In the current version of CytofDR, SAUCIE can be run only once during each session. We don't currently have a workaround for this, except restarting Python which is annoying. We are recently made aware that we can run tf.rset_default_graph() to reset SAUCIE. This should be incorporated into CytoDR.

This bug should affect all versions of currently maintained CytofDR.

Expected behavior

We should be able to run SAUCIE more than once in CytofDR. Also, we need to remove the warning in the documentation when fixed: https://cytofdr.readthedocs.io/en/stable/installation.html.

Your environment:

• OS: Windows 11
• Python Distribution: Anaconda
• CytofDR version: All
• CytofDR distribution: Anaconda

[Feature Request] Update CI pipelines

What feature would you like to propose?

It may be time to update our CI pipelines to include newer versions of Python: at least 3.10 and 3.11. This will ensure better compatibility going forward.

How do you envision this feature to be implemented?
We can just try it and see if there are problems.

Update reference for all relevant READMEs

The references section and the link to the article should be updated for all relevant README and documentation pages. We can retroactively update v1.0.x (but no new patch beyond EOL).

[Feature Request] CytofDR Playground Documentation

I think we should have a page on our documentation website that documents CytofDR Playground. This will greatly help users connect different components of our paper.

We should highlight a few things:

• A screenshot of what it looks like as a start.
• Explanation of the choices and outputs.
• Have some connections with CytofDR Playground and CytofDR Package.

[BUG] Numpy Error for Conda

What issues are you experiencing?
There is an annoying issue with the conda build. Namely, when downloading CytofDR from conda, the following message is the generated:

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Upon investigation, it has something to do the numpy behavior and perhaps this specific version of numpy.

To Reproduce
Install CytofDR from conda and using the configurations below.

Your environment:

• OS: Windows 11
• Python Version: 3.9.12
• Python Distribution: Anaconda
• Numpy version: 1.20.3 from conda forge.

Note from Kevin: The PyPI version works right now. I will try to figure this out in the meantime.