broadinstitute / wot Goto Github PK
View Code? Open in Web Editor NEWA software package for analyzing snapshots of developmental processes
Home Page: https://broadinstitute.github.io/wot/
License: BSD 3-Clause "New" or "Revised" License
A software package for analyzing snapshots of developmental processes
Home Page: https://broadinstitute.github.io/wot/
License: BSD 3-Clause "New" or "Revised" License
Hallo again. I'm honestly impressed by the modifications and fixes you're doing to the program. Thanks a lot for that.
I would like to better understand the trajectory and trajectory_trends plots I can now create from the command line.
Running wot trajectory
withe the --embedding
options creates one png file woth the plot. The file name though get this tilte traj_1.0.4_None_trajectory.png
. What is the meaning of None?
This says even when using specific cell_sets.
Running wot trajectory_trends
with --plot
create also png file(s). Also these files are somehow not defined. The file name get this format
'trends_1.0.4_Unnamed: 2.mean.txt'
and 'trends_1.0.4_Unnamed: 2.png'
.
2.1 It would be great if no spaces or symbols are added to the names, so that it would be easier to handle.
2.2 What is the meaning of Unnamed?
2.3 When I use the --gene_filter
option (in my example i took only two genes) i get one plot, but when not using this option I get multiple files
'trends_1.0.4_Unnamed: 1.mean.txt'
'trends_1.0.4_Unnamed: 1.png'
'trends_1.0.4_Unnamed: 2.mean.txt'
...
'trends_1.0.4_Unnamed: 5.png'
'trends_1.0.4_Unnamed: 6.mean.txt'
'trends_1.0.4_Unnamed: 6.png'
'trends_1.0.4_Unnamed: 7.mean.txt'
'trends_1.0.4_Unnamed: 7.png'
I have stopped the command than, as I can't see the differences. Can you explain this?
thanks again for all the help
The update was successful, but unfortunately all the scripts and command afterwards breaks with errors. first i have tried to run the example scripts you provided with the github repository, which all don't work any more (issue #49).
Now I have tried to run calculate the transport maps, but it also breaks with an error.
$ wot optimal_transport --matrix p2_matrix.transposed.txt --cell_days p2_timepoint.txt --out tmaps_p2_timepoint --format loom --local_pca 30
Traceback (most recent call last):
File "/home/yeroslaviz/miniconda3/envs/Waddington/bin/wot", line 11, in <module>
load_entry_point('wot', 'console_scripts', 'wot')()
File "/home/yeroslaviz/miniconda3/envs/Waddington/lib/python3.7/site-packages/wot/wot/__main__.py", line 23, in main
cmd.main(command_args)
File "/home/yeroslaviz/miniconda3/envs/Waddington/lib/python3.7/site-packages/wot/wot/commands/optimal_transport.py", line 23, in main
ot_model = wot.commands.initialize_ot_model_from_args(args)
File "/home/yeroslaviz/miniconda3/envs/Waddington/lib/python3.7/site-packages/wot/wot/commands/util.py", line 47, in initialize_ot_model_from_args
covariate=args.covariate if hasattr(args, 'covariate') else None
File "/home/yeroslaviz/miniconda3/envs/Waddington/lib/python3.7/site-packages/wot/wot/ot/initializer.py", line 39, in initialize_ot_model
return OTModel(ds, **kwargs)
File "/home/yeroslaviz/miniconda3/envs/Waddington/lib/python3.7/site-packages/wot/wot/ot/ot_model.py", line 111, in __init__
raise ValueError("Days information not available for matrix")
ValueError: Days information not available for matrix
I have all the file in the same directory with the correct format
$ tree -L 1 ./
./
├── p11_ca140_geneset.gmt
├── p2_ca143_geneset.gmt
├── p2_gene_set.AllTP.gmt
├── p2_gene_set.gmt
├── p2_matrix.transposed.txt
├── p2_timepoint.txt
$ head p2_timepoint.txt
id day
p2_cortex_143_AAACCTGAGGCTAGGT 2
p2_cortex_143_AAACGGGCAATCGGTT 2
p2_cortex_143_AAAGTAGCATAAAGGT 2
...
gene expression matrix
id Xkr4 Mrpl15 Lypla1 ...
p2_cortex_143_AAACCTGAGGCTAGGT -0.219520647107675 -0.729889404055677 ...
...
any ideas?
Hi!
I was going to test run 'wot' with the example data in the homepage, but I got stuck in the beginning.
I copied the command and pasted to run the following command
wot neighborhood_graph --matrix data/ExprMatrix.var.genes.h5ad --space dmap --neighbors 50 --pca_comps 100 --diff_comps 20 --out fle-input
and obtained a following error
wot: error: argument command: invalid choice: 'neighborhood_graph' (choose from 'convert_matrix', 'cells_by_gene_set', 'census', 'diff_exp', 'fates', 'gene_set_scores', 'optimal_transport', 'optimal_transport_validation', 'trajectory', 'trajectory_divergence', 'trajectory_trends', 'transition_table')
I saw 'neighborhood_graph' command in the homepage, but the error said like no such tool existed.
What could be the problem here?
Thank you!
Hi,
I'm running the commands using the supplied data from the examples folder. Unfortunately I keep getting the error when running the following command.
wot local_enrichment --score t_test --matrix1 ../wotExamples/trends_Lymphocytes.mean.txt --variance1 ../wotExamples/trends_Lymphocytes.variance.txt --matrix2 ../wotExamples/trends_Granulocytes.mean.txt --variance2 ../wotExamples/trends_Granulocytes.variance.txt
I get this error:
Traceback (most recent call last):
File "/home/USER/.local/bin/wot", line 10, in <module>
sys.exit(main())
File "/home/USER/.local/lib/python3.7/site-packages/wot/__main__.py", line 23, in main
cmd.main(command_args)
File "/home/USER/.local/lib/python3.7/site-packages/wot/commands/local_enrichment.py", line 60, in main
ds2.variance = ds2.variance[:, ds_indices]
AttributeError: 'AnnData' object has no attribute 'variance'
All the other commands have worked so far.
But I'm not sure why this is happening.
I would appreciate your help.
thanks
Hi, I have a question to your example script in docs/examples/code/run_example.sh
.
In there there is a way to calculate the cell-sets using gene scores. This is done by first computing the gene scores via the script wot gene_set_scores ...
. This create several two-column lists for each of the clusters which can than be used in the script
wot cells_by_gene_set --score LIST_of_SCORES ...
But unfortunately this is not working anymore, as the script complains about missing files:
wot cells_by_gene_set --score gene_set_scores.txt_Cluster10.txt --score gene_set_scores.txt_Cluster11.txt ... --score gene_set_scores.txt_Cluster9.txt --out cell_sets_scores.gmt
usage: cells_by_gene_set.py [-h] --matrix MATRIX --gene_sets GENE_SETS
[--out OUT] [--format {gmt,gmx,txt}]
[--quantile QUANTILE]
cells_by_gene_set.py: error: the following arguments are required: --matrix, --gene_sets
Is there a way to test this way of calculating cell-sets? Why was this function defunct?
Hi WOT team,
I am trying to compute the log odds in the same way you did it for the paper. It sounds trivial but I don't get it from your toy example. I can see you do some tricks for setting up colors but I don't understand how to explicitly obtain log(p/1-p) for two trajectories. Could you please elaborate on how to do it in real data?
Thanks!
Ivan
I was wondering if there is an option to visualize the results/calculation in the loom files?
I understand the the loom files are a simple matrix of TP1xTP2 dimensions.
Is there a way to create a heatmap from this data?
Would it make sense at all to visualize such data? What can I gain from viewing the heatmap?
Hi, what happened to the script 02_plotting_two_features.py
?
it disappeared from the repository and running it now causes an error
$ python 02_plotting_two_features.py
Traceback (most recent call last):
File "02_plotting_two_features.py", line 18, in <module>
wot.io.add_row_metadata_to_dataset(ds, days_path=days_file)
TypeError: add_row_metadata_to_dataset() got an unexpected keyword argument 'days_path'
EDIT
the same error apears when running the script 03_plotting_cell_sets.py
EDIT2
I tried to change the days_path
parameters to days
.
It just gave me the next error
Traceback (most recent call last):
File "03_plotting_cell_sets.py", line 33, in <module>
cell_sets = wot.get_cells_in_gene_sets(gene_sets, ds,
AttributeError: module 'wot' has no attribute 'get_cells_in_gene_sets'
Originally posted by @frymor in #37 (comment)
I have created the necessary files. But when running the cells_by_gene_set
command I get no cells in the file.
the command I use is
wot cells_by_gene_set --matrix matrix.transposed.txt --gene_sets gene_set.gmt --out cell_sets.gmt --format gmt --quantile 0.5
As the files are very big, I copy-pasted here only a small subset of it. I hope it is enough.
the matrix
looks like that:
id Xkr4 Mrpl15 Lypla1 Tcea1 Rgs20 Atp6v1h ...
cell_143_AAACCTGAGGCTAGGT -0.219520647107675 -0.729889404055677 -0.389549050946858 1.35781559781124 -0.254530089148578 2.28680657293266 ...
cell_143_AAACGGGCAATCGGTT -0.219520647107675 -0.729889404055677 -0.389549050946858 -0.638558866199748 -0.254530089148578 -0.640469100310059 ...
cell_143_AAAGTAGCATAAAGGT -0.219520647107675 -0.729889404055677 -0.389549050946858 -0.638558866199748 -0.254530089148578 -0.640469100310059 ...
cell_143_AAATGCCGTGTGTGCC -0.219520647107675 -0.729889404055677 -0.389549050946858 -0.638558866199748 -0.254530089148578 -0.640469100310059 ...
...
the gene_set.gmt
files is like that:
1 cluster Igfbpl1 Ccnd2 Tubb3 Cd24a Sox11 Dlx6os1 Arx ...
2 cluster Igf1 Cnr1 Crh Snhg11 Syt1 ...
3 cluster Cspg5 Ednrb Pdgfra Cdo1 Emid1 ...
In total I have 18 different clusters containing the genes in the gene_set.gmt
file.
But when I'm running the command it create a file with this results
1 -
2 -
3 -
4 -
5 -
6 -
7 -
8 -
9 -
10 -
11 -
12 -
13 -
14 -
15 -
16 -
17 -
18 -
any ideas what wrong here?
a few assumptions I'm making now:
thanks
Hello Joshua,
I'm a postdoc working on engineered T cells at Stanford university and hope to use WOT.
I've generated maps from the attached data matrix and days file using the following command:
wot optimal_transport --matrix matrix.txt --cell_days days.txt --local_pca 0 --out tmaps
However, I can't seem to solve the error in calculating the trajectory generated by this command (screenshot attached):
wot trajectory --tmap tmaps --cell_set naive.gmt --day 10 --out wot_trajectory.txt
I'm also attaching the gmt file used for this command that contains the following information.
Tip1 naive CD8_HA_D0_213 CD8_HA_D0_298 CD8_HA_D0_362 CD8_HA_D0_413 CD8_HA_D0_561 CD8_HA_D0_722 CD8_HA_D0_798 CD8_HA_D0_829 CD8_HA_D0_893 CD8_HA_D0_896
The issue appears to be with getting information out of the loom files. Could ou please help?
Thanks in advance,
Zina
The 0.52 version fixed most of bugs in simulated data, but the last one (aka Plotting validation summary) still has a bug as attached.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-9-14f90d916de1> in <module>()
14 ot_model = wot.ot.initialize_ot_model(matrix_file, days_file,
15 covariate=covariate_file, growth_iters=1, tmap_prefix='val')
---> 16 vs = wot.commands.compute_validation_summary(ot_model)
17 vs['time'] = (vs['interval_start'] + vs['interval_end']) / 2
18 vs['type'] = vs['pair0'].astype(str).str[0]
~/.local/lib/python3.6/site-packages/wot/commands/optimal_transport_validation.py in compute_validation_summary(ot_model, day_pairs_triplets, save_interpolated, interp_size, compute_full_distances)
55
56 if has_covariate:
---> 57 ot_model.compute_all_transport_maps(with_covariates=True)
58 if compute_full_distances:
59 ot_model.compute_all_transport_maps()
~/.local/lib/python3.6/site-packages/wot/ot/ot_model.py in compute_all_transport_maps(self, with_covariates)
218 else:
219 for x in day_pairs:
--> 220 self.compute_transport_map(*x)
221
222 def compute_transport_map(self, t0, t1, covariate=None):
~/.local/lib/python3.6/site-packages/wot/ot/ot_model.py in compute_transport_map(self, t0, t1, covariate)
264
265 config = {**self.ot_config, **local_config, 't0': t0, 't1': t1, 'covariate': covariate}
--> 266 tmap = OTModel.compute_single_transport_map(self.matrix, config)
267 if tmap is not None:
268 wot.io.write_dataset(tmap, output_file, output_format=self.output_file_format)
~/.local/lib/python3.6/site-packages/wot/ot/ot_model.py in compute_single_transport_map(ds, config)
341 delta_days = t1 - t0
342 config['g'] = config['g'] ** delta_days
--> 343 tmap = wot.ot.transport_stable_learn_growth(C, **config)
344 return anndata.AnnData(tmap, p0.obs.copy(), p1.obs.copy())
TypeError: transport_stable_learn_growth() got an unexpected keyword argument 'tmap_prefix'
hi,
when I run the tutorial with the example, ot_model.compute_all_transport_maps()
produce an error
TypeError: transport_stable_learn_growth() missing 1 required positional argument: 'scaling_iter'
I find that there is no default value of scaling_iter
in transport_stable_learn_growth()
. but when I adding 3000 to scaling_iter
, another error emergence
~/anaconda3/envs/zachary35/lib/python3.5/site-packages/wot/ot/optimal_transport.py in transport_stablev2(C, lambda1, lambda2, epsilon, scaling_iter, g, pp, qq, numInnerItermax, tau, epsilon0, extra_iter)
277 # stabilization
278 iterations_since_epsilon_adjusted += 1
--> 279 if (max(max(abs(a)), max(abs(b))) > tau):
280 u = u + epsilon_i * np.log(a)
281 v = v + epsilon_i * np.log(b) # absorb
TypeError: unorderable types: float() > NoneType()
My WOT version is 0.2.2.
Also, I noticed that WOT version 0.2.0 is ok when running ot_model.compute_all_transport_maps()
.
Thankyou
Im analysing a big data set from 10x Genomics data.
to make life easier I created a subset with "only" three time points. my gene matrix contains 10482 cells with 18418 genes.
When running the command:
wot cells_by_gene_set --matrix matrix --gene_sets gene_set.gmt --out cell_sets.gmt --format gmt --quantile 0.99
I get no results, but if I sub-set the file to contain only a sample of the genes I can get a cell-set, but all (~99%, with some few exceptions) of them are grouped in one cluster). I did this randomly with different number of cells and genes and each time a different number of cells is clustered in this one cluster, but not in other cluster.
I was wondering if there is a reason for this kind of behavior, or whether or not this is a computational problem.
I would appreciate your help. If needed I can also send the original files.
thanks
Assa
P.S.
Just an afterthought.
To make sure that the missing results has nothing to do with the data-set structure, I would like to mention, that the gene-sets used to run the above command were created from only one of the time-points. The list of unique genes in gene-sets contains 13710 genes in 18 clusters, but only 6189 unique entries, while the list of genes in the matrix has 18417 genes. So obvious not all genes from the gene matrix are in the gene-set.
Another thought I had - should the gene matrix used as input be in any way normalized or scaled? We are using for input data from the Seurat package. Before inputting it to wot we normalize and scale the data matrix.
After installing version 1.0.2 and running the command
wot optimal_transport --matrix data/ExprMatrix.h5ad --cell_days data/cell_days.txt --out tmaps1.0.2/tmaps1.0.2
I get the following error
Traceback (most recent call last):
File "/home/yeroslaviz/miniconda3/envs/wot1.0/bin/wot", line 6, in <module>
from wot.__main__ import main
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/wot/__init__.py", line 3, in <module>
import wot.io
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/wot/io/__init__.py", line 2, in <module>
from .io import *
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/wot/io/io.py", line 5, in <module>
import anndata
File "/home/yeroslaviz/.local/lib/python3.7/site-packages/anndata/__init__.py", line 1, in <module>
from .base import AnnData
File "/home/yeroslaviz/.local/lib/python3.7/site-packages/anndata/base.py", line 21, in <module>
from scipy.sparse.sputils import IndexMixin
I have bin able to temporarily fix it by downgrading scipy
to version 1.2.1 inside my conda environment.
pip install scipy==1.2.1
I'm excited to try out WOT on some public data! But, I ran into an obstacle when testing modifications of notebook 4 on a small subset of the data.
tmap_model.trajectories
seems to only work when 3 or more time-points are present. You can verify this in notebook 4 by deleting all but the 17.5-18.0 map. The error with only these two time-points is ValueError: No cells found at the given day
, which is confusing because there are plenty of 18.0 cells in the relevant Population
object. It would be nice to have an informative error message when only two time points are present.
This may be related to #25 .
I have updated to version 1.0.1. Thanks for that.
When trying to run again the command line examples, I get an error, when trying to calculate the ancestor census.
$wot census --tmap tmaps --cell_set cell_sets_scores.gmt --out census --day 4
Traceback (most recent call last):
File "/home/yeroslaviz/miniconda3/envs/wot1.0/bin/wot", line 11, in <module>
load_entry_point('wot', 'console_scripts', 'wot')()
File "/home/yeroslaviz/projects/AnnB/TestingEnv/wot1.0.0/wot/wot/__main__.py", line 21, in main
cmd.main(command_args)
File "/home/yeroslaviz/projects/AnnB/TestingEnv/wot1.0.0/wot/wot/commands/census.py", line 28, in main
timepoints, census = tmap_model.ancestor_census(cell_sets_matrix, *populations.values())
AttributeError: 'list' object has no attribute 'values'
Hi,
Running gene_set_scores returns:
TypeError: score_gene_sets() got an unexpected keyword argument 'verbose'
WOT v1.01 (5/13/19)
Hi,
I'm trying to re-calculate transport maps via the command line using WOT v1.0 (installed via pip 5/13/19), but it returns a type error. From what I can see, the argument is specified in optimal_transport.py but not in /ot/ot_model.py.
File "/miniconda2/envs/waddingtonOT/bin/wot", line 10, in
sys.exit(main())
File "/miniconda2/envs/waddingtonot/lib/python3.7/site-packages/wot/main.py", line 21, in main
cmd.main(command_args)
File "/miniconda2/envs/waddingtonot/lib/python3.7/site-packages/wot/commands/optimal_transport.py", line 27, in main
tmap_out=args.out)
TypeError: compute_all_transport_maps() got an unexpected keyword argument 'no_overwrite'
Hey, could you please explain why this error comes. I am new to programming,so as far i understand the error is coming from matric file.
wot trajectory_trends --trajectory tmaps/traj.txt --matrix matrix2.txt --cell_days days.txt --out trends
/home/brightann/.local/lib/python3.7/site-packages/wot/io/io.py:786: FutureWarning: read_table is deprecated, use read_csv instead.
engine='python', sep=None, dtype={'day': np.float64})
Traceback (most recent call last):
File "/home/brightann/.local/bin/wot", line 11, in
sys.exit(main())
File "/home/brightann/.local/lib/python3.7/site-packages/wot/main.py", line 23, in main
cmd.main(command_args)
File "/home/brightann/.local/lib/python3.7/site-packages/wot/commands/trajectory_trends.py", line 24, in main
matrix = wot.io.read_dataset(args.matrix)
File "/home/brightann/.local/lib/python3.7/site-packages/wot/io/io.py", line 584, in read_dataset
np_arrays.append(np.array(tokens[1:], dtype=np.float64))
ValueError: could not convert string to float: 'X_gene'
Hi WOT developers,
I am experiencing some problems with diff_exp after updating wot. I am able to compute the transport maps and the fates matrices but I now get an error regarding data dimensions that I think is related to changes in the anndata object but I just don't know how to fix it.
The error message is:
9 days
BP1 vs BP2, day 6.5, day 6.5
Trying to set attribute .X
of view, making a copy.
Traceback (most recent call last):
File "/homes/iimaz/miniconda3/bin/wot", line 8, in
sys.exit(main())
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/wot/main.py", line 23, in main
cmd.main(args)
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/wot/commands/diff_exp.py", line 71, in main
compare=compare, delta_days=delta_days)
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/wot/tmap/diff_exp.py", line 77, in diff_exp
features=features)
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/wot/tmap/diff_exp.py", line 114, in __do_comparison
scores[np.isnan(scores)] = 0
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/anndata/core/views.py", line 29, in setitem
container[idx] = value
IndexError: boolean index did not match indexed array along dimension 0; dimension is 2661 but corresponding boolean dimension is 5469
The number 5469 corresponds to the number of genes in my matrix but I have no idea what is the 2661 number. I tried different gene sets and the same error is produced.
Do you have any clue about what might be happening? is the latest version of anndata likely the problem?
Is there any other information I could share with you to help you to figure out this issue?
Many thanks,
Ivan
PS. I am really happy with this tool. It has helped me a lot in my project.
Hi,
I think a file is missing here, either it was not added to the repo or it was moved and its
location was not updated.
The issue occurs when running
wot gene_set_scores --matrix <matrixfile>
In wot/commands/gene_set_scores.py, on line 55,
the gene_sets variable defaults to 'wot/resources/growth_scores_gene_sets.gmt'.
However, that file is not present, which triggers a FileNotFoundError at runtime.
Thanks !
Hi WOT team,
wot neighborhood_graph is supposed to produce .csv and .h5ad files that is used for following steps, but now it produces .gexf file instead, and there is no document about how to use it.
Thanks!
Hi,
After resolution of the previous TypeError with scoring gene sets, I encountered the following error with both test files and my own data both using API and the command line interface (this is after updating the gene_set_scores file pushed to master earlier). From what I can see, my guess would be a bug in the read_sets function.
/miniconda2/envs/waddingtonot/lib/python3.7/site-packages/wot/gene_set_scores.py:101: RuntimeWarning: invalid value encountered in true_divide
x = (x - mean) / std
Hi,
I'm really sure if this is correct to be posted here, but I'll give it a try anyway.
I'm trying to understand the algorithm and work my way through the examples. In the first example by generating the data you offer a method to visualise the data, but the script offers to plot only two features. What does it mean?
What two features of the data are plotted here? Does it means two columns of the matrix.txt data set?
Is there a way to plot more than just these two features?
thanks
In the new version you calculate the cell sets over gene scores.
How are these scores being calculated?
Hi WOT developers! In your Cell paper, it looks like the mathematical supplement here seems to have a lot of mis-rendered LaTeX. You may want to check with the journal and see if something went wrong. Here's a screenshot.
Thanks for putting this together; it looks like a titanic amount of work!
wot trajectory_trends --trajectory traj.txt --matrix matrix.txt --out trends --cell_days days.txt
/home/brightann/.local/lib/python3.7/site-packages/wot/io/io.py:786: FutureWarning: read_table is deprecated, use read_csv instead.
engine='python', sep=None, dtype={'day': np.float64})
Traceback (most recent call last):
File "/home/brightann/.local/bin/wot", line 11, in
sys.exit(main())
File "/home/brightann/.local/lib/python3.7/site-packages/wot/main.py", line 23, in main
cmd.main(command_args)
File "/home/brightann/.local/lib/python3.7/site-packages/wot/commands/trajectory_trends.py", line 26, in main
results = wot.tmap.compute_trajectory_trends_from_trajectory(trajectory_ds, matrix)
File "/home/brightann/.local/lib/python3.7/site-packages/wot/tmap/transport_map_util.py", line 78, in compute_trajectory_trends_from_trajectory
for j in range(trajectory_ds.X.shape[1]):
IndexError: tuple index out of range
In the Plotting trajectory trends part, https://broadinstitute.github.io/wot/notebook/, error comes when I run the script:
AttributeError: module 'wot.ot' has no attribute 'compute_trajectory_trends_from_trajectory'
After I searched the source code, I found that wot.ot should be wot.tmap to get compute_trajectory_trends_from_trajectory function.
However, after fixing that, a new error comes
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-20-b60afaa2d212> in <module>()
24 # timepoints, means, variances = tmap_model.compute_trajectory_trends(ds, population)
25 trajectory_ds = tmap_model.compute_trajectories({target_cell_set: all_populations[target_cell_set]})
---> 26 results = wot.tmap.compute_trajectory_trends_from_trajectory(trajectory_ds, ds)
27 means, variances = results[0]
28 timepoints = means.obs.index.values
~/.local/lib/python3.6/site-packages/wot/tmap/transport_map_util.py in compute_trajectory_trends_from_trajectory(trajectory_ds, ds)
76 mean_list = []
77 variance_list = []
---> 78 for j in range(trajectory_ds.X.shape[1]):
79 mean_list.append(None)
80 variance_list.append(None)
IndexError: tuple index out of range
The shape of trajectory_ds.X is (153545,), there is no second dimension. I am not able to fix this issue yet.
I would like to install wot but can't install. I get the following error message.
pip install wot
Collecting wot
Could not find a version that satisfies the requirement wot (from versions: )
No matching distribution found for wot
note: I have Python 3.6.8 (default, Oct 7 2019, 12:59:55)
Thanks!
when running the command
wot trajectory --tmap tmaps1.0.2 --cell_set Output1.0.2/cell_sets_scores.gmt --day 10 --out traj_
I get the following warning (?)
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Can it be ignored? How can I avoid it or modify the inout file, so that it would not show again?
where does it look for/find duplicated entries?
By manually looking at the file I couldn't find any duplicated gene entries
thanks
Thanks for providing the codes.
I was wondering if you provide Jupiter notebooks that take the real data in the paper, and produce corresponding figures. This would be helpful for us to learn how to play with our own data. I only found simulated data example in this repo and document. Thanks!
Hi, in notebook 1 you offer the possibility to calculate gene scores, as well as to work with the pre-defined cell-sets.
What if I have different gene-sets, i have calculated the gene scores for?
Is it possible to add an option to calculate also new cel sets?
Calculating the validation throws back an error:
$ wot optimal_transport_validation --matrix matrix.txt --cell_days days.txt --covariate covariate.txt
local_pca set to 30, above gene count of 7. Disabling PCA
Traceback (most recent call last):
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2657, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'full'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/yeroslaviz/miniconda3/envs/wot1.0/bin/wot", line 11, in <module>
load_entry_point('wot', 'console_scripts', 'wot')()
File "/home/yeroslaviz/projects/AnnB/TestingEnv/wot1.0.0/wot/wot/__main__.py", line 21, in main
cmd.main(command_args)
File "/home/yeroslaviz/projects/AnnB/TestingEnv/wot1.0.0/wot/wot/commands/optimal_transport_validation.py", line 73, in main
summary_stats = summary[summary['full'] == False]
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/pandas/core/frame.py", line 2927, in __getitem__
indexer = self.columns.get_loc(key)
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2659, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'full'
I cannot find the "wot" in environment after "pip install --user wot" was done.
Hi there,
I was looking into the code to read in data (read_anndata function) - in the case of reading from a matrix file, as it would be if reading the output of the cellranger pipeline, the gene-level annotation file is now called features.tsv.gz, instead of genes.tsv.gz . It would be great to modify the function to look for that as well, instead of throwing an error (it's an easy fix either way, but wanted to point it out!).
Thanks for developing a great trajectory analysis tool!
Gabi
In the first script from http://broadinstitute.github.io/wot/notebook/all_scripts.zip, the last lines are
wot.io.write_dataset_metadata(ds.row_meta, days_file, 'day')
wot.io.write_dataset_metadata(ds.row_meta, covariate_file, 'covariate')
The error will come as ds has no attribution: row_meta
. While in other codes provided in your GitHub, the last lines are
wot.io.write_dataset_metadata(ds.obs, days_file, 'day')
wot.io.write_dataset_metadata(ds.obs, covariate_file, 'covariate')
, which works fine.
Thanks for developing such nice analysis tool.When I run
wot optimal_transport --matrix my_matrix_file_path --cell_days my_cell_day_file_path--out my_tmaps
but I got the error "Days information missing for (the cell number) cells
No day pairs"
I got the two files form as follows
https://broadinstitute.github.io/wot/cli_documentation/#matrix_file
I do no know what is the problem,I got stuck in this for few days,can you help with this?
Thanks in advance!
Hello!
First, thank you for developing a great pseudotime tool.
We have a time course: day 0, 8, 15, 30. From prior knowledge, we know there are terminal cells at day 8, 15, and 30. There should be branding points that that end at day 8 and others that go on to the later timepoints. When we calculate wot trajectory, all cells at day 8 have an intermediate trajectory value.
Is it possible for this method to detect terminal cells at intermediate time points? Is there any way to indicate this in function arguments?
Thank you!!!!
Jo
Hi,
I am trying to install wot using python 3.6. For some reason after installing the dependencies when I try to
pip install wot
I get an error
Collecting wot
Could not find a version that satisfies the requirement wot (from versions: )
No matching distribution found for wot
Any help would be great!
Hi,
I am trying to install WOT using pip on my Mac OSX but I am getting this error. I have xcode and command line tools installed already.
ERROR: Failed building wheel for POT
Running setup.py clean for POT
Failed to build POT
Installing collected packages: POT, wot
Running setup.py install for POT ... error
ERROR: Complete output from command /Users/bibaswan/anaconda3/bin/python -u -c 'import setuptools, tokenize;file='"'"'/private/var/folders/lv/26lkh7m902n85b7k_m_0gz_80000gn/T/pip-install-6zgo6l_s/POT/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/lv/26lkh7m902n85b7k_m_0gz_80000gn/T/pip-record-9h8k9k_e/install-record.txt --single-version-externally-managed --compile:
ERROR: running install
running build
running build_py
creating build
creating build/lib.macosx-10.7-x86_64-3.7
creating build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/bregman.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/plot.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/stochastic.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/datasets.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/init.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/dr.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/utils.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/smooth.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/gromov.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/optim.py -> build/lib.macosx-10.7-x86_64-3.7/ot
copying ot/da.py -> build/lib.macosx-10.7-x86_64-3.7/ot
creating build/lib.macosx-10.7-x86_64-3.7/ot/gpu
copying ot/gpu/bregman.py -> build/lib.macosx-10.7-x86_64-3.7/ot/gpu
copying ot/gpu/init.py -> build/lib.macosx-10.7-x86_64-3.7/ot/gpu
copying ot/gpu/utils.py -> build/lib.macosx-10.7-x86_64-3.7/ot/gpu
copying ot/gpu/da.py -> build/lib.macosx-10.7-x86_64-3.7/ot/gpu
creating build/lib.macosx-10.7-x86_64-3.7/ot/lp
copying ot/lp/init.py -> build/lib.macosx-10.7-x86_64-3.7/ot/lp
copying ot/lp/cvx.py -> build/lib.macosx-10.7-x86_64-3.7/ot/lp
creating build/lib.macosx-10.7-x86_64-3.7/ot/externals
copying ot/externals/funcsigs.py -> build/lib.macosx-10.7-x86_64-3.7/ot/externals
copying ot/externals/init.py -> build/lib.macosx-10.7-x86_64-3.7/ot/externals
running build_ext
building 'ot.lp.emd_wrap' extension
creating build/temp.macosx-10.7-x86_64-3.7
creating build/temp.macosx-10.7-x86_64-3.7/ot
creating build/temp.macosx-10.7-x86_64-3.7/ot/lp
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/bibaswan/anaconda3/include -arch x86_64 -I/Users/bibaswan/anaconda3/include -arch x86_64 -Iot/lp -I/Users/bibaswan/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/private/var/folders/lv/26lkh7m902n85b7k_m_0gz_80000gn/T/pip-install-6zgo6l_s/POT/ot/lp -I/Users/bibaswan/anaconda3/include/python3.7m -c ot/lp/emd_wrap.cpp -o build/temp.macosx-10.7-x86_64-3.7/ot/lp/emd_wrap.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
In file included from /Users/bibaswan/anaconda3/lib/gcc/x86_64-apple-darwin11.4.2/4.8.5/include-fixed/syslimits.h:7:0,
from /Users/bibaswan/anaconda3/lib/gcc/x86_64-apple-darwin11.4.2/4.8.5/include-fixed/limits.h:34,
from /Users/bibaswan/anaconda3/include/python3.7m/Python.h:11,
from ot/lp/emd_wrap.cpp:28:
/Users/bibaswan/anaconda3/lib/gcc/x86_64-apple-darwin11.4.2/4.8.5/include-fixed/limits.h:168:61: fatal error: limits.h: No such file or directory
#include_next <limits.h> /* recurse down to the real one */
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command "/Users/bibaswan/anaconda3/bin/python -u -c 'import setuptools, tokenize;file='"'"'/private/var/folders/lv/26lkh7m902n85b7k_m_0gz_80000gn/T/pip-install-6zgo6l_s/POT/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/lv/26lkh7m902n85b7k_m_0gz_80000gn/T/pip-record-9h8k9k_e/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/lv/26lkh7m902n85b7k_m_0gz_80000gn/T/pip-install-6zgo6l_s/POT/
Any help would be awesome.
Hello. I've tried to run wot on my data, and am getting a rather confusing error when attempting to infer the trajectory:
ubuntu@mapcloud1:/mnt/endometrium/N6.5-organoids-split$ wot trajectory --tmap tmaps --cell_set clusters.gmt --time 3 --out traj.txt
/usr/local/lib/python3.6/dist-packages/numba/errors.py:104: UserWarning: Insufficiently recent colorama version found. Numba requires colorama >= 0.3.9
warnings.warn(msg)
Traceback (most recent call last):
File "/usr/local/bin/wot", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/wot/__main__.py", line 23, in main
cmd.main(command_args)
File "/usr/local/lib/python3.6/dist-packages/wot/commands/trajectory.py", line 23, in main
trajectory_ds = tmap_model.compute_trajectories(populations)
File "/usr/local/lib/python3.6/dist-packages/wot/tmap/transport_map_model.py", line 97, in compute_trajectories
var=pd.DataFrame(index=population_names))
File "/usr/local/lib/python3.6/dist-packages/anndata/base.py", line 671, in __init__
filename=filename, filemode=filemode)
File "/usr/local/lib/python3.6/dist-packages/anndata/base.py", line 880, in _init_as_actual
self._check_dimensions()
File "/usr/local/lib/python3.6/dist-packages/anndata/base.py", line 1878, in _check_dimensions
.format(self._n_obs, self._obs.shape[0]))
ValueError: Observations annot. `obs` must have number of rows of `X` (23071), but has 28789 rows.
The 23071 looks familiar, as that's the number of cells I provided in my original matrix. The 28789 I don't recognise at all. Any idea what is going on here?
Error:
File "08_plotting_validation_summary.py", line 15 in
vs = wotcommands.compute_vvalidation_summary(ot_model)
TypeError: compute_validation_summary() missing 1 required positional argument: 'day_pairs_triplets'
The function in the provided example code is not using a required parameter.
Hi, I am trying to run wot on the simulated data provided with the tutorial. But i keep getting this error. Can you please help on why this error is occurring. I wanted to first run the commands on this dataset and apply it on our dataset.
wot trajectory --tmap tmaps --cell_set cell_sets.gmt --time 10 --out traj.txt
Traceback (most recent call last):
File "/home/brightann/.local/bin/wot", line 11, in
sys.exit(main())
File "/home/brightann/.local/lib/python3.7/site-packages/wot/main.py", line 23, in main
cmd.main(command_args)
File "/home/brightann/.local/lib/python3.7/site-packages/wot/commands/trajectory.py", line 23, in main
trajectory_ds = tmap_model.compute_trajectories(populations)
File "/home/brightann/.local/lib/python3.7/site-packages/wot/tmap/transport_map_model.py", line 71, in compute_trajectories
while self.can_pull_back(*populations):
File "/home/brightann/.local/lib/python3.7/site-packages/wot/tmap/transport_map_model.py", line 187, in can_pull_back
return self.timepoints.index(wot.tmap.unique_timepoint(*populations)) > 0
File "/home/brightann/.local/lib/python3.7/site-packages/wot/tmap/util.py", line 24, in unique_timepoint
raise ValueError("Empty population list")
ValueError: Empty population list
when I validate the transport maps as https://broadinstitute.github.io/wot/cli_documentation/#matrix_file
,but I can not get the covariate file,what is it? And how can I get it?
Thanks in advance for help.
Hi WOT team,
Thanks for this awesome tool. I am finding a problem when computing the trajectory trends after running the trajectory tool with no issues (at least the output looks normal).
The entire pipeline used to work fine. Then a friend help me with some other python package and I guess he changed something and now the trajectory trend command is crashing. I have reinstalled everything as well as updated wot but the error is still there. In summary the same input that used to work is not doing it anymore. Do you recognise this error:
wot trajectory_trends --trajectory traj.txt
--matrix matrix.mtx
--out trends --cell_days time.txt
Traceback (most recent call last):
File "/homes/iimaz/.local/bin/wot", line 11, in
sys.exit(main())
File "/homes/iimaz/.local/lib/python3.6/site-packages/wot/main.py", line 23, in main
cmd.main(command_args)
File "/homes/iimaz/.local/lib/python3.6/site-packages/wot/commands/trajectory_trends.py", line 24, in main
matrix = wot.io.read_dataset(args.matrix)
File "/homes/iimaz/.local/lib/python3.6/site-packages/wot/io/io.py", line 472, in read_dataset
header=None)
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/pandas/io/parsers.py", line 702, in parser_f
return _read(filepath_or_buffer, kwds)
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/pandas/io/parsers.py", line 429, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/pandas/io/parsers.py", line 895, in init
self._make_engine(self.engine)
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/pandas/io/parsers.py", line 1122, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "/homes/iimaz/miniconda3/lib/python3.6/site-packages/pandas/io/parsers.py", line 1853, in init
self._reader = parsers.TextReader(src, **kwds)
File "pandas/_libs/parsers.pyx", line 545, in pandas._libs.parsers.TextReader.cinit
pandas.errors.EmptyDataError: No columns to parse from file
Best,
Ivan
in calculating the Trajectories:
--out
should be modified to be the output file prefix.--out traj.txt
, it creates a file with the term as a prefix, but still adds the trajectory.txt to create a file named traj.txt_trajectory.txt
Calculating the trajectory trends:
--cell_days
.:
and spaces in the names of both the output and the plots. This is the command i used:wot trajectory_trends --trajectory traj.txt_trajectory.txt --matrix matrix.txt --out trends --cell_days days.txt --plot
. The files are get are 'trends_Unnamed: 1.mean.txt'
and 'trends_Unnamed: 1.png'
.
When trying to calculate the gene scores with the new version (1.0.2), I encounter a few problems.
An error creating the gene-set scores appears when trying to get a gct
file format.
when running the command
wot gene_set_scores --matrix oldData/matrix.txt --gene_sets oldData/gene_sets.gmt --out gene_set --format gct --method mean_z_score
I get the following error:
Traceback (most recent call last):
File "/home/yeroslaviz/miniconda3/envs/wot1.0/bin/wot", line 10, in <module>
sys.exit(main())
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/wot/__main__.py", line 21, in main
cmd.main(command_args)
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/wot/commands/gene_set_scores.py", line 100, in main
path=name, output_format=args.format)
File "/home/yeroslaviz/miniconda3/envs/wot1.0/lib/python3.7/site-packages/wot/io/io.py", line 660, in write_dataset
'\t' + str(len(ds.var.columns)) + '\n')
Another request would be to maybe create all the gene sets in just one file. This would make runnging the cell-set calculation much easier and streamlined.
thanks Assa
Hi WOT devs! Very impressive work with this tool. The documentation in notebook 4 implies that trajectory inference should work forwards and backwards: "The method trajectories takes a list of probability vectors and returns the trajectories containing descendant distributions at later time points and ancestor distributions at earlier time points." But, if you edit the notebook to focus on a middle time-point, for instance by changing at_time=18
to at_time=17
when constructing populations, trajectory inference yields an object with NaN
's in the variable metadata, and it cannot be used downstream as expected.
It may be an easy fix: the issue seems to go away if you fill in the NaNs with the appropriate names, i.e.
trajectory_ds.var.index = [p.name for p in populations]
I'm not sure if this is related, but the probability vectors are also not as advertised: their sums are >>1 .
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.