Comments (13)
francois-a
commented on August 25, 2024
Hi,
Can you please try to run the latest commit? This should be fixed here. I'll make a new release shortly.
from tensorqtl.
aarbduarte
commented on August 25, 2024
Done, retrieve a different error.
`[Mar 09 15:46:02] Running TensorQTL: cis-QTL mapping
* WARNING: using CPU!
* using seed 123456789
* reading phenotypes (/home/aduarte/files_map/phenotypes_22.bed.gz)
* reading covariates (/home/aduarte/files_map/covariates.txt)
Mapping files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:12<00:00, 4.02s/it]
cis-QTL mapping: empirical p-values for phenotypes
* 83 samples
* 72424 phenotypes
* 22 covariates
* 11548697 variants
* using seed 123456789
** dropping 464 constant phenotypes
* checking phenotypes: 72424/72424
** dropping 5 phenotypes without variants in cis-window
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/aduarte/tensorqtl/tensorqtl/__main__.py", line 2, in <module>
tensorqtl.main()
File "/home/aduarte/tensorqtl/tensorqtl/tensorqtl.py", line 109, in main
logger=logger, seed=args.seed, verbose=True)
File "/home/aduarte/tensorqtl/tensorqtl/cis.py", line 460, in map_cis
igc = genotypeio.InputGeneratorCis(genotype_df, variant_df, phenotype_df, phenotype_pos_df, group_s=group_s, window=window)
File "/home/aduarte/tensorqtl/tensorqtl/genotypeio.py", line 398, in __init__
self.phenotype_df = self.phenotype_df.loc[valid_ix]
File "/home/aduarte/tensorqtl/venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 1767, in __getitem__
return self._getitem_axis(maybe_callable, axis=axis)
File "/home/aduarte/tensorqtl/venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 1953, in _getitem_axis
return self._getitem_iterable(key, axis=axis)
File "/home/aduarte/tensorqtl/venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 1594, in _getitem_iterable
keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
File "/home/aduarte/tensorqtl/venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 1552, in _get_listlike_indexer
keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
File "/home/aduarte/tensorqtl/venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 1654, in _validate_read_indexer
"Passing list-likes to .loc or [] with any missing labels "
KeyError: 'Passing list-likes to .loc or [] with any missing labels is no longer supported, see https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike'
`
from tensorqtl.
francois-a
commented on August 25, 2024
It's hard to tell from this, but what do your phenotype IDs look like in phenotypes_22.bed.gz? Are there any missing entries?
from tensorqtl.
aarbduarte
commented on August 25, 2024
I'm using GTEx data so my samples are named GTEx-XXXX-YYYY-aa-bbbb. These are the collumn names on the phenotypes file:
#Chr Start End Id GTEX-1122O-0004-SM-5DWSJ GTEX-11EM3-0004-SM-5DWTY GTEX-11EMC-0004-SM-5DWTM [...]
from tensorqtl.
francois-a
commented on August 25, 2024
I'm asking about the phenotype IDs in phenotypes_22.bed.gz -- these should be the entries in the Id column of the file -- and specifically, whether there are any missing/NA entries.
from tensorqtl.
aarbduarte
commented on August 25, 2024
No missing, NA or duplicate IDs, they are however very long maxing at 92 chars long
from tensorqtl.
francois-a
commented on August 25, 2024
ok. are you able to reproduce the error with a small subset of your input files that you could share? The length should not be an issue.
from tensorqtl.
aarbduarte
commented on August 25, 2024
Unfortunately I can't show the data but could it be due to having the genotype format as GT:GL:DS ?
from tensorqtl.
francois-a
commented on August 25, 2024
No that should be fine -- the PLINK conversion will parse the GT field, and the variants seem to have loaded fine. The error indicates that this is an issue with the phenotypes file: self.phenotype_df.loc[valid_ix], but I haven't had a chance to reproduce this.
from tensorqtl.
aarbduarte
commented on August 25, 2024
Trying to solve it, I subset my data removing the rows that contained NA's. Now getting a different error:
aduarte@leia:~/files_map$ python3 -m tensorqtl ~/files_map/genotypes22.vcf.gz ~/files_map/phenotypes_6.bed.gz results --covariates ~/files_map/covariates.txt.gz --mode cis_nominal --seed 123456789 --maf_threshold 0.1
[Mar 13 15:48:31] Running TensorQTL: cis-QTL mapping
* WARNING: using CPU!
* using seed 123456789
* reading phenotypes (/home/aduarte/files_map/phenotypes_6.bed.gz)
* reading covariates (/home/aduarte/files_map/covariates.txt.gz)
Mapping files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:11<00:00, 3.99s/it]
cis-QTL mapping: nominal associations for all variant-phenotype pairs
* 83 samples
* 29067 phenotypes
* 22 covariates
* 11548697 variants
* checking phenotypes: 29067/29067
* Computing associations
* Mapping chromosome chr1
processing phenotype 2534/29067 time elapsed: 0.32 min
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/__main__.py", line 2, in <module>
tensorqtl.main()
File "/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/tensorqtl.py", line 119, in main
group_s=None, window=args.window, output_dir=args.output_dir, logger=logger, verbose=True)
File "/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/cis.py", line 315, in map_nominal
if start < chr_res['maf']:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
The line in question is below, it doesn't seem to have anything special about it.
aduarte@leia:~/files_map$ zcat phenotypes_6.bed.gz |head -n 2535 | tail -n 1
chr1 248856422 248856373 SE_1_-_248856514_248856422_248856373_248855946_ZNF692 0.89010989010989 0.818181818181818 0.940677966101695 0.927927927927928 0.935483870967742 0.968 0.8 0.8125 0.952380952380952 0.929577464788732 0.890909090909091 0.941176470588235 0.847457627118644 0.789473684210526 0.869565217391304 0.9428571428571430.923371647509579 0.918032786885246 0.964705882352941 0.889908256880734 0.933014354066986 0.90990990990991 0.953125 0.907692307692308 0.950310559006211 0.862068965517241 1 0.957446808510638 0.978723404255319 0.944664031620553 0.933333333333333 0.929577464788732 0.855421686746988 0.921259842519685 0.9310344827586210.862068965517241 0.878048780487805 0.958762886597938 0.885714285714286 0.956284153005464 0.965811965811966 0.927152317880795 0.980582524271845 0.9444444444444440.975903614457831 0.934959349593496 0.927272727272727 0.851485148514851 0.952755905511811 0.944444444444444 0.934156378600823 0.949367088607595 0.9405940594059410.952380952380952 0.87037037037037 0.846153846153846 0.946308724832215 0.954022988505747 0.947368421052632 0.826086956521739 0.958762886597938 0.92 0.954887218045113 0.833333333333333 0.914285714285714 0.980645161290323 0.987421383647799 0.917241379310345 0.827586206896552 0.978947368421053 0.936170212765957 0.86046511627907 0.975308641975309 0.833333333333333 0.833333333333333 0.951219512195122 1 0.888888888888889 0.882352941176471 0.9375 0.962962962962963 0.862595419847328 0.969465648854962
from tensorqtl.
francois-a
commented on August 25, 2024
That bug has been fixed here. Please update to v1.0.3.
from tensorqtl.
aarbduarte
commented on August 25, 2024
Thank you, cis_nominal mode is working flawlessly but cis mode with permutation is still delivering the same error:
aduarte@leia:~$ tensorqtl ~/files_map/genotypes22.vcf.gz ~/files_map/phenotypes_6.bed.gz results --covariates ~/files_map/covariates.txt.gz --mode cis --seed 123456789 --maf_threshold 0.1 --permutations 1000
[Mar 14 16:16:50] Running TensorQTL: cis-QTL mapping
* WARNING: using CPU!
* using seed 123456789
* reading phenotypes (/home/aduarte/files_map/phenotypes_6.bed.gz)
* reading covariates (/home/aduarte/files_map/covariates.txt.gz)
Mapping files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:12<00:00, 4.02s/it]
cis-QTL mapping: empirical p-values for phenotypes
* 83 samples
* 29067 phenotypes
* 22 covariates
* 11548697 variants
* using seed 123456789
* checking phenotypes: 29067/29067
* computing permutations
processing phenotype 9/29067/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/core.py:217: RuntimeWarning: invalid value encountered in sqrt
return 2*stats.t.cdf(-np.abs(np.sqrt(tstat2)), dof)
/home/aduarte/.local/lib/python3.6/site-packages/scipy/stats/_distn_infrastructure.py:903: RuntimeWarning: invalid value encountered in greater
return (a < x) & (x < b)
/home/aduarte/.local/lib/python3.6/site-packages/scipy/stats/_distn_infrastructure.py:903: RuntimeWarning: invalid value encountered in less
return (a < x) & (x < b)
/home/aduarte/.local/lib/python3.6/site-packages/scipy/stats/_distn_infrastructure.py:1827: RuntimeWarning: invalid value encountered in greater_equal
cond2 = (x >= np.asarray(_b)) & cond0
WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 12/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 14/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 21/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 22/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 23/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 27/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 28/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 29/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 30/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 33/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 39/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 44/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 45/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 48/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 53/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 54/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 67/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 68/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 69/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 70/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 78/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 93/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 94/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 101/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 102/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 107/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 108/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 110/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 113/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 115/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 120/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 122/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 123/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 124/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 126/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 127/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 131/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 132/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 134/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 135/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 136/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 137/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 138/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 142/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 144/29067/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/core.py:216: RuntimeWarning: divide by zero encountered in true_divide
tstat2 = dof * r2 / (1 - r2)
WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/core.py:231: RuntimeWarning: divide by zero encountered in log
return (1.0-shape1)*np.sum(np.log(x)) + (1.0-shape2)*np.sum(np.log(1.0-x)) + len(x)*logbeta
processing phenotype 146/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 147/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 148/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 149/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 155/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 157/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 158/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 163/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 165/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 166/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 168/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 171/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 176/29067WARNING: scipy.optimize.newton failed to converge (running scipy.optimize.minimize)
processing phenotype 178/29067Traceback (most recent call last):
File "/home/aduarte/.local/bin/tensorqtl", line 7, in <module>
from tensorqtl import __main__
File "/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/__main__.py", line 2, in <module>
tensorqtl.main()
File "/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/tensorqtl.py", line 109, in main
logger=logger, seed=args.seed, verbose=True)
File "/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/cis.py", line 460, in map_cis
res = calculate_cis_permutations(genotypes_t, phenotype_t, residualizer, permutation_ix_t)
File "/home/aduarte/.local/lib/python3.6/site-packages/tensorqtl/cis.py", line 59, in calculate_cis_permutations
r2_perm_t,_ = corr_t[~torch.isnan(corr_t).any(1),:].max(0)
RuntimeError: cannot perform reduction function max on tensor with no elements because the operation does not have an identity
from tensorqtl.
francois-a
commented on August 25, 2024
It looks like you're still running an older version:
The second to last line in the log, r2_perm_t,_ = corr_t[~torch.isnan(corr_t).any(1),:].max(0), should look like this: https://github.com/broadinstitute/tensorqtl/blob/master/tensorqtl/cis.py#L59
The many warnings suggest that some phenotypes have very low complexity (most values are the same, usually 0 before normalization). If that's the case, it's best to filter these out.
from tensorqtl.
Related Issues (20)
- Issues with TensorQTL in Trans Mode HOT 1
- AttributeError: 'dict' object has no attribute 'T' tensorqtl HOT 2
- broken link HOT 1
- Tensorqtl installation HOT 3
- No credible sets output from susie.map because pval_nominal=0 HOT 3
- ValueError: array must not contain infs or NaNs HOT 1
- interaction HOT 1
- [Susie] ValueError: prior variance must be non-negative HOT 4
- [trans] trans.map_permutations(): KeyError: 'r2' HOT 2
- [map_trans] R2 not returned in interaction analysis HOT 2
- [pval post-processing] Interaction output for cis/trans HOT 1
- Which beta coresponds to which allele in the outputs? HOT 3
- cis-mapping with interaction HOT 1
- Errors in the BackgroundGenerator cause main thread to get stuck HOT 1
- Questions about the `map_permutations` and `apply_permutations` functions
- cis_independent requires covariate_df HOT 1
- Phenotype file format HOT 3
- Any way i can go around this error? HOT 2
- Categorical covariates HOT 1
- Corections for multiple tests for trans-eQTL HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tensorqtl.