independent eQTL process Missing results

[broadinstitute]

independent eQTL process Missing results

[zpliu1126]

it works. thank you

[francois-a]

Hi, that shouldn't be happening (even if the gene has only one independent signal it should be in the output). Does the number in the log match, i.e., are you expecting 399 or 400 genes in the output? Would it be possible to share the input files just for this gene?

[zpliu1126]

Hi, that shouldn't be happening (even if the gene has only one independent signal it should be in the output). Does the number in the log match, i.e., are you expecting 399 or 400 genes in the output? Would it be possible to share the input files just for this gene?

After I filter the permute results according to FDR<0.05, in order to get the results faster; I run the module for each chromosome separately. for example there have 399 eGene in chr12, but in the result I only get 398 eGene which have multiple independent signals or one signals.

[francois-a]

The 399 genes appear to be correctly parsed. What happens if you subset the input to just the missing gene? Would you be able to share that data so that I can debug this?

[zpliu1126]

This is log of the missing gene

  * WARNING: using CPU!
  * reading phenotypes (/data/cotton/zhenpingliu、eQTLgenes_Allstages_340samples/indepent_eQTL/GeneExpression/0DPA_AllSamples_370_FPKM_AllStages_norm_new.bed.gz)
  * reading covariates (/data/cotton/zhenpingliu/eQTLgenes_Allstages_340samples/peer_factor_num/0DPA/0DPA_covariate_peer_20.txt)
Mapping files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:11<00:00,  3.78s/it]
/public/home/zpliu/.local/lib/python3.6/site-packages/tensorqtl/genotypeio.py:146: PerformanceWarning: Slicing is producing a large chunk. To accept the large
chunk and silence this warning, set the option
    >>> with dask.config.set(**{'array.slicing.split_large_chunks': False}):
    ...     array[indexer]

To avoid creating the large chunks, set the option
    >>> with dask.config.set(**{'array.slicing.split_large_chunks': True}):
    ...     array[indexer]
  self.bed = self.bed[:,ix]
cis-QTL mapping: conditionally independent variants
  * 370 samples
  * 1/1 significant phenotypes
  * 23 covariates
  * 2658921 variants
  * computing independent QTLs
    processing phenotype 1/1
Traceback (most recent call last):
  File "/public/home/software/opt/bio/software/Python/3.6.5/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/public/home/software/opt/bio/software/Python/3.6.5/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/tensorqtl/__main__.py", line 2, in <module>
    tensorqtl.main()
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/tensorqtl/tensorqtl.py", line 146, in main
    logger=logger, seed=args.seed, verbose=True)
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/tensorqtl/cis.py", line 855, in map_independent
    res_df = pd.concat(res_df, axis=0, sort=False)
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 284, in concat
    sort=sort,
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 331, in __init__
    raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate

By the way, do I need to share you the vcf file or just the permute output file?

[francois-a]

I'll need all the input files unless you can generate a different reproducible example, but only need the variants in the cis-window of this gene. And you can scramble the dosages (and the other inputs as well).

[zpliu1126]

the data and commands I used are stored in this repositories.

I also find there is a tss_distance calculation error in eGene results with multiple independent QTLs.
for example:
The eGene Ghir_A12G000020 with a independent eVariant SNP1357144 in the result file independ_out.cis_independent_qtl.txt.gz

tss_distance(367358) + tss(8494)+1 != vcf file (377500)

[francois-a]

Thanks again for reporting this and providing the detailed example! This was a bug due to missingness, with the variants added to the covariates not being properly imputed (fixed in 2ae860f).

The TSS distance should be specified as a single position in the input BED file. I added a check for this in 639515b.

[zpliu1126]

Thank you very much for answering my questions patiently! I will have a try for the version of v1.0.6.

[zpliu1126]

Thanks again for reporting this and providing the detailed example! This was a bug due to missingness, with the variants added to the covariates not being properly imputed (fixed in 2ae860f).

The TSS distance should be specified as a single position in the input BED file. I added a check for this in 639515b.

I followed the prompts to modify the coordinates in the gene expression compressed file. It generated a new error in version v1.0.6, and run scuessfully in version v1.0.5.

[May 06 15:14:43] Running TensorQTL: cis-QTL mapping
  * WARNING: using CPU!
  * reading phenotypes (test_gene_expression.bed.gz)
  * reading covariates (covariates.txt)
Mapping files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 45.97it/s]
Traceback (most recent call last):
  File "/public/home/software/opt/bio/software/Python/3.6.5/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/public/home/software/opt/bio/software/Python/3.6.5/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/tensorqtl/__main__.py", line 2, in <module>
    tensorqtl.main()
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/tensorqtl/tensorqtl.py", line 146, in main
    logger=logger, seed=args.seed, verbose=True)
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/tensorqtl/cis.py", line 667, in map_independent
    signif_df = signif_df[cols]
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/pandas/core/frame.py", line 2908, in __getitem__
    indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/pandas/core/indexing.py", line 1254, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "/public/home/zpliu/.local/lib/python3.6/site-packages/pandas/core/indexing.py", line 1304, in _validate_read_indexer
    raise KeyError(f"{not_found} not in index")
KeyError: "['af'] not in index"

[francois-a]

This error is due to another change in v1.0.6, returning allele fraction ('af') instead of minor allele fraction ('maf') in the output files. This was done for consistency, since the minor allele is not necessary the ALT/effect allele. As a workaround, you can rename the column from your input instead of re-running it (that column isn't used in this step, just checked for consistency).