ValueError: Names should be an ordered collection.

[broadinstitute]

ValueError: Names should be an ordered collection.

[francois-a]

Hi, thanks for reporting this! This error is indeed due to the most recent pandas version, and occurs in pandas_plink. I posted an issue (limix/pandas-plink#18). In the meantime, please use Pandas 1.0.5.

[francois-a]

No, R is only used at the very end, to calculate q-values. The code above crashes on batch 8, which suggests that the error originates with the first batch processed on the GPU (batches are pre-loaded in a separate thread). Without seeing the python stack trace it's impossible to guess what is causing this.

[aarbduarte]

I downgraded pandas to 1.0.5 by editing the requirements.
Furthermore when performing cis nominal mapping I retrieve another error:

`cis = cis.map_nominal(genotype_df, variant_df, phenotype_df, phenotype_pos_df, covariates_df, prefix)
cis-QTL mapping: nominal associations for all variant-phenotype pairs

• 445 samples
• 19836 phenotypes
  Traceback (most recent call last):
  File "", line 1, in
  File "/home/aduarte/tensorqtl/tensorqtl/cis.py", line 157, in map_nominal
  assert np.all(phenotype_df.columns==covariates_df.index)
  AssertionError`

I'm using the provided example dataset as assurance of input quality

[francois-a]

The error in example/tensorqtl_examples.ipynb is fixed in d812a2a. The argument order of map_nominal was changed to make covariates optional.

[aarbduarte]

Thank you for the quick comeback, it fixed that problem.

Now I can go all the way to nominal mapping where it chrashes hard:

 import tensorqtl
 from tensorqtl import genotypeio, cis, trans
 print('PyTorch {}'.format(torch.__version__))
PyTorch 1.6.0
 print('Pandas {}'.format(pd.__version__))
Pandas 1.0.5
 plink_prefix_path = './example/GEUVADIS.445_samples.GRCh38.20170504.maf01.filtered.nodup.bed'
 expression_bed = './example/GEUVADIS.445_samples.expression.bed.gz'
 covariates_file = './example/GEUVADIS.445_samples.covariates.txt'
 prefix = 'Geuvadis_results.'
 phenotype_df, phenotype_pos_df = tensorqtl.read_phenotype_bed(expression_bed)
\t', index_col=0).T
 covariates_df = pd.read_csv(covariates_file, sep='\t', index_col=0).T
 pr = genotypeio.PlinkReader(plink_prefix_path)
Mapping files: 100%|████████████████████████| 3/3 [00:23<00:00,  7.67s/it]
 genotype_df = pr.load_genotypes()
 variant_df = pr.bim.set_index('snp')[['chrom', 'pos']]
 cis.map_nominal(genotype_df, variant_df,
...                 phenotype_df.loc[phenotype_pos_df['chr']=='chr18'],
...                 phenotype_pos_df.loc[phenotype_pos_df['chr']=='chr18'],
...                 prefix, covariates_df=covariates_df)
cis-QTL mapping: nominal associations for all variant-phenotype pairs
  * 445 samples
  * 301 phenotypes
  * 26 covariates
  * 13369268 variants
  * checking phenotypes: 301/301
  * Computing associations
    Mapping chromosome chr18
    processing phenotype 301/301
    time elapsed: 0.83 min
    * writing output
done.
 cis_df = cis.map_cis(genotype_df, variant_df,
...                      phenotype_df.loc[phenotype_pos_df['chr']=='chr18'],
...                      phenotype_pos_df.loc[phenotype_pos_df['chr']=='chr18'],
...                      covariates_df=covariates_df, seed=123456)
cis-QTL mapping: empirical p-values for phenotypes
  * 445 samples
  * 301 phenotypes
  * 26 covariates
  * 13369268 variants
  * using seed 123456
  * checking phenotypes: 301/301
  * computing permutations
    processing phenotype 8/301R[write to console]:
 *** caught illegal operation ***
R[write to console]: address 0x7fe9da6ea409, cause 'illegal operand'
R[write to console]:
Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:```

[francois-a]

The entire notebook runs without problems for me, with the same module versions. What environment are you running this in, and why are you getting R errors at the end?

[aarbduarte]

I'm using RStudio's terminal on a linux machine to run all commands

uname -a
Linux rey 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[francois-a]

Can you try to run this within python/ipython/jupyter instead?

[ramiromagno]

@francois-a: Aren't the R errors resulting from R code being used through rpy2?

[francois-a]

The original error is fixed in limix/pandas-plink#19. @aarbduarte please open a new issue if you still can't get the example notebook to run.