Collapse and combine GTEx TPM and expected_count to KFDRC collapsed RSEM files

[kgaonkar6]

Collapse and combine GTEx TPM and expected_count to KFDRC collapsed RSEM files

[runjin326]

Yes. Now it's good! Thanks!

[jharenza]

@zhangb1 do you have an eta for this ticket?

[zhangb1]

@jharenza can someone from your team take care of the ticket, maybe run?

I really need to focus on the data assembly cwl develop this week.

[jharenza]

I don't think she has ec2 access yet and we have a lot of open issues for OT that need to be done as well - @yuankunzhu can someone else help with these upstream processes?

[zhangb1]

Trying to run locally but have the memory issue. can't process

I need to wrap the cwl and try to run on cavatica project.

the error

➜  GTEx Rscript 00-collapse_matrices.R -i gene-expression-rsem-tpm.rds -g gencode.v27.primary_assembly.annotation.gtf.gz -m gene-expression-rsem-tpm-collapsed.rds -t gene-expression-rsem-tpm-collapsed_table.rds -n GTEx_Analysis_2017-06-05_v8_RNASeQCv1.1.9_gene_tpm.gct.gz
[1] "Generating input matrix and drops table...!"
[1] "Read merged GTEx data"

── Column specification ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
cols(
  .default = col_double(),
  Name = col_character(),
  Description = col_character()
)
ℹ Use `spec()` for the full column specifications.



[1]    95949 killed     Rscript 00-collapse_matrices.R -i gene-expression-rsem-tpm.rds -g  -m  -t  -n

[kgaonkar6]

I had to run it on a large EC2, GTEx files are huge

[zhangb1]

I ran the gtex collapsed data on cavatica project
here: https://cavatica.sbgenomics.com/u/zhangb1/test-download/tasks/2d842fee-65cf-4c5b-b1aa-0e458b2dfa29/

and I ran the pbta-gmkf-gene-expression-rsem-tpm-collapsed.rds locally.

when I tried to merge the files using the notebook.

> common_genes <- intersect(rownames(gtex),rownames(pbta_gmkf))
> length(common_genes)
[1] 0

the length of the common_genes is 0.

@kgaonkar6 can you take a look?

[zhangb1]

wait. I think I found the issue. l will check

[zhangb1]

@jharenza @kgaonkar6

the merged files has been updated in the bucket:

2021-07-21 14:54:03 1073668357 gene-counts-rsem-expected_count-collapsed.rds
2021-07-21 14:28:30 1611312452 gene-expression-rsem-tpm-collapsed.rds

md5 also updated

[kgaonkar6]

Thanks a lot @zhangb1 !!

@runjin326 can you try to download these files with the updated download-data.sh pointing to v7 s3 bucket? Since you need for fusion_filteering rerun?

[runjin326]

@kgaonkar6, @zhangb1, thanks so much! Yes I have downloaded the data and should be able to re-run fusion_filtering now.

[runjin326]

Oh sorry forget to mention, @kgaonkar6 - the md5sum did not match. I am guessing that is because they are updated?

[kgaonkar6]

Could you removegene-counts-rsem-expected_count-collapsed.rds and gene-expression-rsem-tpm-collapsed.rds in your local data folder and rerun download-data.sh

[komalsrathi]

Closing this as we have this data available in OT.