Can't complete test run

[encode-dcc]

Can't complete test run

[leepc12]

Your locally install R conflicts with the pipeline.

Please remove your locally installed R from $PATH in your startup script (~/.bashrc) or do not module load any R.

[leepc12]

Please run the following for debugging.

$ source activate encode-atac-seq-pipeline
$ which R
$ conda list | grep spp

[jwtobias]

That's odd, while I have had R 3.5.0 installed in the past, I thought I hid all traces. Here are the responses to the commands above:

$ source activate encode-atac-seq-pipeline

(encode-atac-seq-pipeline) $ which R
~/software/miniconda3/envs/encode-atac-seq-pipeline/bin/R

(encode-atac-seq-pipeline) $ conda list | grep spp
r-spp 1.13 r3.2.2_0 bioconda

[leepc12]

Did you get any error while installing conda/install_dependencies.sh?

[jwtobias]

Not that I saw.

[jwtobias]

If it's helpful, I uninstalled/reinstalled the environment, activated, and went to the the appropriate directory and reran what I believe is the failing command. Attached are logs of the uninstall/reinstall and the screen output of the redone Rscript command.
conda uninstall reinstall.txt
failing Rscript command output .txt

[leepc12]

Please run these for debugging.

$ source activate encode-atac-seq-pipeline
$ which conda
$ which R
$ echo $R_HOME
$ echo $R_LIBS
$ Rscript <(echo "installed.packages()") | grep spp
$ Rscript <(echo "installed.packages()")

BTW you can disable xcor by activating a flag "atac.disable_xcor" : true, in your input JSON.

[jwtobias]

Results below and attached. It looks like $R_HOME and $R_LIBS are not set.
troubleShootOutput.txt

[jtobias@node062 ~]$ source activate encode-atac-seq-pipeline
(encode-atac-seq-pipeline) [jtobias@node062 ~]$ which conda
~/software/miniconda3/bin/conda
(encode-atac-seq-pipeline) [jtobias@node062 ~]$ which R
~/software/miniconda3/envs/encode-atac-seq-pipeline/bin/R
(encode-atac-seq-pipeline) [jtobias@node062 ~]$ echo $R_HOME

(encode-atac-seq-pipeline) [jtobias@node062 ~]$ echo $R_LIBS

(encode-atac-seq-pipeline) [jtobias@node062 ~]$

[leepc12]

It's weird. Everything looks okay. Let's try to re-install spp.

$ source activate encode-atac-seq-pipeline
$ conda uninstall spp -y
$ conda install phantompeakqualtools -c bioconda -y

[leepc12]

Please run these too.

$ source deactivate
$ which Rscript
$ source activate encode-atac-seq-pipeline
$ Rscript -e ".libPaths()"
$ Rscript -e "Sys.getenv('R_LIBS_USER')"

[leepc12]

Before testing, please add between command { and python $(which encode_xcor.py) \ for further debugging.

	command {

		#PLEASE ADD THESE LINES TO atac.wdl
		which Rscript
		which conda
		echo $R_HOME
		echo $R_LIBS

		python $(which encode_xcor.py) \
			${ta} \
			${if paired_end then "--paired-end" else ""} \
			${"--subsample " + select_first([subsample,25000000])} \
			--speak=0 \
			${"--nth " + select_first([cpu,2])}
	}

[jwtobias]

Ok, in response to your comment about reinstalling spp, before you sent the others, I uninstalled spp using:
$ source activate encode-atac-seq-pipeline
$ conda uninstall spp -y
It didn't seem to find spp to uninstall. Nonetheless I ran:
$ conda install phantompeakqualtools -c bioconda -yconda install phantompeakqualtools -c bioconda -y

Before seeing your two additional comments, I reran the test installation and it ran SUCCESSFULLY.

Here are the results for your Rscript commands:
[jtobias@consign starOut]$ source activate encode-atac-seq-pipeline
(encode-atac-seq-pipeline) [jtobias@consign starOut]$ Rscript -e ".libPaths()"
[1] "/home/jtobias/software/miniconda3/envs/encode-atac-seq-pipeline/lib/R/library"
(encode-atac-seq-pipeline) [jtobias@consign starOut]$ Rscript -e "Sys.getenv('R_LIBS_USER')"
[1] "~/R/x86_64-pc-linux-gnu-library/3.2"

The second one seems to be pointing to a non-existent directory.

Should I still rerun the test with the additional lines in atac.wdl?

Just to complicate things further, I began to install the deprecated bds pipeline as a backup approach, but I've not yet tried to run it. It's possible, though, that this installation somehow changed my environment, allowing the wdl test to run to completion.

[leepc12]

Could you run these too? This should be the final one.

$ source deactivate
$ which Rscript

[jwtobias]

[jtobias@node065 ~]$ which Rscript
/usr/bin/Rscript

[leepc12]

Okay, It looks like Conda's R conflicted with a globally installed R on your system. This will be fixed in the next release.

Until then, for every pipeline run, please define two environment variables after activating the Conda environment and then run pipelines (java -jar ...).

$ source activate encode-atac-seq-pipeline

$ export R_HOME=/home/jtobias/software/miniconda3/envs/encode-atac-seq-pipeline/lib/R
$ export R_LIBS=/home/jtobias/software/miniconda3/envs/encode-atac-seq-pipeline/lib/R/library

$ java -jar -Dconfig.file=backend/backend.conf ............... # your pipeline command line

[leepc12]

Please let me know if this works.

[jwtobias]

Yes, the test run is working. I made the changes you suggested about the R environment variables and had another successful run. It has been running for me, however, since I uninstalled / re-installed spp per your suggestion:
$ source activate encode-atac-seq-pipeline
$ conda uninstall spp -y
$ conda install phantompeakqualtools -c bioconda -y

[leepc12]

@jwtobias: Thanks, closing this.