blackrim/treepl

Phylogenetic penalized likelihood

treepl's Issues

calculating without gradient (might want to try a different optad=VALUE)

I have run TreePl by first running Prime and copying and pasting the generated values at the bottom of the configuration file:

opt = 4
moredetail
optad = 1
moredetailad
optcvad = 1

I then run the cv part of the program and change the smoothing parameter. Once I run the tree my results seem generally fine but "calculating without gradient (might want to try a different optad=VALUE)" is output a few times. Same with "calculating without gradient (might want to try a different opt=VALUE). These values (opt and optad) were generated from the Prime output, so how do I decide what other numbers to try?

Thanks

announcement: docker treePL

This is probably too late in the game because I'm guessing Stephen simply has no time for this anymore - but it would be wonderful if there was a dockerized treePL. A conspicuously large number of issues deals with installation problems, usually actually dealing with ADOL-C. I suspect it would be possible for somebody who knows what they're doing to get this right in a Dockerfile and then have the rest of the world enjoy that, both on Mac and Linux.

how to choose the best smoothing from treePL

Hi,
I also attached my questions here, hope the answers would be helpful for other people, or some other people can help as well.

I'm using treePL to do the dating. and I acknowledged that the smoothing parameter is very important, which affects the penalty for rate variation over the tree.

So before I do the dating, I have conducted cross-validation procedure (thorough, prime, randomcv) to determine the best smoothing parameter with different "smooth =" settings (e.g.,smooth = 0.1,1,10,100,1000,10000) in the cross-validation
configure file.

now I'm confused facing these “priming” results, which should I choose? or none of them are best?

When "smooth = 0.1", the result is "best: 1(1) bestad: 1 (1) bestcv: 3 (0)", is this best? what does "0" in "bestcv: 3 (0)" mean?

Thank you very much!

Miao

Parts of results are attached below:

==> treePL2769972_0.1.out <==
you may want to set the plsimaniter > 100000
result: 4
final: 3128607.6 diff: 355884.83
best: 1(1) bestad: 1 (1) bestcv: 3 (0)
PLACE THE LINES BELOW IN THE CONFIGURATION FILE
opt = 1
moredetail
optad = 1
moredetailad
optcvad = 3

==> treePL2769972_1.out <==
final: 3207121.5 diff: 110930.04
setting NLOPT parallel : LD_MMA
you may want to set the plsimaniter > 100000
result: 4
final: 3000710.2 diff: 317341.38
best: 1(0) bestad: 1 (0) bestcv: 3 (0)
PLACE THE LINES BELOW IN THE CONFIGURATION FILE
opt = 1
optad = 1
optcvad = 3

==> treePL2769972_10.out <==
you may want to set the plsimaniter > 100000
result: 4
final: 3050738.9 diff: 2588895
best: 1(1) bestad: 1 (1) bestcv: 0 (0)
PLACE THE LINES BELOW IN THE CONFIGURATION FILE
opt = 1
moredetail
optad = 1
moredetailad
optcvad = 0

==> treePL2769972_100.out <==
setting NLOPT parallel : LD_MMA
you may want to set the plsimaniter > 100000
result: 4
final: 3046957.5 diff: 280371.75
best: 1(0) bestad: 1 (1) bestcv: 0 (0)
PLACE THE LINES BELOW IN THE CONFIGURATION FILE
opt = 1
optad = 1
moredetailad
optcvad = 0

==> treePL2769972_1000.out <==
setting NLOPT parallel : LD_MMA
you may want to set the plsimaniter > 100000
result: 4
final: 3149824.2 diff: 254039.98
best: 1(0) bestad: 1 (1) bestcv: 3 (0)
PLACE THE LINES BELOW IN THE CONFIGURATION FILE
opt = 1
optad = 1
moredetailad
optcvad = 3

==> treePL2769972_10000.out <==
final: 3333963.1 diff: 95658.855
setting NLOPT parallel : LD_MMA
you may want to set the plsimaniter > 100000
result: 4
final: 3164714.8 diff: 264907.15
best: 1(0) bestad: 1 (0) bestcv: 3 (0)
PLACE THE LINES BELOW IN THE CONFIGURATION FILE
opt = 1
optad = 1
optcvad = 3

checks for constraints

Seems like a bunch of relatively easy checks could be implemented:

1. Check if one constraint is in conflict with another e.g. a min is larger than an ancestor's max
   • currently the program keeps trying (in vain) for viable start dates before giving up.
2. Check if different mrcas are mapped to the same node
   • currently the constraint which gets implemented appears to have do with a tree traversal (rather than the order which constraints are declared or implemented in the config file).

These checks would be useful for analyzing a distribution of trees (i.e. where fossil placement differs depending on topology).

Here is a config file (for use with test.tre in the examples) which demonstrates number 2 above:

treefile = test.tre
smooth = 100
numsites = 1000

mrca = nrt a b
mrca = rt a f

mrca = ae a e
mrca = be b e
mrca = ce c e
mrca = de d e

min = nrt 50
max = rt 100

## max test ##
# be wins
max = ae 80
max = be 75

## min test ##
# de wins
min = de 63
min = ce 60

[prime]
opt = 5
moredetail
optad = 2
optcvad = 2
moredetailcvad

[cv]
[cvstart = .0001]
[cvstop = 10000]
[randomcv]
[cvmultstep = 0.1]
[verbose]
thorough
lfsimaniter = 0
plsimaniter = 0

Problem with mrca and tiny branch length at internal node.

Dear Experts,

This is my first run with treePL program for priming analysis and got message like

set thorough: true (MAY TAKE A WHILE)
setting log penalty
PRIMING the optimization parameters and then exiting
outfile: prime_outTreeRax
finished reading config file
using system clock for random number seed = 1611032383
tiny branch length at internal node. setting to 5.2042675e-06
problem with mrca DIP (probably no mrca named this)

But no output file has been generated even I have multiplied branch length with 10^6 following Kévin J. L. Maurin tutorial, but the this tiny branch length message is coming. Moreover I could not understand why the mrca could not recognizes DIP that I have already mentioned in the config file.
My configuration file is like that

[1 Input tree]

treefile = 335TemTRax10e6.tre


[2 General command]

numsites = 192150
thorough
log_pen


[3 Calibration command]

mrca DIP taxon23 taxon42
min = DIP 230
max = DIP 290

mrca TRE taxon15 taxon43
max = TRE 325


[4 Priming command]

prime


[5 Best optimisation param]


[6 Cross-validation analysis]


[7 Best smooting value]


[8 Output]

outfile = prime_outTreeRax

And
Tree File is

((((((taxon49:10000000,((((taxon42:2000000,(taxon45:1000000,taxon46:1000000):1000000):1000000,taxon52:3000000):2000000,(taxon37:4000000,((taxon38:1000000,taxon41:1000000):2000000,((taxon40:1000000,taxon39:1000000):1000000,taxon36:2000000):1000000):1000000):1000000):4000000,((taxon20:4000000,(taxon54:3000000,(taxon17:2000000,(taxon18:1000000,taxon16:1000000):1000000):1000000):1000000):4000000,(taxon35:7000000,(taxon50:6000000,((taxon29:4000000,((taxon51:2000000,(taxon32:1000000,taxon26:1000000):1000000):1000000,(taxon53:1000000,taxon33:1000000):2000000):1000000):1000000,((taxon34:1000000,taxon27:1000000):2000000,((taxon31:1000000,taxon28:1000000):1000000,taxon30:2000000):1000000):2000000):1000000):1000000):1000000):1000000):1000000):1000000,taxon25:11000000):1000000,(((taxon24:1000000,taxon19:1000000):2000000,((taxon21:1000000,taxon22:1000000):1000000,(taxon48:1000000,taxon47:1000000):1000000):1000000):8000000,(taxon44:10000000,(taxon1:9000000,(((taxon14:1000000,taxon4:1000000):6000000,((taxon7:1000000,taxon9:1000000):5000000,((taxon8:4000000,(((taxon11:1000000,taxon3:1000000):1000000,taxon13:2000000):1000000,taxon5:3000000):1000000):1000000,((taxon6:1000000,taxon12:1000000):1000000,taxon10:2000000):3000000):1000000):1000000):1000000,taxon2:8000000):1000000):1000000):1000000):1000000):1000000,taxon23:13000000):1000000,taxon43:14000000):7500000,taxon15:7500000);

It will be an immense help if someone kindly correct me.

parallel on mac

openmp things don't seem to work right on the mac. not sure why. happens with 10.6.8. As long as nthreads isn't used it is ok for now.

Quick question : is there a linux version of TreePL ?

Thanks a lot !

Lua
phD student

Segmentation fault with mac line-endings

Pretty self explanatory: if the config file has mac line-ending, treePL dies. Not sure this is worth fixing; maybe just warn users that unix line-endings are expected.

Any documentation at all would be nice :-)

Meaning of numsites

Hello,

I had an interesting discussion today regarding the meaning of 'numsites' in the configuration file.

1. Does it mean the total length of the alignment used to generate the phylogeny?

OR

2. Does it mean the number of distinct alignment patterns used to build the tree (i.e. from RAxML)?

Thanks,
Cody Coyotee

Node ages

Hi, I'm confused in node ages, what parameters we need to optimize in order to get correct node ages. Every time I got tree with greater node ages when I used different values for smooth (0.001-10000). How could I possibly get lower node ages?
Thank you,
Best,
Uzma

What exactly does "linear search failed" mean in terms of tree quality?

I'm receiving a lot of "linear search failed" lines in my run of a tree with 1457 tips. What does this mean?

Writing a for loop for treepl?

Hello,

I am trying to write a simple for loop in bash for treepl. I am novice at programming and have been stuck on this for awhile. I was wondering if yall had any suggestions? This is what I have so far:

for filename in ./TESTLOOP/*.tre;
do treePl cd.. config8 ;
done;

for the input tree = "$filename" # basically a directory with all the tree files
output tree = "$filename".dated.tre # I am confused about how to set up the output directory

When I run this i get this error multiple times:

treePL version 1.0
you need more arguments.

Any suggestions are greatly appreciated, thanks!

really large tree non gradient crash

For extremely large trees (> 30,000 tips), there is a crash for most of the non gradient optimization in nlopt.

Tree is invalid: missing concluding semicolon

Hello,

I am trying to run a treePL analysis with a .tre file but I get this error message:
Tree is invalid: missing concluding semicolon. Exiting

I have checked my .tre file, and all the semicolons are here at the end of each line. Here is the structure of the file (I can email it for debugging if needed):
#NEXUS
begin taxa;
dimensions ntax=19;
taxlabels
{names of my taxa}
;
end;
begin trees;
tree TREE1 = [&R] {tree};
end;
begin figtree;
{list of "set" commands, each with their semicolon}
end;

I have tried removing or adding bits of code like round brackets, square brackets, the #NEXUS tag, figtree block, etc... nothing solves the problem. Also, no online help found for this case. I am running treePL on Ubuntu.

Many thanks in advance for your help.
Kind regards,
Kévin

adol-c issue

Hi Stephen,

I had some trouble compiling treePL because of an issue in ADOL-C that might be noted in your src/README. Alternatively, someone could sign up for gitorious, clone ADOL-C, make the fix and send them a merge request. At least, I didn't see an issue tracker there so I will vent here instead.

As follows: I think they have recently moved a bunch of headers in a separate folder in their source tree. The idea is then that they would use these headers as <adolc/foo.h>, which they've done pretty consistently except for the case where they include "adolc_openmp.h", which should be included as <adolc/adolc_openmp.h>, notably in tape_handling.cpp, line 1004.

Hope this is useful.

Best wishes,

Rutger

CV hanging

We have been attempting to do cross-validation with values determined from the 'prime' analysis; however, the analysis gets stuck (crashing my computer), appearing to be going in circles, printing this sort of thing to the terminal:

after opt calc2: 42151.165
exit siman: 42151.165
setting NLOPT: LD_LBFGS
result: 4
after opt calc1: 42151.164
setting NLOPT: LD_LBFGS
result: -1
after opt calc2: 42151.165
exit siman: 42151.165
setting NLOPT: LD_LBFGS
result: 3
after opt calc1: 42151.164
setting NLOPT: LD_LBFGS
result: 3
after opt calc2: 42151.165
exit siman: 42151.165
setting NLOPT: LD_LBFGS
result: 3
after opt calc1: 42151.164
setting NLOPT: LD_LBFGS
result: -1
after opt calc2: 42151.165
exit siman: 42151.165
setting NLOPT: LD_LBFGS
result: 4
after opt calc1: 42151.164
setting NLOPT: LD_LBFGS
result: 3
after opt calc2: 42151.165
exit siman: 42151.165
setting NLOPT: LD_LBFGS
result: -1
after opt calc1: 42151.164
setting NLOPT: LD_LBFGS
result: -1
after opt calc2: 42151.165
exit siman: 42151.165
setting NLOPT: LD_LBFGS

Note how the decimals are only ever .164 or .165.

Additionally, the output files stop writing.

The optimization parameters are as follows:
nthreads = 10 opt = 1 moredetail optad = 1 moredetailad optcvad = 1 moredetailcvad

I had previously did the same analysis using a preliminary phylogeny, but with slightly different optimization parameters:
nthreads = 10 opt = 2 moredetail optad = 2 moredetailad optcvad = 1 moredetailcvad
This preliminary analysis runs fine, the only difference being a couple of parameters and a slightly different starting tree.

Thanks

Confidence Intervals: Bayesian vs ML

Hello:

I know this is not an issue with treePL itself, but I figured there might not be anywhere better to post this question then a group devoted to the program.

Has anyone used Bayesian methods with treePL to generate confidence intervals? All I see is with ML based tree methods. Please share citation or title if you have seen it.

Second question- has anyone been able to produce "large" confidence intervals (somewhat overlapping between adjacent nodes) with ML methods? The ones I usually see appear to be very tight and narrow in comparison to trees generated with Bays.

Thanks!

badly formed tree input causes out of memory crash

If the last tree in the input treefile is not followed by a semicolon, the program will apparently keep reading until it completely fills the memory of the computer. I have crashed my 48GB RAM desktop several times doing this.

treepl: command not found

Dear Experts,

I have installed treepl using brew command brew install brewsci/bio/treepl in UBUNTU 18.04. But after that when I type treepl it displays treepl: command not found. The testing command brew test -v treepl shows long list run.

git config --replace-all homebrew.devcmdrun true ==> Testing brewsci/bio/treepl ==> /home/linuxbrew/.linuxbrew/Cellar/treepl/2018.05.22/bin/treePL clock.cppr8s outfile: 1000.tre cv number of full iterations: 3 cv number of simulated annealing iterations: 1000 cvstart: 1000 cvstop: 0.0001 cvmultstep: 0.1 ftol: 1e-05 setting opt: 0 setting optad: 2 setting optcvad: 4 setting log penalty finished reading config file using system clock for random number seed = 1610905047 tiny branch length at internal node. setting to 1e-05 setting ingroup min: 100 setting ingroup max: 100 preorder prep calculating character durations setting min and max setting up all constraints getting feasible start dates start rate 0.0026265772 numparams:199 initial calc: 12532054 LF SIM exit siman: 2633695 nfeval: 60 rc: Linear search failed after opt calc1: 776163.03 setting NLOPT: LD_TNEWTON_PRECOND_RESTART result: 3 after opt calc2: 1643.5564 LF SIM exit siman: 1643.5564 nfeval: 20 rc: Linear search failed after opt calc1: 1643.5564 setting NLOPT: LD_TNEWTON_PRECOND_RESTART result: 3

It will be an immense help if someone kindly guide me.
Thanks.

Strange exit(0) on "valid" input

Hi,

on one of our data sets we're getting an NaN somewhere that is causing treePL to exit. Looking at the code a bit we're a bit puzzled by what's happening around here:

https://github.com/blackrim/treePL/blob/master/src/pl_calc_parallel.cpp#L684

We can't figure out why that padding (or indeed, anything) is still happening after the exit, and what the intent is here. If I comment the exit out and re-run we somehow do end up with an ultrametric tree with the calibration point at the right age, but we're not particularly confident about our results. What do you think could be happening? Can we send you a simple example so you can replicate what's going on?

Thanks,

Rutger

How to run the "wiggle" option mentioned in the paper?

While reading the paper I noticed the option for creating confidence intervals, "We have also provided a ‘wiggle’ option that will report divergence times that are within two log likelihood units of the best likelihood, suggested by Edwards (1992) as a confidence window." However, when looking at the options for parameters, I cannot find this wiggle feature. My question is was this parameter removed?

Calibration points missing node bars

Hello,

Followed the advice of several people here (#30) to calculate the error in node ages for a phylogeny. Briefly, I

1. made 100 bootstrap trees from an alignment using RAxML
2. ran RAxML on those bootstrap trees using a constraint to get branch lengths
3. rerooted the resulting trees using the same outgroup (phyx)
4. ran treePL prime, CV and the final smoothing run on all the trees found in a file
5. Converted the resulting trees to NEXUS after inputting them into FigTree
6. Summarized using TreeAnnotator

Now I have error bars around all nodes except where I placed calibration points (see image; nodes missing any error bar is where a calibration point was placed). The points themselves had minimum and maximum ages so one would expect there to be error bars around those nodes as well. I cannot figure out why this is happening.

Any advice appreciated.

Thanks,
Cody Coyotee

segmentation fault

Hello,
I am trying to date a RAxML tree created on geneious prime and keep failing.
I have the input tree file named as "intree.tre" in a newick format and the config file as "treePL_conf.txt":

treefile = intree.tre
smooth = 100
numsites = 1023
mrca = GLOMEROMYCOTINA ETAR2012_STI1_14_HE615033 JDMZ2015_CM2_VTX00057_LN618084
min = GLOMEROMYCOTINA 505
max = GLOMEROMYCOTINA 505
outfile = intree.dated.tre
thorough

When running the code , I get:
~$ ../linuxbrew/.linuxbrew/bin/treePL treePL_conf.txt
outfile: intree.dated.tre
set thorough: true (MAY TAKE A WHILE)
finished reading config file
using system clock for random number seed = 1590420698
Segmentation fault

Any idea how I could solve this issue?

Thanks!

MRCA

Hello, I'm new to treePL, is there any detailed description or manual or google group for this software. I wouldn't understand MRCA (most recent common ancestor) in run option because there are two names of species. is this the root of tree? secondly in which format I need to save this configure file?
Thank you

configure does not honour options specified

I tried passing the following to configure:

./configure CXX=/usr/local/bin/g++-9 CC=/usr/local/bin/gcc-9

but the Makefile looks like this:

CC := $(CC)
CP := $(CXX) -std=c++11

Unfortunately on mac, this points to clang, so no openmp.

We have this option in phyx, so probably just a matter of copying over the relevant bits.

treePL with Mac OS 11.1

Hi,

I have a quick question: it seems I can't get treePL to run since I've updated my Mac Os to "El Capitan" (11.1). Is that a known problem, or is it just me having made a mistake or confusing something?

Thank you,
Johannes

error in installation

When I use the command line "brew install brewsci/bio/treepl" to install treePL in ubantu 16.04.1, it showed Error: undefined method `rebuild' for #BottleSpecification:0x0000000213bdc8. When I download the source code to install and finished, I could not run with error " while loading shared libraries: libadolc.so.1: cannot open shared object file: No such file or directory ".

link error on ubuntu 13.10 with apt-get installed libadolc2/libadolc-dev

g++ -O3 -g3 -o treePL ./main.o ./node.o ./pl_calc_parallel.o ./tnc.o ./tree.o ./tree_reader.o ./tree_utils.o ./optimize_tnc.o ./optimize_nlopt.o ./myradops.o ./siman_calc_par.o ./optim_options.o ./utils.o -lm -fopenmp -ladolc -L/usr/lib64 -lnlopt
./main.o: In function `ADOLC_OpenMP_NC':
/usr/include/adolc/adolc_openmp.h:46: undefined reference to `ADOLC_parallel_doCopy'
/usr/include/adolc/adolc_openmp.h:47: undefined reference to `beginParallel()'
./main.o: In function `~ADOLC_OpenMP_NC':
/usr/include/adolc/adolc_openmp.h:50: undefined reference to `endParallel()'
collect2: error: ld returned 1 exit status
make: *** [treePL] Error 1

Prime: "Linear search failed" concerns

Hello everyone,

I have been receiving a message while running the 'prime' step of setting up a configuration file. I am curious if this is causing issues with my analysis. I have been attempting to get treePL to run for awhile now but the CV step takes a long time (over 25 days on a computer cluster) and the priming step always worries me (See below). The tree contains 12,780 tips and was built from a supermatrix. I used ExaML to build the tree. Happy to supply input tree if needed. Thanks for any suggestions on how to proceed.

-----Config file-----
treefile = monocots_rerooted.tre
smooth = .000001
numsites = 7489
mrca = ALISMA Ranalisma_rostratum Echinodorus_bracteatus
min = ALISMA 20
max = ALISMA 20
mrca = APONO Aponogeton_fenestralis Aponogeton_lakhonensis
min = APONO 81.13
max = APONO 81.13
mrca = LEMNOI Lemna_gibba Spirodela_intermedia
min = LEMNOI 66
max = LEMNOI 66
mrca = LOMAN Cordyline_australis Cordyline_pumilio
min = LOMAN 22
max = LOMAN 22
mrca = AGAVO Anemarrhena_asphodeloides Chlorophytum_subpetiolatum
min = AGAVO 14.5
max = AGAVO 16.2
mrca = ASTEL Astelia_alpina Astelia_grandis
min = ASTEL 23.2
max = ASTEL 23.2
mrca = GOODY Pachyplectron_arifolium Gonatostylis_vieillardii
min = GOODY 15
max = GOODY 20
mrca = XANTH Eremurus_stenophyllus Caesia_contorta
min = XANTH 38.0
max = XANTH 47.8
mrca = CYCLAN Carludovica_rotundifolia Cyclanthus_bipartitus
min = CYCLAN 47
max = CYCLAN 47
mrca = RIPO Ripogonum_scandens Ripogonum_elseyanum
min = RIPO 51
max = RIPO 52
mrca = ZINGI Siphonochilus_aethiopicus Rhynchanthus_beesianus
min = ZINGI 72.1
max = ZINGI 83.6
mrca = TYPHA Typha_angustifolia Sparganium_eurycarpum
min = TYPHA 51.66
max = TYPHA 51.66
mrca = CYPER Cladium_mariscus Chorizandra_cymbaria
min = CYPER 47
max = CYPER 47
mrca = DIOSCO Dioscorea_antaly Dioscorea_arachidna
min = DIOSCO 27.23
max = DIOSCO 27.23
mrca = ARECA Phoenix_canariensis Salacca_ramosiana
min = ARECA 83.6
max = ARECA 86.3
outfile = monocot_fixed_dated_sm1e6.tre
prime
#randomcv
#cv
plsimaniter = 500000
cvsimaniter = 200000
logpen

iterate until convergence, takes longer but a good idea

thorough

number of threads

nthreads = 8
#opt =
#optad =
#optcvad =

--------Example PRIME OUTPUT------
a bunch of 'tiny branch length at internal node. setting to 0.00013352918" warnings
......
preorder prep
calculating character durations
setting min and max
setting up all constraints
getting feasible start dates
start rate 0.00097994697
numparams:12771
initial calc: 5293281.4
LF SIM
exit siman: 764076.6
now priming regular
nfeval: 29 rc: Linear search failed
final: 764076.6 diff: 1.1641532e-10
setting NLOPT: LD_LBFGS
result: -1
final: 763870.27 diff: 206.32594
setting NLOPT: LD_TNEWTON_PRECOND_RESTART
result: 5
final: 729583.62 diff: 34492.973
setting NLOPT: LD_MMA
result: 3
final: 764071.95 diff: 4.6418695
setting NLOPT: LD_VAR2
result: 5
final: 615831.62 diff: 148244.98
setting NLOPT : NLOPT_LD_MMA
you may want to set the plsimaniter > 100000
result: 3
final: 764071.95 diff: 4.6418695

now priming AD
nfeval: 149 rc: Linear search failed
final: 762341.01 diff: 1735.5861
setting NLOPT: LD_LBFGS
result: -1
final: 763870.27 diff: 206.326
setting NLOPT: LD_TNEWTON_PRECOND_RESTART
result: 5
final: 725095.58 diff: 38981.019
setting NLOPT: LD_MMA
result: 3
final: 764071.95 diff: 4.6416944
setting NLOPT: LD_VAR2
result: 5
final: 614199.28 diff: 149877.32
setting NLOPT : NLOPT_LD_MMA
you may want to set the plsimaniter > 100000
result: 3
final: 764071.95 diff: 4.6416944

now priming CV (AD)
nfeval: 330 rc: Linear search failed
final: 640800.97 diff: 123275.63
setting NLOPT parallel : LD_LBFGS
result: -1
final: 763629.47 diff: 447.12907
setting NLOPT parallel : LD_TNEWTON_PRECOND_RESTART
result: 5
final: 756486.44 diff: 7590.1535
setting NLOPT parallel : LD_MMA
result: 4
final: 648082.5 diff: 115994.1
setting NLOPT parallel : LD_VAR2
result: 5
final: 625347.82 diff: 138728.78
setting NLOPT parallel : LD_MMA
you may want to set the plsimaniter > 100000
result: 4
final: 648082.5 diff: 115994.1
best: 4(0) bestad: 4 (0) bestcv: 4 (0)
PLACE THE LINES BELOW IN THE CONFIGURATION FILE
opt = 4
optad = 4
optcvad = 4
------END-----

Estimating substitution rates

Hey Guys,

Does treePL generate substitution rates that can be viewed somewhere? I see what looks like these rates as edge lengths in .r8s files, but I am just not sure.
If this is possible, could you explain where to get them and how to properly interpret the rates (are these per calibration unit [?my]).

Thanks everyone!

Frank

How to run treePL?

Hello,

Here is a very basic question, but surprisingly I cannot find any answer to it online: how do I run treePL once installed?

The test run works, but when I am trying to replace "test" in the command line brew test -v treepl by my script (a .txt file), with the path to it, I get a Unknown command: [my file] error message.
I have tried many combinations of this command line, like swapping treepl and my file, not putting brew, adding ./ before treepl, adding the path to the treePL script, etc. but nothing works.

Can someone please tell me how they run it?
Many thanks

Unstable smooth values

Hi All,

First, a bit of context:

• I am trying to produce dates and 95% intervals for the nodes of a 19-sequence tree from species that are of the same plant order, with a few outgroups.
• Here is the outline of my configuration file for cross-validation:

treefile = /home/kjlm1/PENtrials/PENtreePLtrials_besttree_noBS.tre
numsites = 46051
mrca = FOSSIL1 taxon1 taxon2
min = FOSSIL1 38.3
max = FOSSIL1 112
(I have a couple more fossils like this but I do not think it is necessary to detail this part)

outfile = PEN_treePL_trials_dated.tre

nthreads = 2
thorough
[prime]

opt = 5
optad = 5
optcvad = 2
moredetailcvad

cv
randomcv
cviter = 10
cvstart = 100000
cvstop = 0.000000001
cvmultstep = 0.1
cvoutfile = randomcvnew2_outfile.txt

I ran the cross-validation analysis several times, and each time I would get a different "best" value for the smoothing parameter, which are very distant from each other. And this happened whether I was using randomcv or not: under a LOOCV I obtained values like 1e-5 to 1e8, and under RSRCV I obtained 1e-9 to 1e-5.

The weird thing is that when I tried dating my tree with very different values, 10 and 1e-6, I pretty much got the same dates (the differences were not significant), and these dates looked the same as the same tree I built in BEAST2, for comparison.

So, my questions are:

• Why do trees built from smoothing values that are so different look so similar?
• Why is the best smoothing value so unstable?
• Is choosing the best smoothing value that crucial then? I am working on a small tree of reasonably closely related species so maybe not, but what if I was working on a 100s-sequence tree of species distributed across the whole tree of Angiosperms?

I am wondering if this floating smoothing value might come from the same issue as the one I saw during the priming step: I got a lot of fail messages like "Linear search failed", sometimes "Unable to progress"; I rarely got positive message like "Converged (|f_n-f_(n-1)| ~= 0)" or "Maximum number of function evaluations reached". These messages changed each time I restarted the prime analysis, and I got different sets of values for opt. I have tried tinkering a lot by changing the values of the cv parameters for example, but I still get the fail messages (I posted in another thread about these). I would be glad if these fail messages could be explained as well, and what impact they have on the analysis.

crashes more often than not

In seemingly simple cases, with few calibration points such as when fixing the root to some arbitrary number, treePL crashes:

initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate -0.25374025
initial calc: 1e+15
Failed setting feasible start rates/dates after 10 attempts. Aborting.

Treepl+RaxML replicate trees

Hello!

I have been using treePL to date a phylogenetic tree but I generated replicate trees with RaxML (same topology but branch lengths varied) and ran treePL on each of these trees in order to get confidence intervals for each node. I already calculate CI values but I am still struggling to find a program in which I can add those CI values separately (such that each node has a bar).

Any suggestions for a good tree viewer?

Thank you!

Carolina.

Is this project still being maintained?

It has been a few years and no commits since 1.0 release.

failures to converge, what to do in general?

Hi Stephen,

we are intermittently running into the issue that treePL fails to arrive at feasible starting rates/dates after 10 attempts. This appears to happen with some combinations of calibration points, but it is not obvious to either @hettling or me what these combinations are and what we might be able to do about it in the general case. Here is a typical example. This is our command file:

treefile = /tmp/XULshU8_lq
smooth = 100
numsites = 26164
outfile = /tmp/VU_SAflRaW
nthreads = 8
seed = 1234
mrca = NFos3 9593 9597 9598 9606
max = NFos3 6.3
min = NFos3 6.3
mrca = NFos4 9593 9597 9598 9600 9606
max = NFos4 15.2
min = NFos4 15.2
mrca = NFos9 100224 164648 170207 222416 33548 36226 36231 373033 43780 54131 54133 54135 54180 61621 61622 75567 78451 78452 88029 9530 9532 9534 9538 9544 9546 9557 9561 9565 9568
max = NFos9 16.8
min = NFos9 16.8
mrca = NFos5 29089 61851 61852 9580 9588 9590 9593 9597 9598 9600 9606
max = NFos5 18.8
min = NFos5 18.8
mrca = NFos1 100224 164648 170207 222416 29089 33548 36226 36231 373033 43780 54131 54133 54135 54180 61621 61622 61851 61852 75567 78451 78452 88029 9530 9532 9534 9538 9544 9546 9557 9561 9565 9568 9580 9588 9590 9593 9597 9598 9600 9606
max = NFos1 32.8
min = NFos1 32.8
mrca = NFos6 100224 118643 164648 170207 198627 222416 29089 30588 30589 30594 30596 30598 33548 36226 36231 373033 43777 43780 52231 54131 54133 54135 54180 57375 58710 61621 61622 61851 61852 66265 70814 70825 70928 75567 78451 78452 867331 88029 9487 9491 9495 9502 9505 9508 9514 9515 9519 9523 9525 9530 9532 9534 9538 9544 9546 9557 9561 9565 9568 9580 9588 9590 9593 9597 9598 9600 9606
max = NFos6 49.8
min = NFos6 49.8
mrca = NFos8 101278 111174 122222 122232 122246 122248 13514 13557 261734 261736 261739 30601 30608 30609 30611 31869 34829 34830 47177 47180 78583 78866 83281 89672 9447 9455 9460 9463 9468 9470 9472
max = NFos8 51.6
min = NFos8 51.6
mrca = NFos7 100224 101278 111174 118643 122222 122232 122246 122248 13514 13557 164648 170207 198627 222416 261734 261736 261739 29089 30588 30589 30594 30596 30598 30601 30608 30609 30611 31869 33548 34829 34830 36226 36231 373033 43777 43780 47177 47180 52231 54131 54133 54135 54180 57375 58710 61621 61622 61851 61852 662464 66265 70814 70825 70928 75567 78451 78452 78583 78866 83281 867331 88029 89672 9447 9455 9460 9463 9468 9470 9472 9478 9487 9491 9495 9502 9505 9508 9514 9515 9519 9523 9525 9530 9532 9534 9538 9544 9546 9557 9561 9565 9568 9580 9588 9590 9593 9597 9598 9600 9606
max = NFos7 77.5
min = NFos7 77.5

This is our tree:

(((((9472:0.011024,261739:0.013202):0.013202,((9463:0.013202,30611:0.021228):0.007521,((89672:0.013202,111174:0.013202):0.013202,(261736:0.007074,30609:0.013202):0.013202):0.013202):0.013202):0.013202,((9470:0.013202,101278:0.013202):0.013202,9468:0.028992):0.010664):0.013202,(((((9460:0.009639,47177:0.013202):0.001311,(((30608:0.013202,122232:0.013202):0.013202,47180:0.013202):0.012450,122248:0.013202):0.013202):0.024734,(((13514:0.008638,34829:0.013202):0.000200,9455:0.013202):0.014976,(9447:0.002842,(13557:0.013202,122222:0.018402):0.013202):0.012615):0.007645):0.004486,(((78866:0.046375,78583:0.013202):0.013202,261734:0.013202):0.007734,(122246:0.047206,(30601:0.010266,83281:0.006046):0.013202):0.013202):0.013202):0.001244,(34830:0.029978,31869:0.013202):0.013202):0.007373):0.013202,((662464:0.013202,9478:0.013202):0.013202,(((((9523:0.013202,9525:0.013202):0.001399,70814:0.021428):0.004367,((30598:0.005837,43777:0.008787):0.013202,(30596:0.013202,70825:0.000622):0.011371):0.013202):0.013202,(((((30588:0.000090,58710:0.013202):0.013202,(9495:0.013202,(57375:0.001958,52231:0.013202):0.011134):0.001295):0.013202,(9487:0.013202,9491:0.003918):0.013202):0.013202,((9515:0.013202,9514:0.013202):0.012203,(66265:0.011246,70928:0.013202):0.002650):0.000480):0.000208,(((9502:0.013202,30589:0.004507):0.003904,((118643:0.002554,9508:0.000929):0.011049,(9519:0.006771,30594:0.013202):0.000442):0.013202):0.013202,(9505:0.013202,867331:0.002739):0.013202):0.013202):0.004358):0.013202,(((((9597:0.000097,9598:0.002133):0.013202,9593:0.013202):0.001745,((((61851:0.013202,36231:0.013202):0.013202,(9588:0.013202,9580:0.000872):0.004219):0.013202,((29089:0.013202,61852:0.013202):0.013202,9590:0.013202):0.000448):0.003998,9600:0.008932):0.000766):0.007699,9606:0.013202):0.004165,((((75567:0.013202,(9544:0.013202,9546:0.001363):0.001126):0.013202,((9557:0.013202,9565:0.013202):0.013202,(9568:0.001191,((9530:0.003512,9532:0.013202):0.013202,9561:0.013202):0.001271):0.013202):0.000289):0.003910,(((100224:0.006358,9534:0.013202):0.013202,(198627:0.000171,36226:0.002060):0.013202):0.013202,(9538:0.000140,54135:0.013202):0.013202):0.000888):0.001384,((((43780:0.013202,170207:0.013202):0.001246,((54133:0.002705,(78452:0.013202,78451:0.001786):0.013202):0.013202,(61621:0.007952,61622:0.002124):0.002263):0.004169):0.000402,(54180:0.013202,(222416:0.001048,88029:0.001168):0.001743):0.000565):0.013202,(164648:0.013202,(373033:0.005325,(33548:0.026403,54131:0.000348)r1:0.000000):0.000000):0.026403):0.013202):0.013202):0.013202):0.013202):0.004509);

And this is the treePL log:

outfile: /tmp/VU_SAflRaW
setting the maximum number of threads to 8
setting the random number seed to 1234
finished reading config file
tiny branch length at internal node. setting to 3.8220456e-05
tiny branch length at internal node. setting to 3.8220456e-05
tiny branch length at internal node. setting to 3.8220456e-05
setting NFos1 min: 32.8
setting NFos3 min: 6.3
setting NFos4 min: 15.2
setting NFos5 min: 18.8
setting NFos6 min: 49.8
setting NFos7 min: 77.5
setting NFos8 min: 51.6
setting NFos9 min: 16.8
setting NFos1 max: 32.8
setting NFos3 max: 6.3
setting NFos4 max: 15.2
setting NFos5 max: 18.8
setting NFos6 max: 49.8
setting NFos7 max: 77.5
setting NFos8 max: 51.6
setting NFos9 max: 16.8
preorder prep
calculating character durations
setting min and max
setting up all constraints
getting feasible start dates
start rate 0.0013070145
numparams:95
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.0013573857
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.0011756186
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.0011923232
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.0011704216
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.001218504
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.0011420022
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.0014572266
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.001355159
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.0015092535
initial calc: 1e+15
problem initializing. trying again.
attempting to get feasible start rates/dates.
new start rate 0.0013136115
initial calc: 1e+15
Failed setting feasible start rates/dates after 10 attempts. Aborting.

We would be very keen to understand better what is going wrong here and what we might be able to do about it, so that we can use treePL somewhat reliably in a pipeline where a variety of topologies/branch lengths/calibration points are going to be thrown at it. Some possible directions that might work include:

• maybe try more attempts? There's probably still going to be situations that will then just result in an infinite loop so perhaps not a general solution.
• us getting a better understanding of which combinations of calibration points can have this effect so that we can make our adjustments to the input before submitting it to treePL
• treePL itself being able to detect combinations of calibration points that will have this effect and somehow making adjustments on the fly
• or maybe we should modify our input files in ways we are not aware of?

Thanks! Hope you're not getting tired of my issues :)

Rutger

Run command

To run the command, we need to go to folder treePL and then type the file name only or what would be the procedure? Thank you,
Best,
Uzma

Configuration of adolc

I am having trouble configuring the adolc dependency.

sudo ./configure --with-openmp-flag=-fopenmp --prefix=/usr

configure: error: cannot find install-sh, install.sh, or shtool in autoconf "."/autoconf

I am unsure how to get around this...
Thanks.

segmentation fault when using CV

I've run into some issues when attempting cross-validation. Below is the text of an example infile that spawned a segmentation fault for me. I'd run this in the /examples directory. The error is copied below the infile (which I'd called test.cppr8s).

INFILE

treefile = test.tre
smooth = 100
numsites = 1000
mrca = nrt a b
mrca = rt a f
min = nrt 50
max = rt 150
cv
cvstart = .0001
cvstop = 10000
randomcv
cvmultstep = 0.1
[verbose]
[thorough]
lfsimaniter = 0
plsimaniter = 0
[prime]
opt = 5
moredetail
optad = 1
optcvad = 2
moredetailcvad

END INFILE

eastmanium:examples jeastman$ treePL test.cppr8s
cvstart: 0.0001
cvstop: 10000
cvmultstep: 0.1
lf number of simulated annealing iterations: 0
pl number of simulated annealing iterations: 0
setting opt: 5
setting optad: 1
setting optcvad: 2
finished reading config file
switching cvstart cvstop: 10000 -- 0.0001
setting nrt min: 50
setting rt max: 150
preorder prep
calculating character durations
setting min and max
setting up all constraints
getting feasible start dates
start rate 0.0058968011
numparams:7
initial calc: 17434.606
LF SIM
exit siman: 17240.166
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425359
setting NLOPT: LD_LBFGS
result: 3
after opt calc2: 43.425359
LF SIM
exit siman: 43.425359
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425359
setting NLOPT: LD_LBFGS
result: 3
after opt calc2: 43.425359
LF SIM
exit siman: 43.425359
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425359
setting NLOPT: LD_LBFGS
result: -1
after opt calc2: 43.425359
exited lf converged :1
lf calc: 43.425359
numparams:18
smoothing:100
43.425459
exit siman: 43.425459
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425459
setting NLOPT: LD_LBFGS
result: -1
calculating without gradient (might want to try a different optad=VALUE)
setting NLOPT : LN_SBPLX
result: 4
after opt calc2: 43.425459
exit siman: 43.425459
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425459
setting NLOPT: LD_LBFGS
result: 3
after opt calc2: 43.425459
exit siman: 43.425459
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425459
setting NLOPT: LD_LBFGS
result: -1
calculating without gradient (might want to try a different optad=VALUE)
setting NLOPT : LN_SBPLX
result: 4
after opt calc2: 43.425459
exit siman: 43.425459
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425459
setting NLOPT: LD_LBFGS
result: -1
calculating without gradient (might want to try a different optad=VALUE)
setting NLOPT : LN_SBPLX
result: 4
after opt calc2: 43.425459
exit siman: 43.425459
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425459
setting NLOPT: LD_LBFGS
result: -1
calculating without gradient (might want to try a different optad=VALUE)
setting NLOPT : LN_SBPLX
result: 4
after opt calc2: 43.425459
after opt calc: 43.425459
eastmanium:examples jeastman$ treePL test.cppr8s
cvstart: 0.0001
cvstop: 10000
cvmultstep: 0.1
lf number of simulated annealing iterations: 0
pl number of simulated annealing iterations: 0
setting opt: 5
setting optad: 1
setting optcvad: 2
finished reading config file
switching cvstart cvstop: 10000 -- 0.0001
setting nrt min: 50
setting rt max: 150
preorder prep
calculating character durations
setting min and max
setting up all constraints
getting feasible start dates
start rate 0.0057966384
numparams:7
initial calc: 17717.293
LF SIM
exit siman: 17659.802
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425359
setting NLOPT: LD_LBFGS
result: 3
after opt calc2: 43.425359
LF SIM
exit siman: 43.425359
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425359
setting NLOPT: LD_LBFGS
result: -1
calculating without gradient (might want to try a different optad=VALUE)
setting NLOPT : LN_SBPLX
result: 4
after opt calc2: 43.425359
LF SIM
exit siman: 43.425359
setting NLOPT : LN_SBPLX
result: 4
after opt calc1: 43.425359
setting NLOPT: LD_LBFGS
result: -1
after opt calc2: 43.425359
exited lf converged :1
lf calc: 43.425359
conducting cross validation analysis
conducting RANDOM SUBSAMPLE cross validation analysis
sampling 1 tips from 7 total tips 10 times
Segmentation fault

stem vs crown

When constraining the MRCA with a min and max is TreePL constraining the stem or the crown? Is there a way to select this?

Node bars

Hi, how do we get node bars in the final dated tree?
Than you,
Best

Estimating error around dates?

First off, great program! It has solved a lot of problems for me.

I was wondering if you had developed a way to generate error around dates? I seem to remember Sanderson suggesting that error in PL dates could be generated by using 1) the curvature of the likelihood surface (not optimal), or 2) estimating PL dates by throwing in a distribution of trees (bootstraps or Pp).

Any ideas on what do here would be much appreciated (I want to include downstream dating error for comparative phylo methods).

Thanks.

Frank

How drive treePL to estimate divergence time under strict global clock model?

Dear author,

I would like to use treePL to estimate divergence time of a simulated phylogeny under clock model not relaxed one. How can I set the configuration file? I know r8s has LF method with nrates=1, but I have not find the corresponding options in treePL. Or the only way is to set a very large smoothing value, like 100000 or even larger?

Thanks and best wishes,
Liang

Updated Mac install with homebrew

I managed to get treePL compiled with some homebrew magic. This will hopefully make it much easier for people get treepl compiled and running. I will look into using this hard-won knowledge to update the makefiles, etc. to make things easier on both mac and linux.

https://github.com/blackrim/treePL/wiki/Installation#mac
https://github.com/jonchang/homebrew-biology/blob/master/treepl.rb

Failed adolc compilation, ubuntu 16.04

Hello

I tried to compiled the provided adol-c sources, but got the following problem on a ubuntu 16.04 server:

cd deps/adol-c
autoreconf -fi
configure.ac:43: error: possibly undefined macro: AC_LIBTOOL_WIN32_DLL
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:44: error: possibly undefined macro: AC_PROG_LIBTOOL
autoreconf: /usr/bin/autoconf failed with exit status: 1

Anyone with a solution ?
Best

Tristan

'wiggle' option

Hi,
I read in the Bioinformatics paper that treePL provides a ‘wiggle’ option that reports divergence times within 2 log-likelihood units of the best known likelihood. I’d like to use this, but can’t find it listed on the wiki.

I'd just like to know if it is implemented and, if so, what it is called.

Best,
A. Ritchie

treePL not working properly

I am trying to install treePL on Mac 10.13.6. I installed Homebrew and then installed treePL with

brew install brewsci/bio/treepl

When i try to run the provided tests to see if it works properly:
brew test -v treepl
I get the following error:
==> /usr/local/Cellar/treepl/2017.05.23/bin/treePL clock.cppr8s
dyld: Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgomp.1.dylib
Referenced from: /usr/local/Cellar/treepl/2017.05.23/bin/treePL
Reason: image not found
Error: brewsci/bio/treepl: failed
undefined method `map' for nil:NilClass
Did you mean? tap

I would appreciate any help with this issue.

How to reduce the number of constraints

Hi everyone,

I am also using treePL with constraints and I got the error message: "Failed setting feasible start rates/dates after 10 attempts. Aborting."
I obtained ages for the nodes to constrain from a tree, previsouly dated using treePL so they shouldn't be conflicting constraints as you present them : one node close to the tips older than one closer to the root. Is there other cases where constraints can be conflicting that I could check for ?
I am dating a tree with 2340 species and using around 200 constraints, is it too much ? how can I choose the best ones ?

Thank you very much for your answers !

Have a nice day !

Luana
phD student
Toulouse, FRANCE

seqfaults on quoted names

My actual example (an NCBI name) is a doozy, containing a bunch of standard tree symbols (brackets, comma) plus spaces: 'Bernieria madagascariensis (Gmelin, 1789)'.

Easy enough fix (we dealt with this in phyx). I might be able to get to this shortly.