Models for dealing with standard discrete morphological data.
Aiming at MK and MKv models from Lewis, 2001 (http://sysbio.oxfordjournals.org/content/50/6/913.short)
A tutorial is available here
Models for dealing with morphological data
License: GNU Lesser General Public License v2.1
Models for dealing with standard discrete morphological data.
Aiming at MK and MKv models from Lewis, 2001 (http://sysbio.oxfordjournals.org/content/50/6/913.short)
A tutorial is available here
If an Mkv model is chosen, constant characters should be removed (or ignored) from the matrix printing a Warning with the details about removed characters.
double should be used instead of integer for calculation line 108
Should the default setting for Lewis MK model be that all frequencies are equal? This is the simplest case of the model and is commonly used for morphological data.
When running the penguins data under Mkv model with frequencies estimated, I get
Likelihood incorrectly calculated: -22865.94213914816 != -22865.76245795682(-0.17968119134093286) Operator: beast.evolution.operators.ScaleOperator
To narrow down the problem, first I only enabled the operators changing the frequencies, it works fine, but once I enable a tree operator, say the treeScaler, I got something like this:
P(posterior) = -22865.76245795682 (was -22865.94213914816) **
P(prior) = -462.11152443659694 (was -462.11152443659694)
P(birthDeath) = -450.6982587466785 (was -450.6982587466785)
P(origin_prior) = -5.075173815233827 (was -5.075173815233827)
P(diversificationRate_prior) = -0.5092379526435218 (was -0.5092379526435218)
P(turnover_prior) = 0.0 (was 0.0)
P(samplingProportion_prior) = 0.0 (was 0.0)
P(MeanRatePrior.c:penguins) = 2.8929944227178996 (was 2.8929944227178996)
P(ucldStdevPrior.c:penguins_morph) = 0.819364145120802 (was 0.819364145120802)
P(ucldStdevPrior.c:penguins_dna) = 0.819364145120802 (was 0.819364145120802)
P(ratePrior.s:morph3) = -1.3068528194400546 (was -1.3068528194400546)
P(ratePrior.s:morph4) = -1.3068528194400546 (was -1.3068528194400546)
P(ratePrior.s:morph5) = -1.3068528194400546 (was -1.3068528194400546)
P(ratePrior.s:morph6) = -1.3068528194400546 (was -1.3068528194400546)
P(gammaShape_morph_prior) = -0.13316535724040832 (was -0.13316535724040832)
P(gammaShape_rag1_prior) = -1.0 (was -1.0)
P(gammaShape_cyt_b_prior) = -1.0 (was -1.0)
P(gammaShape_coi_prior) = -1.0 (was -1.0)
P(gammaShape_12srna_prior) = -1.0 (was -1.0)
P(gammaShape_16srna_prior) = -1.0 (was -1.0)
P(likelihood) = -22403.65093352022 (was -22403.83061471156) **
P(treeLikelihood.morph2) = -825.6129223393757 (was -825.7926035307171) **
P(treeLikelihood.morph3) = -535.803294293985 (was -535.803294293985)
P(treeLikelihood.morph4) = -223.3398850703398 (was -223.3398850703398)
P(treeLikelihood.morph5) = -56.00861874956596 (was -56.00861874956596)
P(treeLikelihood.morph6) = -83.35182917539032 (was -83.35182917539032)
P(treeLikelihood.morph7) = -21.20816564864092 (was -21.20816564864092)
P(treeLikelihood.rag1) = -5564.232312404348 (was -5564.232312404348)
P(treeLikelihood.cyt_b) = -5213.817694448532 (was -5213.817694448532)
P(treeLikelihood.coi) = -2412.530466591211 (was -2412.530466591211)
P(treeLikelihood.12srna) = -3583.1025130164585 (was -3583.1025130164585)
P(treeLikelihood.16srna) = -3884.6432317823724 (was -3884.6432317823724)
At sample 190000
Likelihood incorrectly calculated: -22865.94213914816 != -22865.76245795682(-0.17968119134093286) Operator: beast.evolution.operators.ScaleOperator
Xml to reproduce the error is attached, with a much smaller xml to reproduce the same thing.
When clicking on the plus sign and choosing "add morphological data" nothing happens. If one adds morphological data from the file menu then it works properly.
BEAUti sets Frequencies to be estimated for Lewis MK model by default however there is no operator for Frequencies in the template.
At the beginning of the run, BEAST2 shows:
java.lang.NullPointerException
at beast.evolution.likelihood.BeagleTreeLikelihood.setUpSubstModel(BeagleTreeLikelihood.java:474)
at beast.evolution.likelihood.BeagleTreeLikelihood.initialize(BeagleTreeLikelihood.java:364)
at beast.evolution.likelihood.BeagleTreeLikelihood.initAndValidate(BeagleTreeLikelihood.java:99)
at beast.core.BEASTInterface.initByName(BEASTInterface.java:105)
at beast.evolution.likelihood.TreeLikelihood.initAndValidate(TreeLikelihood.java:132)
at beast.util.XMLParser.initBEASTObjects(XMLParser.java:377)
at beast.util.XMLParser.parse(XMLParser.java:535)
at beast.util.XMLParser.parseFile(XMLParser.java:302)
at beast.app.BeastMCMC.parseArgs(BeastMCMC.java:311)
at beast.app.BeastMCMC.main(BeastMCMC.java:567)
although the run will continue and will have results.
A toy xml will reproduce the problem:
test.xml.zip
I am trying to run beast with morph-traits, using the latest versions from github [beast: commit 892e22a227b37e4c56eedddd2abe03e3885c8a9e, Fri Mar 13 16:13:06 2015 morph-models: commit c11f1d7, Fri Mar 13 16:28:51 2015]. I am running them from Eclipse's debug mode.
Using both a beauti generated beast xml configuration or examples/M3982.xml
from this repository, I get the error logs below.
File: M3982.xml seed: 127 threads: 1
Probing: beagle.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/beagle.jar: contains classs beagle.Beagle that is already loaded
Probing: colt.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/colt.jar: contains classs cern.clhep.PhysicalConstants that is already loaded
Probing: commons-math3-3.1.1.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/commons-math3-3.1.1.jar: contains classs org.apache.commons.math3.linear.OpenMapRealVector$OpenMapEntry that is already loaded
Probing: debug-1.0.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/debug-1.0.jar: contains classs org.jdesktop.swinghelper.debug.CheckThreadViolationRepaintManager$1 that is already loaded
Probing: fest-sources.jar URL file:/usr/home/gereon/lang/beast/beast2/lib/fest-sources.jar is already in the CLASSPATH
Probing: fest.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/fest.jar: contains classs net.jcip.annotations.GuardedBy that is already loaded
Probing: jam.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/jam.jar: contains classs jam.console.ConsoleApplication$1 that is already loaded
Probing: junit-4.8.2.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/junit-4.8.2.jar: contains classs junit.extensions.ActiveTestSuite$1 that is already loaded
Probing: BDSSM.addon.jar Loaded URL file:/home/gereon/.beast/2.2/BDSKY/lib/BDSSM.addon.jar
Probing: GEO.addon.jar Loaded URL file:/home/gereon/.beast/2.2/GEO_SPHERE/lib/GEO.addon.jar
Probing: SubstBMA.addon.jar Loaded URL file:/home/gereon/.beast/2.2/SubstBMA/lib/SubstBMA.addon.jar
Probing: colt.jar Skip loading file:/home/gereon/.beast/2.2/SubstBMA/lib/colt.jar: contains classs cern.clhep.PhysicalConstants that is already loaded
Uniform => beast.math.distributions.Uniform
Exponential => beast.math.distributions.Exponential
LogNormal => beast.math.distributions.LogNormalDistributionModel
Normal => beast.math.distributions.Normal
Beta => beast.math.distributions.Beta
Gamma => beast.math.distributions.Gamma
LaplaceDistribution => beast.math.distributions.LaplaceDistribution
prior => beast.math.distributions.Prior
InverseGamma => beast.math.distributions.InverseGamma
OneOnX => beast.math.distributions.OneOnX
Alignment(M3982)
55 taxa
16 sites
16 patterns
Filter 1,2,4,7,8,12,15,16
55 taxa
8 sites
8 patterns
java.lang.RuntimeException: number of stationary frequencies does not match number of states.
at beast.evolution.substitutionmodel.LewisMK.setFrequencies(LewisMK.java:53)
at beast.evolution.substitutionmodel.LewisMK.initAndValidate(LewisMK.java:82)
at beast.util.XMLParser.initPlugins(XMLParser.java:314)
at beast.util.XMLParser.parse(XMLParser.java:443)
at beast.util.XMLParser.parseFile(XMLParser.java:244)
at beast.app.BeastMCMC.parseArgs(BeastMCMC.java:253)
at beast.app.BeastMCMC.parseArgs(BeastMCMC.java:238)
at beast.app.BeastMCMC.main(BeastMCMC.java:517)
Error 110 parsing the xml input file
validate and intialize error: number of stationary frequencies does not match number of states.
Error detected about here:
<beast>
<run id='mcmc' spec='MCMC'>
<distribution id='posterior' spec='util.CompoundDistribution'>
<distribution id='likelihood' spec='util.CompoundDistribution'>
<distribution id='morphTreeLikelihood.M39822' spec='TreeLikelihood'>
<siteModel id='morphSiteModel.s:M39822' spec='SiteModel'>
<substModel id='LewisMK.s:M39822' spec='LewisMK'>
data
File: test.xml seed: 127 threads: 1
Probing: beagle.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/beagle.jar: contains classs beagle.Beagle that is already loaded
Probing: colt.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/colt.jar: contains classs cern.clhep.PhysicalConstants that is already loaded
Probing: commons-math3-3.1.1.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/commons-math3-3.1.1.jar: contains classs org.apache.commons.math3.linear.OpenMapRealVector$OpenMapEntry that is already loaded
Probing: debug-1.0.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/debug-1.0.jar: contains classs org.jdesktop.swinghelper.debug.CheckThreadViolationRepaintManager$1 that is already loaded
Probing: fest-sources.jar URL file:/usr/home/gereon/lang/beast/beast2/lib/fest-sources.jar is already in the CLASSPATH
Probing: fest.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/fest.jar: contains classs net.jcip.annotations.GuardedBy that is already loaded
Probing: jam.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/jam.jar: contains classs jam.console.ConsoleApplication$1 that is already loaded
Probing: junit-4.8.2.jar Skip loading file:/usr/home/gereon/lang/beast/beast2/lib/junit-4.8.2.jar: contains classs junit.extensions.ActiveTestSuite$1 that is already loaded
Probing: BDSSM.addon.jar Loaded URL file:/home/gereon/.beast/2.2/BDSKY/lib/BDSSM.addon.jar
Probing: GEO.addon.jar Loaded URL file:/home/gereon/.beast/2.2/GEO_SPHERE/lib/GEO.addon.jar
Probing: SubstBMA.addon.jar Loaded URL file:/home/gereon/.beast/2.2/SubstBMA/lib/SubstBMA.addon.jar
Probing: colt.jar Skip loading file:/home/gereon/.beast/2.2/SubstBMA/lib/colt.jar: contains classs cern.clhep.PhysicalConstants that is already loaded
Uniform => beast.math.distributions.Uniform
Exponential => beast.math.distributions.Exponential
LogNormal => beast.math.distributions.LogNormalDistributionModel
Normal => beast.math.distributions.Normal
Beta => beast.math.distributions.Beta
Gamma => beast.math.distributions.Gamma
LaplaceDistribution => beast.math.distributions.LaplaceDistribution
prior => beast.math.distributions.Prior
InverseGamma => beast.math.distributions.InverseGamma
OneOnX => beast.math.distributions.OneOnX
java.lang.Exception: Either input 'stateNumber' or 'datatype' needs to be specified
at beast.core.Input.validate(Input.java:702)
at beast.core.BEASTInterface.validateInputs(BEASTInterface.java:315)
at beast.util.XMLParser.createObject(XMLParser.java:697)
at beast.util.XMLParser.parseInputs(XMLParser.java:830)
at beast.util.XMLParser.createObject(XMLParser.java:693)
at beast.util.XMLParser.parseInputs(XMLParser.java:830)
at beast.util.XMLParser.createObject(XMLParser.java:693)
at beast.util.XMLParser.parseInputs(XMLParser.java:830)
at beast.util.XMLParser.createObject(XMLParser.java:693)
at beast.util.XMLParser.parseInputs(XMLParser.java:830)
at beast.util.XMLParser.createObject(XMLParser.java:693)
at beast.util.XMLParser.parseInputs(XMLParser.java:830)
at beast.util.XMLParser.createObject(XMLParser.java:693)
at beast.util.XMLParser.parseRunElement(XMLParser.java:561)
at beast.util.XMLParser.parse(XMLParser.java:442)
at beast.util.XMLParser.parseFile(XMLParser.java:244)
at beast.app.BeastMCMC.parseArgs(BeastMCMC.java:253)
at beast.app.BeastMCMC.parseArgs(BeastMCMC.java:238)
at beast.app.BeastMCMC.main(BeastMCMC.java:517)
Error 110 parsing the xml input file
validate and intialize error: Either input 'stateNumber' or 'datatype' needs to be specified
Error detected about here:
<beast>
<run id='mcmc' spec='MCMC'>
<distribution id='posterior' spec='util.CompoundDistribution'>
<distribution id='likelihood' spec='util.CompoundDistribution'>
<distribution id='treeLikelihood.anolis' spec='TreeLikelihood'>
<siteModel id='SiteModel.s:anolis' spec='SiteModel'>
<substModel id='LewisMK.s:anolis' spec='LewisMK'>
The combination shows BEAST Model Test as site model when importing morphological data. Perhaps the MM template has a site model name that does not match the default gamma site model?
BEAST2 packages should not use Java package names beast.* as this increases the likelihood of package name clashes in BEAST2 installations when multiple packages clash with each other or the core. BEAST2 packages should have their own top level package name.
When importing data where there is a constant site, the importer creates a partition for that site, but the LewisMK model fails at getTransitionProbabilities since it tries to calculate stateCount/(stateCount-1)
, which with stateCount=1
gives a division by zero error.
Possible solutions:
When using the ordinal model and ordinal data with a different number of states, beast fails to initialize the model giving -infinite
likelihood.
This is an input user error and should be caught during the model verification stage (such as when the sum of frequencies is checked) and communicated with an appropriate error.
During creating a sample test case with short sequences, beast even managed to run (although gave a few "incorrect likelihood was calculated" errors), this should not have happened.
Suggested fix: Additional assert/check that the number of states in the model agrees with the data.
Sample example: XML with randomly generated sequences and, to my best knowledge, vanilla ordinal model specification.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.