Myrmecos

The trouble with MrBayes

Sorry for an uncharacteristically technical post.  But, I’ve produced an excellent example of a problem that’s been plaguing the widely-used phylogenetics program MrBayes and thought it might be of interest to the handful of systematists who read this blog.

I’ve been running analyses on the Azteca y’all sent after my desperate plea last month and noticed something odd. 

I set MrBayes to do two runs of 4 chains each.  After 10 million generations they produced post-stationarity consensus trees that were topologically identical (that is, the avg. st. dev. splits frequency fell to .005 after 10 million generations and the two trees looked the same).  But the scale bars showed the two analyses off by an order of magnitude in estimating how fast the DNA sequences had changed.  Look:

A portion of Run 1: note the scale bars at right, each tick represents .01 changes/site

The corresponding portion of Run 2: an order of magnitude more evolution

In fact, the two runs agreed on nearly all parameters except for overall likelihood (Run 1, the slow one, was slightly more likely) and the rate multipliers:

PRSF should approach 1 for all parameters as analyses converge; 3 of my partitions had rate multipliers that were well off

What’s going on?

Lucky for me, this anomaly has surfaced before.   A timely paper in Systematic Biology dissects the problem:

…inaccurate branch-length estimates result from either 1) poor mixing of MCMC chains or 2) posterior distributions with excessive weight at long tree lengths. Both effects are caused by a rapid increase in the volume of branch-length space as branches become longer.

I’ve gotten results back before from MrBayes where the rates seem oddly fast.  Since the independent runs normally converge to the same (wrong?) answer, though, I didn’t think much of it.   This Azteca example is the first I’ve seen where the runs fall out both ways.

I hope the next version (purportedly called “RevBayes”, and due out in 2010) resolves the problem.  In the meantime, it’s probably a good idea to check MrBayes rate estimates against estimates from likelihood analyses to make sure they are comparable before publishing.


source: Brown, J. M., Hedtke, S. M., Lemmon, A. R., Lemmon, E. M. 2009. When Trees Grow Too Long: Investigating the Causes of Highly Inaccurate Bayesian Branch-Length Estimates. Systematic Zoology Advance Access published on December 10, 2009, DOI 10.1093/sysbio/syp081.

Comments

  1. #1 Matt Brandley
    December 21, 2009

    The following paper just came out and deals with this very problem: David C. Marshall. 2010. Cryptic Failure of Partitioned Bayesian Phylogenetic Analyses: Lost in the Land of Long Trees. Syst. Biol.

    http://sysbio.oxfordjournals.org/cgi/content/abstract/59/1/108

    It’s an issue with the branch length scaling, and while this may be important for some comparative analyses, it fortunately does not affect topological relationships.

  2. #2 Ted Schultz
    December 21, 2009

    Alex et al.:

    Phil Ward, Sean Brady, Brian Fisher, and I encountered this tree length problem big time in a recent (in press) phylogeny of the Dolichoderinae. The Marshall papers are the key literature on this. What helped a lot was to alter the branch-length priors at the start of the run, adjusting them to 1/10th of the current mrbayes default value. That command is “prset applyto=(all) brlenspr=Unconstrained:Exponential(100); [sets the branch length prior mean to 1/100=0.01]“. We also did two other things, which may or may not have helped: We assigned ratepriors to each of the partitions (e.g., “prset applyto=(1,2,3) ratepr=dirichlet(2,1,10);”) and we changed the proposal frequency for the rateprior MCMC parameter (“props; [values entered in real time from command line as follows: parameter=26; proposal rate=10; Dirichlet=250]“). I am going to run some tests to tease apart the relative effects of each of these three things on the tree length problem. My hunch is that the important one is the first one, reducing the branch length priors.

    Another thing we tried but were unable to do due to a bug in mrbayes was to give mrbayes a starttree with branch lengths and to make all those branch lengths 0.01.

    Ted

  3. #3 myrmecos
    December 22, 2009

    Thanks for the detailed comment, Ted. I just re-ran the analysis twice using the lower branch length priors, and it worked a treat!

  4. #4 Kevin Z
    December 22, 2009

    Alex, thanks for the post. I am trying to self-learn Bayesian phylogenetics (I came from the School of Nei…). In addition to the comments above I recently learned of this paper in PLoS ONE:

    Kolaczkowski B, Thornton JW (2009) Long-branch attraction bias and inconsistency in bayesian phylogenetics. PLoS ONE 4:e7891.

    It suggests that Bayesian methods performs worse (more error) than likelihood methods. But I sort of felt it might be fixed by adjusting the algorithms for the evolutionary models. I need to wade into it more though. But thought you might like the reference if you haven’t yet seen it.

  5. #5 Gunnar
    December 23, 2009

    Is this a problem concerning the Mr.Bayes software package, or is it a problem for Bayesian methods in general?

  6. #6 myrmecos
    December 23, 2009

    Thanks for that paper, Kevin. It seems Jon Eisen has an interview with the author, too.

  7. #7 myrmecos
    December 23, 2009

    I’m still reading through the literature, but from what I gather it seems to be more of an issue of the default settings in MrBayes leading data sets with particular properties into an area where Bayesian inference can fail.

    The advantage of Bayesian methods as I see them is tremendous flexibility in estimating all sorts of parameters simultaneously. ML does not provide as much freedom. With great freedom, of course, comes great responsibility, and these recent problems may be a case of giving biologists too much rope and they’re hanging themselves with it.

  8. #8 Jeremy Brown
    December 23, 2009

    Alex et al.,

    I’m glad that you found our study timely! We delved into this problem because it kept arising in our own work. I’m quite certain that it’s an extremely widespread phenomenon, but often goes unnoticed or unmentioned. I just wanted to comment/follow up on a few things that have been mentioned:

    - Regarding Matt’s point about the effect on topological accuracy: it’s true that topological inference often isn’t affected enough to change the majority-rule consensus tree that one builds from posterior samples. However, for some bipartitions in some data sets, there can be substantial differences in the inferred posterior probability when branch lengths are way off. Take home: it’s best to redo the analyses and fix the problem, if at all possible.

    - Regarding the relationship between our work and that of Marshall: he explores this phenomenon more thoroughly for the case of partitioned analyses, but doesn’t fully tease apart the root causes. Our paper primarily explores unpartitioned analyses and tries to dissect the ultimate causes. I encourage anyone interested in this problem to read both. The phenomenon seems to occur more easily with partitioned analyses (since partitions are scaled independently), but it’s certainly not restricted to cases of partitioning.

    - I think a lot of instances of upwardly biased branch lengths are due to stochastic entrapment (as seems to be the case with Alex’s data, where one run finds the “correct” solution and one seems far too long) and can be fixed (to a large degree) by using starting trees with shorter branch lengths or using a whole-tree scaling move in the MCMC. To the extent that any piece of software (e.g., MrBayes) doesn’t use short starting branch lengths or whole-tree scaling, the problem is with that software.

    - After doing the work for this study, I now firmly believe that exponential branch length priors are going to have an effect on posterior branch lengths unless there is a LOT of data. The effect won’t always be huge (if the default mean is close to the true mean), but it’ll be there. This problem is not specific to MrBayes, but lies in the prior specification that nearly all Bayesian phylogenetic software employs. For now, a temporary fix is for empiricists to think carefully about what they really expect the mean branch length in their tree to be and set the exponential mean accordingly. A longer term solution is to explore other distributions for branch-length priors. I’m now a postdoc in the Huelsenbeck lab and doing some dabbling along these lines.

    If anybody’s interested in talking about this stuff in more detail, feel free to email me.

    Apologies for the long comment!

    Jeremy