State-Dependent Diversification

Macroevolution · PB 495/595 Plant Evolutionary Biology

George P. Tiley

3 November 2026

Extensions of speciation and extinction

  • Lineages are born and die
  • Species richness is uneven
  • Sister clade comparisons can show that diversification rates \(r\) vary.
  • Today: Is speciation and extinction influenced by traits?
  • Thursday: practical analysis of the Goldberg et al. (2010) data.

Learning objectives

By the end you should be able to:

  • Conceptualize how traits are associated with diversification based on empirical examples
  • Apply basic concepts of likelihood to speciation and extinction rates
  • Explain why speciation \(\lambda\) and extinction \(\mu\) are harder to estimate than the net diversification rate \(r\)
  • Follow the basic concept of the Binary State-dependent Speciation and Extinction (BiSSE) model
  • Explain why SSE models give false positives, and how hidden-states address this

Empirical examples

Heterostyly

  • Basic tenet of population genetics: bigger populations mean more genetic diversity
  • When genetic diversity is high, selection can act more efficiently
  • Thus, more diversity should mean lower extinction rates

A great way to increase genetic diversity is to have mechnical barriers to inbreeding!

Heterostyly in Primulaceae

Example of self-incompatibility through heterostyly.

Fig. 1 from de Vos et al. (2014). Only the different stigma and anther morphs are compatible in Primula. Morphs are controlled by a single bi-allelic locus.

Heterostyly in Primulaceae

Variation in diversification rates and extinction fractions in Primulaceae.

Fig. 2 from de Vos et al. (2014). Variable diversification rates and extinction fractions associated with clades of Primulaceae

Heterostyly

Heterosytly by the numbers

Rank Count
Genera 199
Families 28
Orders 15
Origins 20

Multiple indepdent origins of mechanisms to promote outcrossing. Does that mean outcrossing always confers a fitness benefit?

Photosynthesis in Poaceae

C4 photosynthesis in Poaceae.

Fig. 1 from Spriggs, Christin, and Edwards (2014). Distribution of C4 photosynthesis in Poaceae

Photosynthesis in Poaceae

Was diversification of C4 lineages a response to changing climate and ecosystems?

C4 photosynthesis in Poaceae.

Fig. 2a from Spriggs, Christin, and Edwards (2014). Evidence for higher net diversification rates in C4 lineages compared to C3 lineages in Poaceae

Polyploidy across angiosperms

There is an apparent cost to polyploidy. A reason for rediploidization?

Polyploidy across angiosperms.

Fig. 1 from Mayrose et al. (2011). Diploids more frequently have higher diversification than polyploids across genera.

Sex determination systems in plants

  • Hermaphroditism - male and female reproductive organs on the same individual, often in the same flower. Most common in angiosperms and assumed the ancestral state.

  • Monoecy - male and female reproductive organs on the same individual, but separate flowers. Corn is a classic example.

  • Dioecy - separate male and female individuals. Plants can have sex chromosomes and sex determination systems too. Aparagus is a classic XY example.

Sex determination systems in plants

Sex determination in garden asparagus.

Fig. 1 from Harkess et al. (2020). Sex determination by two Y-linked genes in garden asparagus

Sex determination systems in plants

System Mechanism Examples
XY (male heterogametic) Y carries male-promoting and female-suppressing factors Asparagus officinalis, Carica papaya, Phoenix dactylifera
ZW (female heterogametic) W carries the female-determining factor Salix purpurea, Populus alba
Haploid (U/V) Sex is set in the gametophyte, not the sporophyte Marchantia polymorpha, Ceratodon purpureus
Multiple sex chromosomes Several X or Y chromosomes, dosage dependent Sorrels (Polygonaceae) - Rumex acetosa, R. hastatulus

Sex determination systems in plants

Not all traits lead to differences in diversification rates among lineages. Although Dioecy is rare, there is no consistent pattern across angiosperms that it has less lineages per unit time. Even when accounting for other factors.

Dioecy across angiosperms.

Fig. 2 from Sabath et al. (2016). Dioecy is rare across angiosperms, but does not consistently lead to lower diversification rates.

Modeling discrete traits

The Mk model

  • A trait with discrete states (e.g. self-compatible or self-incompatible) changes along the branches at rates in a matrix \(Q\) (Lewis 2001):

\[ Q = \begin{pmatrix} -q_{01} & q_{01} \\ q_{10} & -q_{10} \end{pmatrix} \]

  • It’s the same continuous-time Markov model as a DNA substitution model, with two states instead of four.
  • Apply the same principles of maximum likelihood to estimate the rates.

Birth–death models

From counting events to a likelihood

  • Lineages split at rate \(\lambda\) and die at rate \(\mu\).
  • Net diversification is \(r = \lambda - \mu\)
  • The extinction fraction is \(\epsilon = \mu/\lambda\).
  • Given a dated tree, which rates surprise us the least?

From counting events to a likelihood

Motivation for a model - many events go unobserved!

Left: a complete tree with blue dots at speciation events and crosses where lineages went extinct. Right: the same tree with the extinct lineages removed.

The Yule likelihood

With \(k\) lineages, the wait to the next split is Exponential(\(k\lambda\)). Multiply over the waits. The last one reaches the present without a split:

\[ L(\lambda) = \Big[\prod_{k=2}^{n-1} k\lambda\, e^{-k\lambda T_k}\Big]\, e^{-n\lambda T_n} \]

  • Maximizing \(\lambda\) gives the number of splits divided by the total branch length.

A six-tip tree with dashed lines at each split; the intervals are labelled k = 2 to 6 lineages, with waiting times Exp(2 lambda) to Exp(6 lambda).

Adding extinction: two probabilities instead of one

Need to keep track of lineages that might go extinct or be unsampled too . - \(D(t)\): probability a lineage alive at time \(t\) gives rise to the observed children. - \(E(t)\): probability a lineage alive at time \(t\) leaves no sampled descendants.

  • With constant rates and every species sampled, \(r = \lambda - \mu\) and \[ D(t) = \frac{r^2 e^{-rt}}{\left(\lambda - \mu e^{-rt}\right)^2}, \qquad E(t) = \frac{\mu\left(1 - e^{-rt}\right)}{\lambda - \mu e^{-rt}}. \]

Statistical non-identifiability of speciation and extinction rates

  • Within 2 log-likelihood units of the MLE, \(\lambda\) ranges from 0.94–1.44 and \(\mu\) from 0.46–1.12.
  • \(r\) is between 0.26–0.52, so we need to measure uncertainty in diversification.
  • This is the easy case, with constant rates. Rate changes through time are more difficult (Louca and Pennell 2020).

Contour map of log-likelihood against speciation and extinction rate for one simulated tree: a narrow diagonal ridge runs along a dashed line of equal net diversification, with the true rates and the best fit both inside it.

Not every species is in the tree

  • Trees rarely have every species. This Thursday’s tree has 356 of about 2,300.
  • Missing species are unsampled taxa and the model does not known those are not extinct.
  • Overcome with the sampling fraction \(f\), the chance that a species made it into the tree (FitzJohn, Maddison, and Otto 2009). Now \(E\) starts at the tips from \(E(0) = 1 - f\), not from 0.
  • BiSSE can take a different \(f\) for each state.
  • \(f\) is something we assume, not something the tree tells us. It treats sampling as random.

Combining Mk and birth-death to get BiSSE

When trait changes affect diversification

Three simulated trees with about 75 percent of tips in state 1, one each from faster speciation in state 1, lower extinction in state 1, and more gains of state 1.

From Mk to BiSSE

  • BiSSE lets two states have their own speciation and extinction rates (Maddison, Midford, and Otto 2007).
  • Six parameters: \(\lambda_0, \lambda_1, \mu_0, \mu_1, q_{01}, q_{10}\).
  • Trait change and diversification are fit together, on the same tree.
  • We can test the hypothesis: is net diversification in state 1 different from state 0?

Four possibilities for a single lineage

\(D_i(t)\) and \(E_i(t)\) are now estimated for each state \(i\). Here is what can happen to it in a short time \(dt\):

Four panels of a short lineage segment. Nothing happens, with probability one minus the sum of the rates times dt. The trait changes, with probability q dt. A split where the side lineage is lost, with probability two lambda E dt. Extinction, greyed out and marked as impossible on a branch we see.
  • A split counts twice, because either daughter could be the one that’s lost.
  • Extinction can’t happen on a branch that reached the present, so \(\mu\) is only accounted for in \(E_i(t)\).

BiSSE is Mk AND birth–death

Allow differnet speciation and extition rates for different traits!

The allows us to test hypotheses by contstraining or relaxing parameters.

Asking a question with BiSSE

Fit with and without each link, then compare by AIC → Oct 6.

lik <- make.bisse(tree, states)
same_rates <- constrain(lik, lambda1 ~ lambda0, mu1 ~ mu0)
speciation_differs <- constrain(lik, mu1 ~ mu0)
extinction_differs <- constrain(lik, lambda1 ~ lambda0)
Model Rates \(\ln L\) \(\Delta\)AIC
same rates 4 −191.0 7.0
speciation differs 5 −186.5 0.0
extinction differs 5 −189.4 5.9
both differ 6 −186.5 2.0

This tree was simulated with different speciation and equal extinction, and AIC picks that model.

Simulated tree with about 100 tips colored by state; state 1, which speciates faster, makes up most tips and forms the bushier clades.

Plant results from SSE models

Trait Group Result
Heterostyly Primroses faster net diversification, through lower extinction (de Vos et al. 2014)
C4 photosynthesis Grasses higher diversification, often after a lag (Spriggs, Christin, and Edwards 2014)
Self-incompatibility Nightshades SI lineages diversify faster (Goldberg et al. 2010)
Polyploidy Many genera polyploids have lower speciation and higher extinction (Mayrose et al. 2011)
Dioecy Dozens of genera faster in some genera, slower in others (Sabath et al. 2016)

Hidden states

The false-positive problem

  • Real trees have rate shifts for many reasons. If a neutral trait happens to sit in a fast clade, BiSSE often credits the trait (Rabosky and Goldberg 2015).
  • BiSSE compares “the trait changes rates” with “rates never vary”. Any rate variation could lead to a false positive.
  • Power is also low on small trees or with rare states. Be careful below about 300 species, or when fewer than 10% are in one state (Davis, Midford, and Maddison 2013).

Simulated tree in which one shaded, fast-diversifying clade carries trait state 1, while the trait has no effect on rates.

Adding a hidden trait

Add a trait we never observed, A or B, with its own rates of change. It can also change \(\lambda\) and \(\mu\), so rate shifts have somewhere else to go (Beaulieu and O’Meara 2016). Boxes of one color share rates:

Grid of rate classes. BiSSE: states 0 and 1 in two colors. CID-2: 0A and 1A share one color, 0B and 1B another. HiSSE: 0A, 1A, 0B and 1B in four colors. CID-4: eight boxes in four colors, with each pair of observed states under a hidden class sharing one.

CID is character-independent diversification: the CID-2 and CID-4 rows, where both observed states share a color. Details next.

In hisse, rates go in the order 0A, 1A, 0B, 1B: CID-2 is c(1, 1, 2, 2), HiSSE is c(1, 2, 3, 4).

Character-Independent Diversification (CID)

  • Character: the observed trait we care about, state 0 or 1.
  • Character-independent: \(\lambda\) and \(\mu\) do not depend on that trait. State 0 and state 1 share rates within a hidden class.
  • Rates still vary among lineages. They depend on a hidden trait, A or B, and a lineage can switch between hidden classes along the tree.
  • The observed trait still evolves between 0 and 1. It has no effect on speciation or extinction.
What the rates depend on Model
nothing observed, but a hidden trait varies CID
the observed trait BiSSE
the observed trait and a hidden trait HiSSE

HiSSE and the CID null

Model Hidden classes Rate classes Rates depend on
BiSSE none 2 the observed trait
CID-2 2 2 a hidden trait only
HiSSE 2 4 observed and hidden traits
CID-4 4 4 a hidden trait only

The observed trait only helps explain diversification if BiSSE beats CID-2 or if HiSSE beats CID-4. Otherwise background rate variation explains the data just as well.

Plant examples with hidden states

  • Onagraceae: self-compatible lineages have higher extinction and lower net diversification, even with hidden states in the model (Freyman and Höhna 2019).
  • Nightshades: diversification is explained better by breeding system plus an unmeasured factor than by ploidy. Polyploidy mostly arose straight from self-incompatible lineages, when whole-genome duplication broke self-incompatibility (Zenil-Ferguson et al. 2019).

Plant examples with hidden states

A helpful way to express models is to draw them. The \(q_{ij}\) parameters can be constrained to test hypotheses.

A model of self incompatibility with hidden states

Fig. 5 from Freyman and Höhna (2019). A model of self incompatibility with hidden states.

Plant examples with hidden states

The self-incompatible vs self-compatible comparison is still consequential and consistent with previous findings.

Fig. 6 from Freyman and Höhna (2019). The self-incompatible vs self-compatible comparison is still consequential and consistent with previous findings.

Plant examples with hidden states

Increasing levels of complexity can be incorporated, such as a third polyploid self-compatible state.

A model of self incompatibility and polyploidy with hidden states

Fig. 2 from Zenil-Ferguson et al. (2019). A model of self incompatibility and polyploidy with hidden states.

What drives angiosperm diversification?

  • A review of 152 SSE studies on angiosperms found that reproductive and morphological traits were often linked to diversification, but no trait had a consistent effect across clades (Helmstetter et al. 2023).
  • Results also tracked the data: larger, older or less completely sampled trees more often found a trait effect.
  • So treat SSE results as hypotheses, compare against hidden-state nulls, and report tree size and sampling.

SSE models and their extensions

Many traits remain to be explored extensively!

Distribution of SSE studies across angiosperms

Fig. 2b from Helmstetter et al. (2023). Distribution of SSE studies across angiosperms.

The SSE family

Key takeaways

Summary

  • A dated tree gives a likelihood for birth–death rates. Extinction leaves a trace, but \(\lambda\) and \(\mu\) suffer from non-identifiability, so \(r\) is usually discussed.
  • BiSSE is Mk and birth–death joined by state-dependent rates.
  • BiSSE is the simplest SSE model, handling binary traits. This can be extended and used to test sophisticated hypotheses.
  • SSE tests give false positives easily, so compare them against hidden-state (CID) nulls.

For Thursday

  • We will use the Goldberg et al. (2010) data.
  • We will use the diversitree R package. Consider installing before class: install.packages("diversitree").
  • We’ll fit BiSSE to the nightshades, then see how far we get with HiSSE.
  • Is selfing an evolutionary dead end?

References

Beaulieu, Jeremy M., and Brian C. O’Meara. 2016. “Detecting Hidden Diversification Shifts in Models of Trait-Dependent Speciation and Extinction.” Systematic Biology 65 (4): 583–601. https://doi.org/10.1093/sysbio/syw022.
Davis, Matthew P, Peter E Midford, and Wayne Maddison. 2013. “Exploring Power and Parameter Estimation of the BiSSE Method for Analyzing Species Diversification.” BMC Evolutionary Biology 13 (1): 38. https://doi.org/10.1186/1471-2148-13-38.
de Vos, Jurriaan M., Colin E. Hughes, Gerald M. Schneeweiss, Brian R. Moore, and Elena Conti. 2014. “Heterostyly Accelerates Diversification via Reduced Extinction in Primroses.” Proceedings of the Royal Society B: Biological Sciences 281 (1784): 20140075. https://doi.org/10.1098/rspb.2014.0075.
FitzJohn, Richard G. 2010. “Quantitative Traits and Diversification.” Systematic Biology 59 (6): 619–33. https://doi.org/10.1093/sysbio/syq053.
———. 2012. “Diversitree: Comparative Phylogenetic Analyses of Diversification in R.” Methods in Ecology and Evolution 3 (6): 1084–92. https://doi.org/10.1111/j.2041-210X.2012.00234.x.
FitzJohn, Richard G., Wayne P. Maddison, and Sarah P. Otto. 2009. “Estimating Trait-Dependent Speciation and Extinction Rates from Incompletely Resolved Phylogenies.” Systematic Biology 58 (6): 595–611. https://doi.org/10.1093/sysbio/syp067.
Freyman, William A., and Sebastian Höhna. 2018. “Cladogenetic and Anagenetic Models of Chromosome Number Evolution: A Bayesian Model Averaging Approach.” Systematic Biology 67 (2): 195–215. https://doi.org/10.1093/sysbio/syx065.
———. 2019. “Stochastic Character Mapping of State-Dependent Diversification Reveals the Tempo of Evolutionary Decline in Self-Compatible Onagraceae Lineages.” Systematic Biology 68 (3): 505–19. https://doi.org/10.1093/sysbio/syy078.
Goldberg, Emma E., and Boris Igić. 2012. “Tempo and Mode in Plant Breeding System Evolution.” Evolution 66 (12): 3701–9. https://doi.org/10.1111/j.1558-5646.2012.01730.x.
Goldberg, Emma E., Joshua R. Kohn, Russell Lande, Kelly A. Robertson, Stephen A. Smith, and Boris Igić. 2010. “Species Selection Maintains Self-Incompatibility.” Science 330 (6003): 493–95. https://doi.org/10.1126/science.1194513.
Harkess, Alex, Kun Huang, Ron van der Hulst, Bart Tissen, Jeffrey L Caplan, Aakash Koppula, Mona Batish, Blake C Meyers, and Jim Leebens-Mack. 2020. “Sex Determination by Two y-Linked Genes in Garden Asparagus.” The Plant Cell 32 (6): 1790–96.
Helmstetter, Andrew J., Rosana Zenil-Ferguson, Hervé Sauquet, Sarah P. Otto, Marcos Méndez, Mario Vallejo-Marin, Jürg Schönenberger, et al. 2023. “Trait-Dependent Diversification in Angiosperms: Patterns, Models and Data.” Ecology Letters 26 (4): 640–57. https://doi.org/10.1111/ele.14170.
Höhna, Sebastian, Michael J. Landis, Tracy A. Heath, Bastien Boussau, Nicolas Lartillot, Brian R. Moore, John P. Huelsenbeck, and Fredrik Ronquist. 2016. “RevBayes: Bayesian Phylogenetic Inference Using Graphical Models and an Interactive Model-Specification Language.” Systematic Biology 65 (4): 726–36. https://doi.org/10.1093/sysbio/syw021.
Lewis, Paul O. 2001. “A Likelihood Approach to Estimating Phylogeny from Discrete Morphological Character Data.” Systematic Biology 50 (6): 913–25. https://doi.org/10.1080/106351501753462876.
Louca, Stilianos, and Matthew W. Pennell. 2020. “Extant Timetrees Are Consistent with a Myriad of Diversification Histories.” Nature 580 (7804): 502–5. https://doi.org/10.1038/s41586-020-2176-1.
Maddison, Wayne P., Peter E. Midford, and Sarah P. Otto. 2007. “Estimating a Binary Character’s Effect on Speciation and Extinction.” Systematic Biology 56 (5): 701–10. https://doi.org/10.1080/10635150701607033.
Mayrose, Itay, Shing H. Zhan, Carl J. Rothfels, Karen Magnuson-Ford, Michael S. Barker, Loren H. Rieseberg, and Sarah P. Otto. 2011. “Recently Formed Polyploid Plants Diversify at Lower Rates.” Science 333 (6047): 1257–57. https://doi.org/10.1126/science.1207205.
Rabosky, Daniel L., and Emma E. Goldberg. 2015. “Model Inadequacy and Mistaken Inferences of Trait-Dependent Speciation.” Systematic Biology 64 (2): 340–55. https://doi.org/10.1093/sysbio/syu131.
Sabath, Niv, Emma E. Goldberg, Lior Glick, Moshe Einhorn, Tia-Lynn Ashman, Ray Ming, Sarah P. Otto, Jana C. Vamosi, and Itay Mayrose. 2016. “Dioecy Does Not Consistently Accelerate or Slow Lineage Diversification Across Multiple Genera of Angiosperms.” New Phytologist 209 (3): 1290–300. https://doi.org/10.1111/nph.13696.
Spriggs, Elizabeth L., Pascal-Antoine Christin, and Erika J. Edwards. 2014. “C4 Photosynthesis Promoted Species Diversification During the Miocene Grassland Expansion.” PLoS ONE 9 (5): e97722. https://doi.org/10.1371/journal.pone.0097722.
Zenil-Ferguson, Rosana, J. Gordon Burleigh, William A. Freyman, Boris Igić, Itay Mayrose, and Emma E. Goldberg. 2019. “Interaction Among Ploidy, Breeding System and Lineage Diversification.” New Phytologist 224 (3): 1252–65. https://doi.org/10.1111/nph.16184.