Our paper, limits to the detection of early warning signals for critical transitions, has now appeared in the Proceedings of the Royal Interface. An open-access copy is available under CC-by license on the ArXiv. The essence of the paper is to bring a more model-based approach to early warning signals, while seeking to remain as general as possible.
Without a model-based approach, it is difficult to leverage statistically meaningful comparisons to identify what is a real warning signal and what is just noise. We use two simple models, (a stationary OU process and temporally heterogeneous OU process) combined with a likelihood ratio bootstrap to do this, though other models and comparisons could be equally promising.
The motivation for this paper comes somewhat from the appendices in the groundbreaking work by Dakos et al 2008, which compares several potential null distributions and shows that Kendall’s tau, the correlation statistic used to identify an increase, takes on almost any value in its range, which doesn’t leave us with much information to go by. I both wanted to confirm this distribution and see if I could improve upon it with the model-based approach. I was glad to see that not only could we quantify uncertainty more precisely with the model based-approach, but also improve upon the sensitivity relative to traditional indicators, as seen in our ROC curves.
My talk at ESA last year gave an introduction to these results.
An Open Notebook Story
This project has gone through many iterations, dead ends, and transformations as it evolved over the past year and a half alongside my other research projects. In the process, there’s also been ample cross-pollination of ideas from different projects. The interplay of these ideas unfolds in the interleaved pages of my online lab notebook.
Here are a few examples of this cross-pollination: Using the likelihood ratio bootstrap as the most powerful comparison of two composite (parameter-estimated) hypotheses first emerged in my evolutionary work which became the paper, Is your phylogeny informative, while solving the likelihood equation for the temporally heterogeneous OU process has it’s roots in my work on the evolutionary release of constraint that became the paper Modeling Stabilizing Selection: Expanding the Ornstein-Uhlenbeck Model of Adaptive Evolution and plays a role in my current wrightscape project. These projects share a central challenge of detecting faint signals of regime shifts from large data sets whilst quantifying uncertainty, despite substantial differences in the details of both model formulation and solution.
The project had several dead ends of things that sounded promising but turn out not to work. A few that I’ve been asked about since include solving the model inference problem in the frequency (Fourier) space (doesn’t work because the power transform destroys the Gaussian property of the noise), and estimating quadratic models (such as the canonical form for the saddle node bifurcation) (page forward on each if your curious).
The project by numbers
My notebook contains 123 posts relevant to this project, tagged under warning-signals, dating back to October 2010, out of a total of 388 posts. The flickr archive has 903 images tagged for this project, the github repository have seen over 40,000 lines of code, mostly in different explorations, consisting of 64 files with commits on 74 days. (This repository has since been split into separate repositories for the [population dynamics simulations] and the early warning signals computations.) Simulations were run on at least four different machines, including the 512 node cluster on campus and NERSC’s Carver supercomputing facility.
Accessing the data and code
My organizational structure has evolved over the course of this project, but mostly conforms to my Github-based R-package structure within the earlywarning repository. The inst/examples directory includes code to reproduce the analysis for the chemostat, simulation, and glaciation data.
References
Boettiger, C., and Hastings, A. (2012). Quantifying limits to detection of early warning for critical transitions. Journal of The Royal Society Interface, 1-29. doi:10.1098/rsif.2012.0125
Dakos, V., Scheffer, M., van Nes, E. H., Brovkin, V., Petoukhov, V., and Held, H. (2008). Slowing down as an early warning signal for abrupt climate change. Proceedings of the National Academy of Sciences, 105(38), 14308-12. doi:10.1073/pnas.0802430105