Lab Notebook

Notes

2013-05-20T00:00:00-07:00

EWS TE revisions

Finalized manuscript(?) Checking references, reply letter, closing remaining issues.
Very annoying to submit to a journal system that takes LaTeX but not any external style or class file dependencies. For instance, how does one add two footnotes to the same author without adding any such dependencies (oh, and writing macros that do not involve pairs of $ which pandoc mistakes for its own macros…) Reasonable author affiliations done otherwise using \and, \thanks, and \footnotemark; see simple.latex pandoc template.
Final edits from Alan.

prosecutor comment revisions

added Figure for Allee model into reply document.
added code for generating figures Figure1.Rmd
combined all four data files into a single tidy data.frame as csv.

ropensci

Example API queries using Whitehouse Open Data CSV-API tool.

CSV file (as displayed from the API)
XML query
JSON query

rfishbase queries

Continue to get questions about performing queries on fishbase that are just not possible with their current awful data model. This time: how can one extract the 400 natural mortality rate estimates mentioned on this page. No, the page entitled popgrowth doesn’t actually show you the table.

pdg-control: policycosts

Over the weekend, re-ran the policy costs analysis with the simpler apples-to-apples comparison (after stream-lining the code a bit more). General patterns remain the same. Also added a block for showing how well we are doing on our grid sampling at getting to a shared npv0 level.

Comparison when adjustment costs reduce net present value by 50% pdg_control/582cc26
The original 25% reduction (e.g. to 75% of the NPV) pdg_control/59dca5f

Misc

Notebook

obfuscated email via javascript, copy-paste and mailto: should still work. (Of course a scraper could use the very javascript code to access the email addresses, but I find this preferable to a solution that intenionally burdens the user). Spam filters should handle the rest.
Interesting reading about signing individual Git commits

Null Distribution Width Minor Puzzle

2013-05-17T00:00:00-07:00

A quick foray into trying to understand why I see the wider distribution (though still symmetric) in the null of the OU model then in the null from the Allee model in the Prosecutor’s fallacy.

Load the original run of the ou model and increase the nulldt data frame to use all points instead of a sample of length 5000

load("beer_run.rda")

ou_dat <- dat
ou_null <- nulldat
#ou_null_ts <- nulldt

null <- timeseries #[1000:6010,]
null <- as.data.frame(cbind(time = 1:dim(null)[1], null))
ndf <- melt(null, id="time")
names(ndf) = c("time", "reps", "value")
levels(ndf$reps) <- 1:length(levels(ndf$reps)) # use numbers for reps instead of V1, V2, etc
nulldt <- data.table(ndf)
ou_null_ts <- nulldt

Plot the final distribution of indicator statistics, showing the width of the null,

ggplot(dat) + geom_histogram(aes(value, y=..density..), binwidth=0.3, alpha=.5) +
 facet_wrap(~variable) + xlim(c(-1, 1)) + 
  geom_density(data=nulldat, aes(value), adjust=2) + xlab("Kendall's tau") + theme_bw()

plot of chunk unnamed-chunk-4

Load the allee model and rename variables appropriately,

load("comment_run.rda")
allee_dat <- dat
allee_null <- nulldat
allee_null_ts <- nulldt

For the Allee model, also plot the final distribution of indicator statistics, showing the width of the null,

ggplot(dat) + geom_histogram(aes(value, y=..density..), binwidth=0.3, alpha=.5) +
 facet_wrap(~variable) + xlim(c(-1, 1)) + 
  geom_density(data=nulldat, aes(value), adjust=2) + xlab("Kendall's tau") + theme_bw()

plot of chunk unnamed-chunk-6

Tidy the data and plot a single replicate from each. Note the OU process has the correspondingly much wider null distribution due to

allee_x <- subset(allee_null_ts, reps==1)
ou_x <- subset(ou_null_ts, reps==1)
allee_x <- data.frame(time = allee_x$time, value = allee_x$value)
ou_x <- data.frame(time = ou_x$time, value = ou_x$value)

ggplot(allee_x, aes(time, value)) + geom_point()

plot of chunk unnamed-chunk-7

ggplot(ou_x, aes(time, value)) + geom_point()

plot of chunk unnamed-chunk-7

Note that subsampling the data at coarser interval doesn’t matter

warningtrend(ou_x[seq(1, length(ou_x$value),by=50),], window_var)

    tau 
-0.5527

Nor does scaling matter (recall Kendall’s is a rank-correlation test)

warningtrend(data.frame(time=ou_x$time, value=(ou_x$value+1)*500), window_var)

    tau 
-0.5516

warningtrend(data.frame(time=ou_x$time, value=ou_x$value), window_var)

    tau 
-0.5516

Lengthening the sample suggests the sampling is not long enough to have converged in distribution of this statistic. Computing on this fine sampling resolution over adequate length of time quickly becomes prohibitive.

sapply(seq(1000, 20000, by=1000), function(i) warningtrend(ou_x[1:i,], window_var))

       tau        tau        tau        tau        tau        tau 
 0.1802475 -0.0832527  0.6365570  0.1787726 -0.1747774 -0.5295226 
       tau        tau        tau        tau        tau        tau 
-0.5403782 -0.6790710 -0.6114706 -0.2950858 -0.0444861  0.0006919 
       tau        tau        tau        tau        tau        tau 
-0.2579410 -0.5220490 -0.6439519 -0.6221482 -0.5545048 -0.5134236 
       tau        tau 
-0.5265975 -0.5515807

Notes

2013-05-16T00:00:00-07:00

Prosecutor

Add figures for both examples with and without ASS
text to accomidate both examples
ROC curves?
proof-reading

ews-review

looking over revisions, touch-ups
See issues log for details

Links

Impressive set of tools provided by the Whitehouse open data project In particular, see the tool for generating a RESTful API from CSV files and the “common core” metadata definitions. Notebook complies with many of these (throught the Dublin core RDFa), but looks like I could benefit from adding some more terms from the Data Catalog Vocabulary. Awesome that this site is built with Twitter-Bootstrap and Jekyll, hosted on Github, and licenced as CC-BY (content) and MIT (code). See for example, the issues log.

Notebook

added black-white theme for readers that prefer higher contrast / more traditional appearance. Otherwise matches the orginal feel reasonably well. (#70)
Dropped the local javascript based search using the stemming engine (#7). The stemming search was reasonably fast, but only matches words instead of phrases (and adds considerable overhead to generating the site).

Notes

2013-05-14T00:00:00-07:00

Exploring potential plots that would allow for some visualization of the uncertainty in each of the models. Example follows up on analysis of e97043d/allen.md, as also in the script from b5d78b9/step_ahead_plots.md

One-step ahead predictor plots and forecasting plots

Set our usual plotting options for the notebook

opts_chunk$set(tidy = FALSE, warning = FALSE, message = FALSE, cache = FALSE, 
    comment = NA, fig.width = 7, fig.height = 4)
library(ggplot2)
opts_knit$set(upload.fun = socialR::flickr.url)
theme_set(theme_bw(base_size = 12))
theme_update(panel.background = element_rect(fill = "transparent", colour = NA), 
    plot.background = element_rect(fill = "transparent", colour = NA))
cbPalette <- c("#000000", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", 
    "#D55E00", "#CC79A7")

Now we show how each of the fitted models performs on the training data (e.g. plot of the step-ahead predictors). For the GP, we need to predict on the training data first:

gp_f_at_obs <- gp_predict(gp, x, burnin=1e4, thin=300)

For the parametric models, prediction is a matter of feeding in each of the observed data points to the fitted parameters, like so

step_ahead <- function(x, f, p){
  h = 0
  x_predict <- sapply(x, f, h, p)
  n <- length(x_predict) - 1
  y <- c(x[1], x_predict[1:n])
  y
}

Which we apply over each of the fitted models, including the GP, organizing the “expected” transition points (given the previous point) into a data frame.

df <- melt(data.frame(time = 1:length(x), stock = x, 
                GP = gp_f_at_obs$E_Ef,
                True = step_ahead(x,f,p),  
                MLE = step_ahead(x,f,est$p), 
                Parametric.Bayes = step_ahead(x, f, bayes_pars), 
                Ricker = step_ahead(x,alt$f, ricker_bayes_pars), 
                Myers = step_ahead(x, Myer_harvest, myers_bayes_pars)
                 ), id=c("time", "stock"))

ggplot(df) + geom_point(aes(time, stock)) + 
  geom_line(aes(time, value, col=variable)) +
    scale_colour_manual(values=colorkey)

Posterior predictive curves

This shows only the mean predictions. For the Bayesian cases, we can instead loop over the posteriors of the parameters (or samples from the GP posterior) to get the distribution of such curves in each case.

We will need a vector version (pmin in place of min) of this function that can operate on the posteriors, others are vectorized already.

ricker_f <- function(x,h,p){
  sapply(x, function(x){ 
    x <- pmax(0, x-h) 
    pmax(0, x * exp(p[2] * (1 - x / p[1] )) )
  })
}

Then we proceed as before, now looping over 100 random samples from the posterior for each Bayesian estimate. We write this as a function for easy reuse.

require(MASS)

step_ahead_posteriors <- function(x){
gp_f_at_obs <- gp_predict(gp, x, burnin=1e4, thin=300)
df_post <- melt(lapply(sample(100), 
  function(i){
    data.frame(time = 1:length(x), stock = x, 
                GP = mvrnorm(1, gp_f_at_obs$Ef_posterior[,i], gp_f_at_obs$Cf_posterior[[i]]),
                True = step_ahead(x,f,p),  
                MLE = step_ahead(x,f,est$p), 
                Parametric.Bayes = step_ahead(x, allen_f, pardist[i,]), 
                Ricker = step_ahead(x, ricker_f, ricker_pardist[i,]), 
                Myers = step_ahead(x, myers_f, myers_pardist[i,]))
  }), id=c("time", "stock"))

}


df_post <- step_ahead_posteriors(x)
ggplot(df_post) + geom_point(aes(time, stock)) + 
  geom_line(aes(time, value, col=variable, group=interaction(L1,variable)), alpha=.1) + 
  facet_wrap(~variable) + 
  scale_colour_manual(values=colorkey, guide = guide_legend(override.aes = list(alpha = 1)))

alternately, try the plot without facets

ggplot(df_post) + geom_point(aes(time, stock)) + 
  geom_line(aes(time, value, col=variable, group=interaction(L1,variable)), alpha=.1) + 
  scale_colour_manual(values=colorkey, guide = guide_legend(override.aes = list(alpha = 1)))

Performance on data outside of observations

Of course it is hardly suprising that all models do reasonably well on the data on which they were trained. A crux of the problem is the model performance on data outside the observed range. (Though we might also wish to repeat the above plot on data in the observed range but a different sequence from the observed data).

First we generate some data from the underlying model coming from below the tipping point:

Tobs <- 8
y <- numeric(Tobs)
y[1] = 4.5
for(t in 1:(Tobs-1))
    y[t+1] = z_g() * f(y[t], h=0, p=p)

Proceed as before on this data:

crash_data <- step_ahead_posteriors(y)

ggplot(crash_data) + geom_point(aes(time, stock)) + 
  geom_line(aes(time, value, col=variable, group=interaction(L1,variable)), alpha=.1) + 
  facet_wrap(~variable) + 
  scale_colour_manual(values=colorkey, guide = guide_legend(override.aes = list(alpha = 1)))

Note that the GP is doing remarkably well even outside the observed range, though with greater uncertainty as well. Other models tend to be overly optimistic, often predicting an increase instead of a decline, hence the many trajectories that continually float above the data.

Forecast Distributions

Another way to visualize this is to look directly at the distribution predicted under each model, one step and several steps into the future (say, at a fixed harvest level). Here we have a simple function that will look one step ahead of the given x (given as index i), then Tobs steps ahead, then 2*Tobs steps ahead, each at a fixed harvest. This lets us compare both the expected outcome over short and long term under a given harvest policy, as well as seeing how the distribution of possible outcomes evolves.

library(expm)
get_forecasts <- function(i, Tobs, h_i){

df <- data.frame(
  x = x_grid,  
  GP = matrices_gp[[h_i]][i,], 
  True = matrices_true[[h_i]][i,], 
  MLE = matrices_estimated[[h_i]][i,], 
  Parametric.Bayes = matrices_par_bayes[[h_i]][i,], 
  Ricker = matrices_alt[[h_i]][i,],
  Myers = matrices_myers[[h_i]][i,])

df2 <- data.frame(
  x = x_grid,
  GP = (matrices_gp[[h_i]] %^% Tobs)[i,], 
  True = (matrices_true[[h_i]] %^% Tobs)[i,], 
  MLE = (matrices_estimated[[h_i]] %^% Tobs)[i,],
  Parametric.Bayes = (matrices_par_bayes[[h_i]] %^% Tobs)[i,], 
  Ricker = (matrices_alt[[h_i]] %^% Tobs)[i,], 
  Myers = (matrices_myers[[h_i]] %^% Tobs)[i,])

T2 <- 2 * Tobs

df4 <- data.frame(
  x = x_grid,
  GP = (matrices_gp[[h_i]] %^% T2)[i,], 
  True = (matrices_true[[h_i]] %^% T2)[i,], 
  MLE = (matrices_estimated[[h_i]] %^% T2)[i,],
  Parametric.Bayes = (matrices_par_bayes[[h_i]] %^% T2)[i,], 
  Ricker = (matrices_alt[[h_i]] %^% T2)[i,], 
  Myers = (matrices_myers[[h_i]] %^% T2)[i,])

forecasts <- melt(list(T_start = df, T_mid = df2, T_end = df4), id="x")

}

i = 15

This takes i, an index to x_grid value (e.g. for i=15 corresponds to a starting postion x = 3.4286)

forecasts <- get_forecasts(i = 15, Tobs = 5, h_i = 1)

ggplot(forecasts) +
  geom_line(aes(x, value, group=interaction(variable, L1), col=variable, lty=L1)) +
  facet_wrap(~ variable, scale="free_y", ncol=2) + 
  scale_colour_manual(values=colorkey)

plot of chunk unnamed-chunk-10

We can compare to a better starting stock,

i<-30

where i=30 corresponds to a starting postion x = 7.102

forecasts <- get_forecasts(i = i, Tobs = 5, h_i = 1)

ggplot(forecasts) +
  geom_line(aes(x, value, group=interaction(variable, L1), col=variable, lty=L1)) +
  facet_wrap(~ variable, scale="free_y", ncol=2) + 
  scale_colour_manual(values=colorkey)

Note the greater uncertainty in both the positive and negative outcomes under the parametric Bayesian models (of correct and incorrect structure).

Misc reading

Apparently I am catching up on my C. Titus Brown reading… Achiving comments for my records & to keep them in my local search index.

Titus Brown on if he really practices Open Science?

Methinks:

I think you undersell your Github practices regarding Open Science. While I agree entirely that anyone reading through your source-code to scoop you is incredibly unlikely; I imagine many researchers would still fear the practice of using a public, Google-indexed repository, particularly if they also practice reasonable literate programming documentation, provide test cases, and use issue tracking that could allow some close rival to scoop them.

It sounds like you are talking more about not marketing your work before it is “finished”, rather not being open. (When you say “to really open up about what we’re doing” I assume you mean advertising the results, rather than something like simply putting your notebook or publication drafts on github). Though “Open Science” is used in in both contexts, marketing your pre-publication science takes additional time that exposing your workflow on Github does not. The latter practice provides transparency, provenance machine-discoverability, (and cool graphs of research contributions like https://github.com/ctb/khmer/c… etc. Perhaps you are too hard on yourself here.

Meanwhile, I think suggesting that being open prematurely would “waste other people’s time, energy, and attention” is misleading and damaging here. Sure, I understand you mean that marketing unfinished results (blog, tweet, present at conferences) would do these things, but when the same reason is frequently given for not sharing data, code, etc this becomes very damaging. Some people may find it useful to blog/tweet/present unfinished work (with appropriate disclaimer) to get feedback, build audience, or for all the same reasons one would do so with published work, but to me that is really just a question of timing & marketing, not a question of “openness”.

On Research Software Reuse

On the Costs of Open Science, which I refute:

Great post and important question. I certainly agree that it is all about incentives, but I think an important missing part of that discussion is in the timescales. Certainly there is a cost to not writing up the low-lying fruit following up on your work, but surely such exercises would take non-trivial time away from doing whatever (presumably more interesting) stuff you did instead. Perhaps the it is the benefits, not the costs, that are hidden:

It sounds like you may have forgone a short-term cost in not publishing these easy follow-ups while gaining a benefit both of time to work on other interesting things and of getting other researchers invested in your work, who might not have taken it up at all if there was no low-hanging fruit to entice them in. Once invested in it, no doubt they can continue to be a multiplier of impact. Meanwhile you break new ground rather than appearing to continue to wring every ounce from one good idea, right? (If I read this correctly, both you and George Gey appear to regard these other publications as not particularly exciting work; the regret comes only because they are intrinsically valued publications)

It sounds like the difficulty with these benefits is that they pay off only on a longer timescale than the gristmill publication strategy. On the longest timescales, e.g. career lifetime, it seems clear that at least in a statistical sense, researchers who keep breaking new ground while allowing others to pick the low-hanging follow-up work will be much more likely to end up as the most distinguished researchers, etc. When the relevant timescale is a tenure clock rather than a career, the perhaps the calculus is different?

Without open science, moving on to other exciting stuff is far less effective, since it leaves both unfinished work and less impact. Perhaps this a somewhat idealized view, and perhaps the timescales for tenure etc work against this strategy. You and others could no doubt better speak to whether the candidate who says “look at all the papers I wrote about X” or the one who says “look at all the papers others have built upon my method X while I break ground in hard problem Y” has the stronger case.

Partial implementation of R to Latex equation converter from @hadleywickam.
Ah ha! Documenting equations in ROxygen. Guess one day I (or devtools) will move to Roxygen3.

Notes

2013-05-13T00:00:00-07:00

Prosecutor’s Fallacy comment

Fallacy comment revisions
Comment reply letter (draft)
ROC curves instead? (examples)
Add example of system with an alternative stable state?

Policy costs (pdg-control)

touched up tex document following Paul’s edits (mostly reflects comments and decisions from Meeting 3). Reminder of why collaborating on TeX documents can be annoying even when co-authors are tex-literate. Attempted quick conversion to markdown but mapping is troublesome.

Misc

Some software providing rather impressive/high fidelity pdf to html conversion… Not just rendering as images – the text is searchable, though in the html source it just looks like a bunch of binary data URIs.
Scathingly honest review of the classical ecological statistics text by Sokal and Rohlf (pdf)
PNAS hates mathematics Not really, but a depressing example of poor reviewing.

Prep for ERE talk

2013-05-09T00:00:00-07:00

Talk prep

Finished preparing slides for Environmental Resources Economics talk

Toying around with animations for final plots. Building up plot by subsetting progressively more of the data each time is a bit of a nuciance (both in coding and efficiency). Can convert replicates to characters and assign as a data.table key for fast join subsetting, but straight-forward subsetting seems best (e.g. once we want reps 1:5 from both “True” and “Ricker” models, dt[J(c("True", "Ricker"), as.character(1:5)] doesn’t quite do this. (Can you guess what it gives? Actually alternates “True”+rep1, “Ricker”+rep2, “True”+rep3, …)

require(animation)
ani.options(loop=TRUE)
saveMovie({    
    for (i in seq(1, OptTime, by=2)) {
      print(
        ggplot(subset(dt, method %in% c("True", "Ricker") & reps < 5 & time <= i), 
               aes(time, fishstock, group=interaction(reps,method), color = method), alpha=.9) + 
          geom_line() + xlim(0, OptTime)
        )
    }
}, interval = 0.1, movie.name = "wrong-model.gif", ani.width = 600, ani.height = 600)

Added plots of one-step-ahead predictors, e.g.

One-step ahead predictions on orginal (training) data by each model

All plots for slides also archived in flickr/ere2013
Which were quickly converted from vector pdfs into decent resolution pngs with a few commands:

for f in *.pdf ; do convert -density 300 "$f" "${f/.pdf}".png ; done
flickr_upload --tag="nonparametric-bayes ere2013 talk" *.png

Reading

(Sirota et al. 2013 ) provide a cute example of a system that can be manipulated through a tipping point of eutrophication in the tiny pools forming inside pitcher plants by adding ground-up dead insects. A nice natural system that provides a more realistic setting than lab manipulations of single species micro-organisms while also being more accessible to replication that the whole-lake experiments in Wisconsin. I do note they critique retrospective analyses on the basis of length, but don’t mention the prosecutor’s fallacy. Five stars for archiving the raw data very nicely (see below) along with the mathematica notebook file used for the analysis. Also notable that the first author is an undergrad at North Dakota State.
Wow, Harvard Forest provides a data repository with EML files for each data entry (example). EML file serves more as a metadata description, raw data provided as an 8.8 MB .csv file in tidy (long) format. Very nice.
From the Whitehouse, executive order: The default state of new and modernized Government information resources shall be open and machine readable I can’t really express just how incredible that is. May it impact government funded science appropriately. Whitehouse Chief technology officer and chief information officer explain the policy in one minute

References

J. Sirota, B. Baiser, N. J. Gotelli, A. M. Ellison, (2013) Organic-Matter Loading Determines Regime Shifts And Alternative States in an Aquatic Ecosystem. Proceedings of The National Academy of Sciences 110 7742-7747 10.1073/pnas.1221037110

Notes

2013-05-07T00:00:00-07:00

Monday

MacKenzie meeting
paperwork to Marc
paperwork to Karthik
Writing

Tuesday

Log

Weak thresholds example – no cost but no profit if stocks are reduced below a certain threshold 09:31 pm 2013/05/07
example using short (20 pts) observation period 02:00 pm 2013/05/07
subset plotting of simulations (for slides potentially) 02:00 pm 2013/05/07
script for the parallel analysis based on policy thresholds 01:59 pm 2013/05/07
short vs long observation time runs 12:54 pm 2013/05/07
online example of replicates comparing all models 12:28 pm 2013/05/07

Misc

latex templates for UCSC AMS dept Gosh sometimes it’s nice to be officially in a department that understands tex.
Best teacher’s know student’s misconceptions Hmm, our misconceptions are probably a bigger barrier than lack of knowledge for most scientists too…
Ohh, Jekyll 1.0 released, changelog
#openoxford’s debate Elseiver rep says that if they disclose prices, prices would be driven so low that they’d go out of buisness. Yikes, what would the economists say!
Finally, you can search within a git repositories (without cloning first and using grep obviously. Before one was had global search options only, albeit filtered by language, etc).
Callaway (2013) Ironic that they quote @mfenner saying that peerJ preprints are too like Nature Preceedings while also pointing out the role of archiving things like slides and posters (as these were categories of objects in Preceedings, but not in arXiv or PeerJ). Of course I agree with Martin about diverse outputs, but there appears to be something about the concept of preprint server that carries more meaning for people. Clearly figshare is a viable preprint server that is both the most discipline agonstic (arXiv isn’t interesting in expanding its scope, and peerJ’s stated scope does not cover earth science / climate science / physical oceanography side of things) and content-type agnostic (paper, poster, data, code, or anything else), and has a rich API that is absent or minimal in the other platforms.

ESA schedule

Sun Aug 4 noon-5pm Conducting Open Science Using R and DataOne
Tues Aug 6 8-10 pm Open Science and Ecology

References

Ewen Callaway, (2013) Biomedical Journal And Publisher Hope to Bring Preprints to Life. Nature Medicine 19 512-512 10.1038/nm0513-512

Reading and potential examples for talk to ERE next week

2013-05-03T00:00:00-07:00

Parametric Bayesian comparisons

Added parameteric Bayesian version of Ricker to the set of comparisons, see allen.Rmd
Adjusted thinning of posterior sample before determining optimum of GP

Reading / Examples

Need a nice set of basic fisheries examples involving potential tipping points for ERE talk. hmm.

As mentioned in Worm et al. (2009) .
Algal-reef example from Aronson & Precht (2000)
Fish collapse following reef distruction Paddack et al. (2009)
Alt stable states in marine systems Petraitis & Dudgeon (2004) , Osman et al. (2010)
Cod examples? Forage fish? Peruvian Anchovy. Overfishing vs climate variability.
And more examples from marine systems in Oguz & Velikova (2010) , Lindegren et al. (2010) , deYoung et al. (2008)
Beyond tipping points: avoiding policy-based thresholds. Pikitch (2012)
http://www.ecotippingpoints.org/our-stories/indepth/usa-hawaii-hanauma-environmental-management.html ?
http://www.ecotippingpoints.org/our-stories/indepth/philippines-apo-marine-sanctuary-coral-reef-fishery.html

References

Richard B. Aronson, William F. Precht, (2000) Herbivory And Algal Dynamics on The Coral Reef at Discovery Bay, Jamaica. Limnology And Oceanography 45 251-255 10.4319/lo.2000.45.1.0251
M Lindegren, R Diekmann, C Möllmann, (2010) Regime Shifts, Resilience And Recovery of A Cod Stock. Marine Ecology Progress Series 402 239-253 10.3354/meps08454
T Oguz, V Velikova, (2010) Abrupt Transition of The Northwestern Black Sea Shelf Ecosystem From A Eutrophic to an Alternative Pristine State. Marine Ecology Progress Series 405 231-242 10.3354/meps08538
RW Osman, P Munguia, RN Zajac, (2010) Ecological Thresholds in Marine Communities: Theory, Experiments And Management. Marine Ecology Progress Series 413 185-187 10.3354/meps08765
Michelle J. Paddack, John D. Reynolds, Consuelo Aguilar, Richard S. Appeldoorn, Jim Beets, Edward W. Burkett, Paul M. Chittaro, Kristen Clarke, Rene Esteves, Ana C. Fonseca, Graham E. Forrester, Alan M. Friedlander, Jorge García-Sais, Gaspar González-Sansón, Lance K.B. Jordan, David B. McClellan, Margaret W. Miller, Philip P. Molloy, Peter J. Mumby, Ivan Nagelkerken, Michael Nemeth, Raúl Navas-Camacho, Joanna Pitt, Nicholas V.C. Polunin, Maria Catalina Reyes-Nivia, D. Ross Robertson, Alberto Rodríguez-Ramírez, Eva Salas, Struan R. Smith, Richard E. Spieler, Mark A. Steele, Ivor D. Williams, Clare L. Wormald, Andrew R. Watkinson, Isabelle M. Côté, (2009) Recent Region-Wide Declines in Caribbean Reef Fish Abundance. Current Biology 19 590-595 10.1016/j.cub.2009.02.041
Peter S. Petraitis, Steve R. Dudgeon, (2004) Detection of Alternative Stable States in Marine Communities. Journal of Experimental Marine Biology And Ecology 300 343-371 10.1016/j.jembe.2003.12.026
E. K. Pikitch, (2012) The Risks of Overfishing. Science 338 474-475 10.1126/science.1229965
B. Worm, R. Hilborn, J. K. Baum, T. A. Branch, J. S. Collie, C. Costello, M. J. Fogarty, E. A. Fulton, J. A. Hutchings, S. Jennings, O. P. Jensen, H. K. Lotze, P. M. Mace, T. R. McClanahan, C. Minto, S. R. Palumbi, A. M. Parma, D. Ricard, A. A. Rosenberg, R. Watson, D. Zeller, (2009) Rebuilding Global Fisheries. Science 325 578-585 10.1126/science.1173146
Brad deYoung, Manuel Barange, Gregory Beaugrand, Roger Harris, R. Ian Perry, Marten Scheffer, Francisco Werner, (2008) Regime Shifts in Marine Ecosystems: Detection, Prediction And Management. Trends in Ecology & Evolution 23 402-409 10.1016/j.tree.2008.03.008

Notebook features: SHA Hashes

2013-05-03T00:00:00-07:00

Note: this entry is part of a series of posts which explain some of the techinical features of my lab notebook.

I version manage all changes to my entry using git. Each page is linked to its source history on Github, which will display a list of all previous edits to the post with an easy-to-read commit log and highlighted diffs. A version history is often considered an essential part of an open lab notebook, where changes to the notebook are documented and preserved. While wikis, google docs, dropbox, wordpress plugins, or just regular backups can provide version history of pages, none come close to comparison with a full version management system such as git. This is because git’s underlying architecture is based on
The magic of cryptographic SHA hashes.

Hashes provide an immutable and verifiable record of any all changes I make. Because the hash is generated by the cryptographic SHA1 algorithm from the contents of the site, it is impossible to make changes without causing the hash to update. By referencing the content’s hash value we can be sure to link to a constant version of the entry. These can be verified by re-generating the hash (requiring the previous state of the repository in this case, see note below). Unlike publication timestamps, versions, or DOIs, this provides a way not only to reference particular versions of a file, but a cryptographically secure way to verify that the version is what it claims to be. Tobias Kuhn has observed that this is a valuable feature we should want to see for all scientific publishing. Each of my posts now displays its SHA hash on the sidebar along with other metadata. While the history button already provides a convient way to browse all previous versions of a post, I chose to display the SHA hash directly so that the hash value would be part of the document metadata, while also highlighting this feature.

In addition to the GitHub repository for my lab notebook, My research code, analyses, and manuscripts are collected into Github repositories by project. This allows my analysis and paper writing to benefit from this same immutable and verifiable record. Because GitHub uses the SHA hashes in its link structure, this also provides a convenient way to link to a particular version of code in a given entry. This way, I can be sure the contents of the file displayed at that link never change, even as I continue to update that file. Even if the file or containing directory is later deleted or moved, the link will still resolve. Only if the entire project repo were deleted or if Github itself dissolved would the link be lost. Even then, using the SHA hash given in the link we could determine the contents of the file from some other copy of the repository (such as a local or figshare archive).

Tobias is actually working on his own SHA hash approach which is somewhat superior to the simpler method of using git. The Github hash corresponds to the state of the entire repository/notebook at the time of the commit, rather than the contents of an individual file. Consequently, one would need a snapshot of the entire repository, available on Github, to perform the verification. Tobias is looking into generating hashes based on the contents of the file directly – so far, only RDF data – that could proivde a unique and verifiable reference for any scholarly data or publication.

Version managing the notebook and code has many more practical day-to-day benefits, such as recovering from a mistaken deleted or corrupted file, merging changes made on different machines or by collaborators, or creating branches to test new features without disrupting current version, and comparing differences as a file evolves.

Scholarly Infrastructure Thoughts

2013-05-02T00:00:00-07:00

Sustainable Research

Happened across a provocative example of why we need a software ecosystem, though it was certainly not intended to be one, which led me to ask myself:

How complex does an algorithm have to be before a talented researcher with expertise in both the relevant mathematics and computer science will make a significant mistake in their first release of the software?

As a corollary we might also ask, “How much less care do we put into research code not destined for release?”

This changelog clearly reflects these difficulties face well-established researchers with long publication records on these very methods. If such individuals can make mistakes in packages, are we supposed to trust the myriad personal implementations of this and more complex algorithms that bulwark our literature today?

Academic knowledge is currently built in the mortar and bricks of publication and citation. Publications advance new claims built up on existing claims (and very occasionally replacing them) through citations. It an approach that does not scale well on many fronts. To verify information we must trace the citation chain, which grows far to quickly to for human processing and is not usually amenable to computer processing. Yet here it is the statistical scaling that concerns me more – a paper advances that a claim is true with a certain probability, given that the methods are implemented without error. The more publications we string together, the higher the probability that we observe false positives, but also that we observe implementation errors. For much of research today, we need not construct the scientific argument in this manner.

Thanks to computational advances of the last several decades, public repositories of the data and the methods (such as can be implemented as software anyway), we can build on existing work by direct analysis of the data and methods, rather than treating the conclusions as given. Evidence would no longer come primarily in the highly circumstantial manner of citations to previous claims, but to direct analysis of data. Using a common pool of data and methods would align incentives better to maintain and improve upon this infrastructure (the way major companies contribute to the underlying ‘plumbing’ provided by shared open source infrastructure), while there is no incentive for the literature to work in such a way.

Cameron Neylon, and link in my comment.

Musings On Conservation Literature

2013-05-02T00:00:00-07:00

Just because I’m a theorist deeply entrenched in methodological concerns about uncertainty and decision making doesn’t mean I don’t think about practical conservation from time to time. Some musings from my comment here, copied over for my own reference.

Though I would like to believe the gap stems from the problems you discuss, I think that differing objectives between research and application may play a much larger role. I suspect that scientific papers that are most useful and influential for conservation practitioners and policymakers are those which confirm what they already believe, or whatever the interests opposed to them least want to hear. Let’s call these “Cassandra” papers, since in this context they usually forecast disaster. For the practitioner it may matters little whether the math is simple or complex, clearly explained or impenetrable, or even right or not so right. Worm et al 2006 paper which the media quickly decided predicted the end of global fisheries within 50 years is perhaps a good example.

Okay, so beyond bolstering arguments already being made by those who propose, implement or legislate conservation against their opposition, there are certainly unknowns that they might turn to research to answer. Resource allocation might be an example of this; e.g. do we prioritize purchasing pristine areas that are not likely to be threatened or less pristine areas in more immediate danger (a la Pfaff). Let’s call these “rule of thumb” papers, where the conclusion is an easily applied guide-line. It seems doubtful that the practitioners would be inhibited by their access and understanding of the math in this case, since they want the research to provide an answer they can trust, and not worry so much about what math justifies it. They are more likely to use proxies of quality (journal, researcher, affiliation, popularity of the method), then working through the assumptions to see if they like them; no?

So there is a third case in which the conclusion is of the sort “apply my method and it will tell you what to do”, as opposed to “here’s what to do”. I think only this case falls at risk to the mathematics being a barrier, though when accompanied by user-friendly software tools perhaps that can be dismissed as well. These “methods” papers are probably the favorite option of many researchers, as they seem the most rigorous, accurate way, reflecting the details of the problem at hand. Scientists are probably least fond of the first example, where even the paper’s authors may feel the conclusions are being overstated, while others feel such work is wrong and counterproductive. I’d guess many researchers are lukewarm towards the middle case, as better than a coin flip. I imagine from the conservation practitioner’s ranking is reversed. To what extent would you agree with this classification? If so, how is the conservation literature distributed across these categories, and how might we want it to be distributed? Do we indeed have the greatest impact writing Cassandra papers rather than writing nice clear methods, and if so, what are the implications?

Integrating Over Posterior Gp

2013-05-01T00:00:00-07:00

Added integration over the posterior distribution of the GP
Added example of parametric Bayesian inference based on Ricker

Log

varying parameters 06:07 pm 2013/05/01
example based on gp means 03:44 pm 2013/05/01
run means example as separate script 02:55 pm 2013/05/01
compare against just using means 02:52 pm 2013/05/01
Merge branch ‘master’ of github.com:cboettig/nonparametric-bayes 02:26 pm 2013/05/01
Merge branch ‘master’ of github.com:cboettig/nonparametric-bayes 02:23 pm 2013/05/01
add gp posteriors to package 02:16 pm 2013/05/01
staging run for integrating over GP posterior (large memory reqs) 02:13 pm 2013/05/01
updated example of gp basics includes mcmc and shows influence of the prior 01:15 pm 2013/05/01
example testing new routine 01:14 pm 2013/05/01
tidying repo 01:07 pm 2013/05/01
merge analyses back into allen.Rmd 12:48 pm 2013/05/01
adapted to the custom version 12:37 pm 2013/05/01
Test examples demonstrating the custom mcmc GP 12:36 pm 2013/05/01
package updates with the custom gp method 12:35 pm 2013/05/01
Example using custom GP routine. In this case, both MLE methods fail, while the GP does fine and the parametric Bayesian method performs best. 12:34 pm 2013/05/01

Bayesian Gp Regression By Metropolis Hastings

2013-05-01T00:00:00-07:00

(Development version for reference; code functions now in R/gp_mcmc.R)

Function definitions

library(mcmc)
#' Basic regression in Gaussian processes  
#' 
#' 
#' @param x Observed x values, (vector or matrix with columns for each dimension of data)
#' @param y Vector of observed y values in the training data
#' @param init_pars the initial guesses for lengthscale l and process noise sigma_n
#' @param n iterations of the metropolis algorithm
#' @details Currently assumes the covariance function.  By default we will use 
#' the squared exponential (also called radial basis or Gaussian, 
#' though it is not this that gives Gaussian process it's name; 
#' any covariance function would do) as the covariance function, 
#' $\operatorname{cov}(X_i, X_j) = \exp\left(-\frac{(X_i - X_j)^2}{2 \ell ^ 2}\right)$
#' @return the MCMC output, as constructed by \link{metrop} from the mcmc package.
#' @export
#' @import mcmc  
gp_mcmc <- function(x, y, init_pars = c(l=1, sigma.n=1), n = 1e4, d.p = c(5,5), s2.p = c(5,5)){

  lpriors <- function(pars){
    prior <- unname(
      dgamma(exp(pars[1]), d.p[1], scale = d.p[2]) *
      dgamma(exp(pars[2]), s2.p[1], s2.p[2]) 
    )
    log(prior)
  }
  posterior <- function(pars, x, y){ 
    l <- exp(pars[1])
    sigma.n <- exp(pars[2])
    
    cov <- function(X, Y) outer(X, Y, SE, l)
    I <- diag(1, length(x))
    K <- cov(x, x) 
    
    loglik <- - 0.5 * t(y) %*% solve(K + sigma.n^2 * I) %*% y -
      log(det(K + sigma.n^2*I)) -
      length(y) * log(2 * pi) / 2
    
    loglik + lpriors(pars)
  }
  
  out <- metrop(posterior, log(init_pars), n, x = obs$x, y = obs$y)  
  out$d.p <- d.p
  out$s2.p <- s2.p
  
  out
}

#' predict the expected values and posterior distributions of the Gaussian Process
#' 
#' @param x_predict the values at which we desire predictions
#' @param covs if TRUE, return covariances instead of variances (set high thinning as this is memory intensive)
#' @import reshape2
#' @export
gp_predict <- function(gp, x_predict, burnin=0, thin=1, covs=FALSE){

  postdist <- cbind(index=1:gp$nbatch, as.data.frame(exp(gp$batch)))
  s <- seq(burnin+1, gp$nbatch, by=thin)
  postdist <- postdist[s,]
  names(postdist) <- c("index", names(gp$initial))
  
  indices <- 1:dim(postdist)[1]
  
  out <- lapply(indices, function(i){
    l <- postdist[i, "l"]
    sigma.n <- postdist[i, "sigma.n"]
    
    cov <- function(X, Y) outer(X, Y, SE, l)
    cov_xx_inv <- solve(cov(obs$x, obs$x) + sigma.n^2 * diag(1, length(obs$x)))
    Ef <- cov(x_predict, obs$x) %*% cov_xx_inv %*% obs$y
    Cf <- cov(x_predict, x_predict) - cov(x_predict, obs$x)  %*% cov_xx_inv %*% cov(obs$x, x_predict)
    list(Ef = Ef, Cf = Cf, Vf = diag(Cf))
  })
  
  Ef_posterior <- sapply(out, `[[`, "Ef")
  Cf_posterior <- sapply(out, `[[`, "Cf")
  Vf_posterior <- sapply(out, `[[`, "Vf")
  
  
  E_Ef <- rowMeans(Ef_posterior)
  E_Cf <- matrix( apply(Cf_posterior, 1, sum) / dim(Cf_posterior)[2], ncol = sqrt(dim(Cf_posterior)[1]) )
  E_Vf <- diag(E_Cf) # same as rowMeans(Vf_posterior)
  
  # list format better for return
  Cf_posterior <- lapply(out, `[[`, "Cf")
  
  list(Ef_posterior = Ef_posterior, 
       Vf_posterior = Vf_posterior,
       Cf_posterior = Cf_posterior,
       E_Ef = E_Ef, E_Cf = E_Cf, E_Vf = E_Vf)
  
}

library(reshape2)
library(ggplot2)
library(plyr)
#' Summary plots showing the trace and posteriors for the gp_mcmc estimates
#' 
#' @param gp a fit of the gaussian process from gp_mcmc
#' @param burnin length of sequence to discard as a transient
#' @param thin frequency of sub-sampling (make posterior distribution smaller if necessary)
#' @return two ggplot2 objects, one plotting the trace and one 
#' plotting the posteriors in black with priors overlaid in red.  
#' @import reshape2
#' @import ggplot2
#' @import plyr
#' @export
summary_gp_mcmc <- function(gp, burnin=0, thin=1){

  postdist <- cbind(index=1:gp$nbatch, as.data.frame(exp(gp$batch)))
  s <- seq(burnin+1, gp$nbatch, by=thin)
  postdist <- postdist[s,]
  names(postdist) <- c("index", names(gp$initial))
  
  # TRACES
  df <- melt(postdist, id="index")
  traces_plot <- 
    ggplot(df) + geom_line(aes(index, value)) + 
    facet_wrap(~ variable, scale="free", ncol=1)
  
  s2.p <- gp$s2.p
  d.p <- gp$d.p
  s2_prior <- function(x) dgamma(x, s2.p[1], s2.p[2])
  d_prior <- function(x) dgamma(x, d.p[1], scale = d.p[2])
  prior_fns <- list(l = d_prior, sigma.n = s2_prior)
  
  prior_curves <- ddply(df, "variable", function(dd){
    grid <- seq(min(dd$value), max(dd$value), length = 100)
    data.frame(value = grid, density = prior_fns[[dd$variable[1]]](grid))
  })
  
  # Posteriors (easier to read than histograms)
  posteriors_plot <- ggplot(df, aes(value)) + 
    stat_density(geom="path", position="identity", alpha=0.7) +
    geom_line(data=prior_curves, aes(x=value, y=density), col="red") + 
    facet_wrap(~ variable, scale="free", ncol=2)
  
  print(traces_plot)
  print(posteriors_plot)
  
  out <- list(traces_plot = traces_plot, posteriors_plot = posteriors_plot)
  invisible(out)
}

# Helper function: The default covariance function
SE <- function(Xi,Xj, l) exp(-0.5 * (Xi - Xj) ^ 2 / l ^ 2)

Example

x_predict <- seq(-5,5,len=50)
obs <- data.frame(x = c(-4, -3, -1,  0,  2),
                  y = c(-2,  0,  1,  2, -1))

gp <- gp_mcmc(obs$x, obs$y, d.p = c(2,.5), s2.p = c(2,.5))

summary_gp_mcmc(gp)

gp_dat <- gp_predict(gp, x_predict)

tgp_dat <- 
    data.frame(  x = x_predict, 
                 y = gp_dat$E_Ef, 
                 ymin = gp_dat$E_Ef - 2 * sqrt(gp_dat$E_Vf), 
                 ymax = gp_dat$E_Ef + 2 * sqrt(gp_dat$E_Vf) )

ggplot(tgp_dat) + geom_ribbon(aes(x,y,ymin=ymin,ymax=ymax), fill="gray80") +
    geom_line(aes(x, y), lwd=2, alpha=0.8) + 
    geom_point(data=obs, aes(x,y), alpha=0.8) + 
    xlab(expression(X[t])) + ylab(expression(X[t+1]))

plot of chunk unnamed-chunk-8

Difficulties with tgp; taking custom mcmc approach for GPs

2013-04-30T00:00:00-07:00

Running into some unpredictable and not consistently replicable memory errors in the tgp function calls. Also rather troublesome that return object provides only posterior means for the process mean and covariance matrix.

Generic GP methods

Implemented MCMC sampler. Excerpt below based on my gaussian-process-basics.md script, as updated to include this case.

Define some example priors

lpriors <- function(pars){
   d.p <- c(5, 5)
  s2.p <- c(5, 5)  
   
  prior <- unname(
    dgamma(exp(pars[1]), d.p[1], scale = d.p[2]) *
    dgamma(exp(pars[2]), s2.p[1], s2.p[2]) 
  )
  
  log(prior)
}

Define the posterior function for our metropolis sampler

posterior <- function(pars, x, y){
  
  l <- exp(pars[1])
  sigma.n <- exp(pars[2])
  
  cov <- function(X, Y) outer(X, Y, SE, l)
  I <- diag(1, length(x))
  K <- cov(x, x) 
  
  loglik <- - 0.5 * t(y) %*% solve(K + sigma.n^2 * I) %*% y -
    log(det(K + sigma.n^2*I)) -
    length(y) * log(2 * pi) / 2

  loglik + lpriors(pars)
}

I do not think we have good conjugate priors we can use in this situation to accomplish Gibbs sampling. Instead we simply MCMC by the Metropolis-Hastings alogrithm,

require(mcmc)
n <- 1e4
out <- metrop(posterior, log(pars), n, x = obs$x, y = obs$y)
out$accept

[1] 0.1509

Very good, we want an acceptance rate around 20%, if the literature can be trusted on this account. We then plot transformed and untransformed traces.

postdist <- cbind(index=1:n, as.data.frame(exp(out$batch)))
names(postdist) <- c("index", names(pars))
df <- melt(postdist, id="index")
# TRACES
ggplot(df) + geom_line(aes(index, value)) + facet_wrap(~ variable, scale="free", ncol=1)


ggplot(df) + geom_line(aes(index, log(value))) + facet_wrap(~ variable, scale="free", ncol=1)

Generate a plot to compare priors and posteriors

d.p <- c(5, 5)
s2.p <- c(5, 5)  
   
s2_prior <- function(x) dgamma(x, s2.p[1], s2.p[2])
d_prior <- function(x) dgamma(x, d.p[1], scale = d.p[2])
prior_fns <- list(l = d_prior, sigma.n = s2_prior)


require(plyr)
prior_curves <- ddply(df, "variable", function(dd){
    grid <- seq(min(dd$value), max(dd$value), length = 100)
    data.frame(value = grid, density = prior_fns[[dd$variable[1]]](grid))
})

# Posteriors (easier to read than histograms)
ggplot(df, aes(value)) + 
  stat_density(geom="path", position="identity", alpha=0.7) +
  geom_line(data=prior_curves, aes(x=value, y=density), col="red") + 
  facet_wrap(~ variable, scale="free", ncol=2)

Note that posteriors closely match our priors, as we have only 4 data points in this toy example and our priors are reasonably informative.

Steve chat

integrate over posteriors in GP case
Ignore measurement uncertainty for the moment
General status updates, etc.

Marc meeting

Ricker case, but with constraint (e.g. maintain above n%)
Consider Ricker model with external management constraint $ (Y / Y_opt) + (1 - ) Pr( )$
Consider time horizon of learning vs implementing; see diagram

Simone talk

General interests: adaptive potential of small pops, demography of small pops, extinction processes in communities, evol life histories & enviroment, ML for species distribution models.
Marble trout system under the influence of regular catastrophic floods

Misc

6 hours on trains, 2 hours on buses, 1.25 hours on bicycle, 6 hours in lab.
Interesting reading in The Secret History of OpenStack, the Free Cloud Software That’s Changing Everything. Sounds just like the unlikely collaboration we need to emerge for a more robust and sustainable underlying scientific infrastructure.

Finishing Comparison Parametric And Nonparametric

2013-04-29T00:00:00-07:00

Implemented in BUGS/allen.md

[x] Inverse gamma priors on variances 36888da
[x] Uniform prior on standard deviations 75df941
[x] Mean plot for parametric fit 75df941
[x] Convergence diagnostics for both parametric and nonparametric MCMC, using similar visual layout 75df941
Oh, Jeromy Anglim has a rather nice collection of jags links

On Variance Priors for the Parametric MCMC

Gelman recommends uniform priors on standard deviations for the noise terms: in the .bugs file we have

stdQ ~ dunif(0,100)
stdR ~ dunif(0,100)
# as "precision" tau instead of stdev sigma
iQ <- 1/(stdQ*stdQ);
iR <- 1/(stdR*stdR);

Where these enter the model as

for(t in 1:(N-1)){
    mu[t] <- x[t] + exp(r0 * (1 - x[t]/K)* (x[t] - theta) )
      x[t+1] ~ dnorm(mu[t],iQ)
}


for(t in 1:(N)){
    y[t] ~ dnorm(x[t],iR)
}

Note that dnorm in JAGS notation is defined in terms of the mean and the precision (reciprocal of the variance), rather than standard deviation (e.g. in R’s dnorm function), see the JAGS manual, particularly Table 6.1 pg 29, below. Note that the standard deviation has the uniform prior, not the precision.

Also try inverse gamma priors,

iQ ~ dgamma(0.0001,0.0001)
iR ~ dgamma(0.0001,0.0001)

(Precision is Gamma distributed when the variances are inverse-gamma distributed.

Both are quite uniformative, results appear quite comparable.

Markov Chain Monte Carlo Convergence Analysis

Add for both parametric and non-parametric cases
Extra long runs before finalizing analysis
Some examples with R code

A variety of tools from coda package. Of course these methods are designed from toy problems and cannot guarentee convergence.

Traceplots, density plots
Running mean, autocorrelation plots autocorr.plot
Metropolis acceptance/rejection rate rejectionRate
Multiple chain diagonostics: Gelman-Rubin gelman.diag, gelman.plot
Geweke (compare means between different sections) geweke.diag
Raferty-Lewis raferty.diag
Heidelberg and Welch heidel.diag

A comparison of nonparametric Bayesian Gaussian process estimates to standard the parametric Bayesian approach

2013-04-27T00:00:00-07:00

Load necessary libraries,

library(pdgControl)  # custom SDP functions
library(nonparametricbayes)  # custom
library(reshape2)  # data manipulation
library(data.table)  # data manipulation
library(plyr)  # data manipulation
library(ggplot2)  # plotting
library(tgp)  # Gaussian Processes
library(MCMCpack)  # Markov Chain Monte Carlo tools
library(R2jags)  # Markov Chain Monte Carlo tools
library(emdbook)  # Markov Chain Monte Carlo tools
library(coda)  # Markov Chain Monte Carlo tools

Plotting and knitr options, (can generally be ignored)

library(knitcitations)
opts_chunk$set(tidy = FALSE, warning = FALSE, message = FALSE, cache = FALSE)
opts_knit$set(upload.fun = socialR::flickr.url)
theme_set(theme_bw(base_size = 10))
theme_update(panel.background = element_rect(fill = "transparent", 
    colour = NA), plot.background = element_rect(fill = "transparent", colour = NA))
cbPalette <- c("#000000", "#E69F00", "#56B4E9", "#009E73", "#F0E442", 
    "#0072B2", "#D55E00", "#CC79A7")

Model and parameters

Uses the model derived in Allen et al, of a Ricker-like growth curve with an allee effect, defined in the pdgControl package,

f <- RickerAllee
p <- c(2, 10, 5) 
K <- p[2]
allee <- p[3]

Various parameters defining noise dynamics, grid, and policy costs.

sigma_g <- 0.05
sigma_m <- 0.0
z_g <- function() rlnorm(1, 0, sigma_g)
z_m <- function() 1+(2*runif(1, 0,  1)-1) * sigma_m
x_grid <- seq(0, 1.5 * K, length=101)
h_grid <- x_grid
profit <- function(x,h) pmin(x, h)
delta <- 0.01
OptTime <- 20  # stationarity with unstable models is tricky thing
reward <- 0
xT <- 0
seed_i <- 111
Xo <- K # observations start from
x0 <- Xo # simulation under policy starts from
Tobs <- 35

Sample Data

obs <- sim_obs(Xo, z_g, f, p, Tobs=Tobs, nz=15, 
                harvest = sort(rep(seq(0, .5, length=7), 5)), seed = seed_i)

Maximum Likelihood

alt <- par_est(obs,  init = c(r = p[1], 
                              K = mean(obs$x[obs$x>0]), 
                              s = sigma_g))
est <- par_est_allee(obs, f, p,  
                     init = c(r = p[1] + 1, 
                              K = p[2] + 2, 
                              C = p[3] + 2, 
                              s = sigma_g))

Non-parametric Bayes

#inv gamma has mean b / (a - 1) (assuming a>1) and variance b ^ 2 / ((a - 2) * (a - 1) ^ 2) (assuming a>2)
s2.p <- c(5,5)  
tau2.p <- c(5,1)
d.p = c(10, 1/0.1, 10, 1/0.1)
nug.p = c(10, 1/0.1, 10, 1/0.1) # gamma mean
s2_prior <- function(x) dinvgamma(x, s2.p[1], s2.p[2])
tau2_prior <- function(x) dinvgamma(x, tau2.p[1], tau2.p[2])
d_prior <- function(x) dgamma(x, d.p[1], scale = d.p[2]) + dgamma(x, d.p[3], scale = d.p[4])
nug_prior <- function(x) dgamma(x, nug.p[1], scale = nug.p[2]) + dgamma(x, nug.p[3], scale = nug.p[4])
beta0_prior <- function(x, tau) dnorm(x, 0, tau)
beta = c(0)
priors <- list(s2 = s2_prior, tau2 = tau2_prior, beta0 = dnorm, nug = nug_prior, d = d_prior, ldetK = function(x) 0)

  gp <- bgp(X=obs$x, XX=x_grid, Z=obs$y, verb=0,
          meanfn="constant", bprior="b0", BTE=c(2000,20000,2),
          m0r1=FALSE, corr="exp", trace=TRUE, 
          beta = beta, s2.p = s2.p, d.p = d.p, nug.p = nug.p, tau2.p = tau2.p,
          s2.lam = "fixed", d.lam = "fixed", nug.lam = "fixed", tau2.lam = "fixed")

hyperparameters <- c("index", "s2", "tau2", "beta0", "nug", "d", "ldetK")
posteriors <- melt(gp$trace$XX[[1]][,hyperparameters], id="index")

# Traces 
ggplot(posteriors) + geom_line(aes(index, value)) + facet_wrap(~ variable, scale="free", ncol=1)

plot of chunk unnamed-chunk-3



prior_curves <- ddply(posteriors, "variable", function(dd){
    grid <- seq(min(dd$value), max(dd$value), length = 100)
    data.frame(value = grid, density = priors[[dd$variable[1]]](grid))
})

# 
ggplot(posteriors) + geom_histogram(aes(x=value, y=..density..), alpha=0.7) +
    geom_line(data=prior_curves, aes(x=value, y=density), col="red", lwd=2) +
    facet_wrap(~ variable, scale="free")

plot of chunk unnamed-chunk-3


ggplot(posteriors, aes(value)) + stat_density(geom="path", position="identity", alpha=0.7) +
  geom_line(data=prior_curves, aes(x=value, y=density), col="red") + 
  facet_wrap(~ variable, scale="free", ncol=2)

plot of chunk unnamed-chunk-3

Parametric Bayes

We initiate the MCMC chain (init_p) using the true values of the parameters p from the simulation. While impossible in real data, this gives the parametric Bayesian approach the best chance at succeeding. y is the timeseries (recall obs has the , pairs)

# a bit unfair to start with the correct values, but anyhow...
init_p = p
names(init_p) = c("r0", "K", "theta")
y <- obs$y[-1] 
N=length(y);

We’ll be using the JAGS Gibbs sampler, a recent open source BUGS implementation with an R interface that works on most platforms. We initialize the usual MCMC parameters; see ?jags for details.

jags.data <- list("N","y")
n.chains = 1
n.iter = 20000
n.burnin = floor(n.iter/2)
n.thin = max(1, floor(n.chains * (n.iter - n.burnin)/1000))

The actual model is defined in a model.file that contains an R function that is automatically translated into BUGS code by R2WinBUGS. The file defines the priors and the model, as seen when read in here

cat(readLines(con="bugmodel-GammaPrior.txt"), sep="\n")

## # Allen's Allee model based on a Ricker
## 
## model{
## 
## # Define prior densities for parameters
## K     ~ dunif(1.0, 22000.0)
## logr0    ~ dunif(-4.0, 2.0)
## logtheta ~ dunif(-4.0, 2.0)
## iQ ~ dgamma(0.0001,0.0001)
## iR ~ dgamma(0.0001,0.0001)
## 
## # Transform parameters to fit in the model
## r0 <- exp(logr0)
## theta <- exp(logtheta)
## 
## # Initial state
## x[1] ~ dunif(0,10)
## 
## # Loop over all states, 
## for(t in 1:(N-1)){
##   mu[t] <- x[t] + exp(r0 * (1 - x[t] / K) * (x[t] - theta) )
##   x[t+1] ~ dnorm(mu[t],iQ)
## }
## 
## # Loop over all observations, 
## for(t in 1:(N)){
##   y[t] ~ dnorm(x[t],iR)
## }
## 
## }

We define which parameters to keep track of, and set the initial values of parameters in the transformed space used by the MCMC. We use logarithms to maintain strictly positive values of parameters where appropriate. Because our priors on the noise parameters are inverse gamma distributed.

jags.params=c("K","logr0","logtheta","iR","iQ") # Don't need to save "x"
jags.inits <- function(){
  list("K"=init_p["K"],"logr0"=log(init_p["r0"]),"logtheta"=log(init_p["theta"]),"iQ"=1/0.05,"iR"=1/0.1,"x"=y,.RNG.name="base::Wichmann-Hill", .RNG.seed=123)
}
set.seed(12345)
time<-system.time(       
  jagsfit <- jags(data=jags.data, inits=jags.inits, jags.params, n.chains=n.chains, 
                  n.iter=n.iter, n.thin=n.thin, n.burnin=n.burnin,model.file="bugmodel-GammaPrior.txt")
)

## Compiling model graph
##    Resolving undeclared variables
##    Allocating nodes
##    Graph Size: 394
## 
## Initializing model

time <- unname(time["elapsed"]);

Convergence diagnostics for parametric bayes

jags_matrix <- as.data.frame(as.mcmc.bugs(jagsfit$BUGSoutput))
par_posteriors <- melt(cbind(index = 1:dim(jags_matrix)[1], jags_matrix), id = "index")

# Traces
ggplot(par_posteriors) + geom_line(aes(index, value)) + facet_wrap(~ variable, scale="free", ncol=1)

plot of chunk unnamed-chunk-8


# posterior distributiosn
ggplot(par_posteriors, aes(value)) + stat_density(geom="path", position="identity", alpha=0.7) +
  facet_wrap(~ variable, scale="free", ncol=2)

plot of chunk unnamed-chunk-8

mcmc <- as.mcmc(jagsfit)
mcmcall <- mcmc[,-2]
who <- colnames(mcmcall)
who

## [1] "K"        "iQ"       "iR"       "logr0"    "logtheta"

mcmcall <- cbind(mcmcall[,1],mcmcall[,2],mcmcall[,3],mcmcall[,4],mcmcall[,5])
colnames(mcmcall) <- who

Phase-space diagram of the expected dynamics

  gp_plot(gp, f, p, est$f, est$p, alt$f, alt$p, x_grid, obs, seed_i)

plot of chunk unnamed-chunk-10

Optimal policies by value iteration

Compute the optimal policy under each model using stochastic dynamic programming.

MaxT = 1000
matrices_gp <- gp_transition_matrix(gp$ZZ.km, gp$ZZ.ks2, x_grid, h_grid)
opt_gp <- value_iteration(matrices_gp, x_grid, h_grid, MaxT, xT, profit, delta, reward)

matrices_true <- f_transition_matrix(f, p, x_grid, h_grid, sigma_g)
opt_true <- value_iteration(matrices_true, x_grid, h_grid, OptTime=MaxT, xT, profit, delta=delta)

matrices_estimated <- f_transition_matrix(est$f, est$p, x_grid, h_grid, est$sigma_g)
opt_estimated <- value_iteration(matrices_estimated, x_grid, h_grid, OptTime=MaxT, xT, profit, delta=delta)

matrices_alt <- f_transition_matrix(alt$f, alt$p, x_grid, h_grid, alt$sigma_g)
opt_alt <- value_iteration(matrices_alt, x_grid, h_grid, OptTime=MaxT, xT, profit, delta=delta)


pardist <- mcmcall
pardist[,4] = exp(pardist[,4]) # transform model parameters back first
pardist[,5] = exp(pardist[,5])


matrices_par_bayes <- parameter_uncertainty_SDP(f, p, x_grid, h_grid, pardist)
opt_par_bayes <- value_iteration(matrices_par_bayes, x_grid, h_grid, OptTime=MaxT, xT, profit, delta=delta)


OPT = data.frame(GP = opt_gp$D, True = opt_true$D, MLE = opt_estimated$D, Ricker.MLE = opt_alt$D, Parametric.Bayes = opt_par_bayes$D)
colorkey=cbPalette
names(colorkey) = names(OPT)

Graph of the optimal policies

policies <- melt(data.frame(stock=x_grid, sapply(OPT, function(x) x_grid[x])), id="stock")
names(policies) <- c("stock", "method", "value")
ggplot(policies, aes(stock, stock - value, color=method)) +
  geom_line(lwd=1.2, alpha=0.8) + xlab("stock size") + ylab("escapement")  +
  scale_colour_manual(values=colorkey)

plot of chunk unnamed-chunk-12

Simulate 100 realizations managed under each of the policies



sims <- lapply(OPT, function(D){
  set.seed(1)
  lapply(1:100, function(i) 
    ForwardSimulate(f, p, x_grid, h_grid, x0, D, z_g, profit=profit, OptTime=OptTime)
  )
})

  
dat <- melt(sims, id=names(sims[[1]][[1]]))
dt <- data.table(dat)
setnames(dt, c("L1", "L2"), c("method", "reps")) 
# Legend in original ordering please, not alphabetical: 
dt$method = factor(dt$method, ordered=TRUE, levels=names(OPT))

ggplot(dt) + 
  geom_line(aes(time, fishstock, group=interaction(reps,method), color=method), alpha=.1) +
  scale_colour_manual(values=colorkey, guide = guide_legend(override.aes = list(alpha = 1)))

plot of chunk unnamed-chunk-13


ggplot(dt) + 
  geom_line(aes(time, harvest, group=interaction(reps,method), color=method), alpha=.1) +
  scale_colour_manual(values=colorkey, guide = guide_legend(override.aes = list(alpha = 1)))

plot of chunk unnamed-chunk-13

Notebook features: reproducible code

2013-04-26T00:00:00-07:00

I now use the dynamic documentation software called knitr to write most entries that shore results and figures. The code to replicate the results is included automatically, ensuring that what I say I did and what code I actually ran to get the results are consistent. Though I have written about knitr before, both regarding its use in my notebook and in my manuscripts, here I provide a quick summary of how a reader might actually reproduce a figure or result they come across in the notebook, as well as some of the possible problems involved.

As the code required for any given analysis can be quite involved, it is not pratical to provide free-standing scripts in this way. Instead, I write the algorithms as functions provided by an R package dedicated to the project, e.g. nonparametric-bayes, multiple-uncertainty, or warning-signals, which is version-managed on Github. The code displayed in the post can then be limited to the specific calls to analysis, data manipulation, and plotting functions unique to the exploration shown, without repeating the code for all algorithms involved.

The code for the analyses is also stored on github using the same dynamic documentation approach with knitr. These scripts are found in the inst/examples directory of my packages. This approach allows a given analysis to evolve with my research in a more tractable way than simply pasting updated copies as successive notebook entries. The notebook entries then become a place for me to synthesize the results of a script.

Though the package functions are usually backwards-compatible, proper reproducibility is only attained by having the version of the package from time of the result. This is easily accomplished by the seemless integration of Github and R using the devtools package. Consider a figure from a page of the notebook, such as the final histogram plot from this entry. The entry links to the script responsible for the figure, https://github.com/cboettig/nonparametric-bayes/blob/9d5cd1f027bdfe5f356dce83756726c95a6fcdd8/inst/examples/myers-exploration.md

We can install the entire research compendium at exactly the state it was at the time of the analysis using the hash (long chain of seemingly random characters, see the (upcoming) entry on hashes) using the clever devtools R package,

install_github("nonparametric-bayes", "cboettig", "9d5cd1f027bdfe5f356dce83756726c95a6fcdd8")

We can then copy and paste the code linked from the figure to replicate the results. This provides a fast and effective way replicate the work appearing in or linked to any entry. More importantly perhaps, this approach also allows one to repeat any given analysis with the most recent version of an algorithm and compare the results, since the package structure provides a logical seperation between algorithm and analysis. In practice such fine-grained control and invistigation is more important than simply being able to regenerate what has already been done without any further input.

This is not entirely failsafe. The package may depend on other packages, which themselves may have changed. For my use cases, it is a deal more reliable than running the current version of a package that is actively changing during my research. Readers interested in even more robust replication and verification should take a look at Roger Peng’s package stashR package and associated publications Eckel & Peng (2012) .

References

Sandy Eckel, Roger Peng, (2012) stashR: A Set of Tools for Administering SHared Repositories. http://CRAN.R-project.org/package=stashR

Notebook features: an introduction

2013-04-26T00:00:00-07:00

In keeping this open lab notebook, I have sought to address three goals (in addition to all the traditional reasons for keeping a lab notebook)

Provide an educational resource
Experiment with scientific infrastructure and tools for sharing and replicating research
Facilitate the rapid and open dissemination of scientific research

which are coincidentally evocative of NSF’s Broader Impacts areas. In this series of posts I plan to explore and illustrate some of my experiments to address these goals through various web-based tools available for an open notebook platform. Many of these have been documented in the notebook itself as I experiment with them (see #notebook-technology). Not all of those experiments pan out and older tools and techniques are often replaced with newer ones as I explore, and these posts are usually more technical notes written to help me think through and remember what I’m trying out. In order to provide a more accessible snapshot of notebook features, I thought it might be helpful to write a series of posts describing these tools and techniques.

Below is an index of posts in this theme that I will continue to update as I have a chance to finish them. If there’s anything about the notebook that you’d like to hear more about, feel free to suggest it in the comments.

Written

Planned

Notes

2013-04-25T00:00:00-07:00

finish pmc post from Tuesday (done).
Submit comment reply (done).

Nonparametric-bayes

Finishing up parametric vs nonparametric Bayes comparison runs
Need to add parametric Bayesian mean to the state-space diagram, fig 1
Compare with uniform priors on inverse standard deviations (precision)
Sensitivity analysis
mcmc convergence analysis for both Bayesian approaches
See allen.md example scripts in BUGS/, adapted to post soon

Code tricks

Plotting tricks

Selecting the order in the simulation legend requires declaring the model types as an ordered factor instead of a character,

dt$method = factor(dt$method, ordered=TRUE, levels=names(OPT))

and then set custom colors and match colors to the desired factor labels with scale_colour_manual, and override alpha for legends,

ggplot(dt) + 
  geom_line(aes(time, fishstock, group=interaction(reps,method), color=method), alpha=.1) +
    scale_colour_manual(values=colorkey, guide = guide_legend(override.aes = list(alpha = 1)))

Avoid having density plots appear as crude polygons using stat_density instead of geom_density and set geom to path. Need a ymax (not clear why?) or set position="identity" as shown,:

ggplot(posteriors, aes(value)) + stat_density(geom="path", position="identity", alpha=0.7)

initialize ssh-agent in .bashrc
Or for single session: ssh-agent bash, ssh-add ~/.ssh/id_rda

Misc

whoops, got into discussion of a fictional world of data publishing in the comments at dynamicecology
Speaking of data publishing issues on dynamicecology…

Empirical Demonstration Of That Clever Derivation

2013-04-24T00:00:00-07:00

Another unexpected gem from keeping an online notebook: Empirical evidence for the expected escape time problem! Getting to that in a moment….

After Marc pointed out to me that there is a clever analytic demonstration to back my claims about realized exit times being fast (posted earlier in my notebook under the title, a clever derivation of realized escape times, I’ve enjoyed a bit of a tangent in understanding this result that has long since been worked out by mathematicians but appears to be periodically rediscovered in ecological and evolutionary work in the most delightful ways.

For instance, Lande (1985) shows how this result can explain a pattern of punctuated equilibrium, such as sudden rapid jumps such as illustrated below in a simulation example from Freidlin et. al. (1998), need not result from some external process but are in fact the most likely trajectories to be seen in a purely stochastic transition.

from Mangel 2006, Theoretical Biologist’s toolbox

Ludwig (1975) considered this result in a more ecological context, addressing a variety of questions about persistence and coexistence that are naturally represented as hitting time problems as well. Note that the this result is interesting precisely in its contrast from the more familiar hitting time problems in that we focus on the time the realized trajectory actually takes to exit, rather than the waiting time until that exit occurs (e.g. the frequency of the transtions shown in the figure above).

These examples appear to be rediscoveries of particular cases that result more generally from the large deviation theory of Fiedlin and Wentzell, which states that the most likely path can be found by minimizing a variation equation (Hamiltonian, analgous to the minimizing action along paths in classical mechanics or the optimization under constraint of Jacobi-Bellman). For a system of discrete steps (e.g. births and deaths) in one dimension this is particularly trivial: the most likely path is the monotonic one (this should be easy to see by considering the likelihood of death events occuring in sequence vs that of death events interrupted by 1 birth and followed by 2 deaths.)

Empirical demonstrations?

Then an email to me earlier this week pointed out that perhaps an example of the phenomenon I discussed there had been demonstrated in Drake & Griffen (2009) :

A testable, counter-intuitive prediction of stochastic population theory is that, on average, for any interior interval of the domain of biologically attainable population sizes, the expected duration of increase equals the expected duration of decline. Here we report the first empirical tests of this hypothesis.

I do feel that this is an overstatement of the stochastic theory as I understand it – these times should only scale in a similar way, but can differ by an arbitrary coefficient. Indeed it is often the goal of the deviation theory analysis to determine this coefficent, see Faure & Schreiber (2011) . The authors seem quite concerned that theory they refer to applies to continuously valued diffusion processes alone, whereas I’m quite certain the more general results of the deviation theory hold just as well in discrete systems. (Perhaps this explains why they do not confirm this prediction in some subset of the data). Nevertheless, the context is the same as we consider – comparing the return time to a stable point from some small population size with the time required to decline from that point to a small size, as clearly indicated in Figure 1 of the manuscript.

Commendably, the authors make the data and code available (as an R script and .Rdata file). (According to the manuscript, this data is at http://dragonfly.ecology.uga.edu/drakelab/data.html, but in an example of link rot and the risk of independent archiving the lab prefers daphnia to dragonfly, and the data can now be found: http://daphnia.ecology.uga.edu/drakelab/datapage.html). These data draw on work from earlier experiments that provide some of the nicest sources of time series leading up to stochastic extinctions. Unfortunately it appears that only the duration of growth and extinction times for the replicates are shown in the data provided for this paper, I don’t see the actual timeseries themselves (which would be interesting to look at more closely).

References

John M. Drake, Blaine D. Griffen, (2009) Speed of Expansion And Extinction in Experimental Populations. Ecology Letters 12 772-778 10.1111/j.1461-0248.2009.01325.x
Mark Freidlin, Alexander Wentzell, (1998) {Random Perturbations of Dynamical Systems (Grundlehren der mathematischen Wissenschaften)}. 429-NA http://www.amazon.com/Perturbations-Dynamical-Grundlehren-mathematischen-Wissenschaften/dp/0387983627
R Lande, (1985) Expected time for random genetic drift of a population between stable phenotypic states. Proceedings of the National Academy of Sciences http://www.pnas.org/content/82/22/7641.short
Mathieu Faure, Sebastian Schreiber, (2011) Quasi-Stationary distributions for Randomly perturbed dynamical systems. arXiv.org http://arxiv.org/abs/1101.3420

We Need More Object Oriented Design In Comparative Methods

2013-04-23T00:00:00-07:00

The bad news is that the latest Geiger version (which is much improved in all respects) breaks my little pmc package, owing to changes in how fitContinuous output is structured. The good news is that the new geiger is much better than the old one, and that the pmc package is pretty trivial. So this should not be read as a complaint, but I think as a scientific developer community as a whole we could learn a few simple lessons here, which can help our code more reproducible and extensible.

Functions define the API, or application programming interface, of a package. Once a package has been released, developers are usually hesitant to change how a function call is made – what arguments it takes or what order they come in, since this would break any existing code. For the most part, the update is a good example of that, with minor exceptions – (for instance, a data.frame used to be a valid input to the fitContinuous dat argument, but now it must be a matrix). More problematic is the lack of an API in the original version of package for handling output objects. If you wanted to access the estimated value of the parameter lambda after running fit <- fitContinuous(phy, dat, model="lambda", ...), you would have needed to use fit[[1]]$lambda. Under the newer version of this function, such a call won’t work. Any code written to use the output of that function will not work either.

To avoid this kind of problem in the future, Jon has written the function coef which will extract the coefficients from the fit. For anyone starting to write packages, writing and using such extraction functions seems like an unnecessary hassle – after all it is often no faster to call the additional function than to just pull the data out of the return object yourself. But there is good reason to implement and use such methods. By drawing the data out using a function, it is possible to change the internal structure of the output without breaking the package API; without breaking code that depends on the package. In future, the coef function can just be similarly updated to point to the correct location for the model coefficients.

Better than providing just any function, the new coef and logLik functions are generic methods that can act on a wide range of model fits in R that return coefficients or log likelihoods. Just type methods("coef") in your R terminal after loading your commonly used libraries to see how many of them define such a function. Following the basic principle of object-oriented design, these generic function calls identify the class of object they are given, and figure out what method to use on that class. For instance, the list of stuff returned by fitContinous now has class gcfit, and a function coef.gcfit which the generic function coef uses to know where to find the coefficients in a gcfit object. Generic functions are particularly nice, since they allow us to write code that can deal with a wide variety of statistical fits without needing to know just how to perform those tasks. The pmc algorithm is a good example of this. Using the generics simulate, update, and logLik, we can write the algorithm as:

## Fit the original data using a function that returns an intelligent object "fit" which has methods "simulate", "update", and "logLik"
fit_A <- my_fitting_function(data, phy, model_pars_A, fitting_pars ...)
fit_B <- another_fitting_function(data, phy, model_pars_B, fitting_pars ...)

orig_LL <- logLik(fit_B) - logLik(fit_A)

## 1000 Simulations under each model
A_sims <- simulate(fit_A, 1000)
B_sims <- simulate(fit_B, 1000)

## here are the four fits
AA <- lapply(A_sims, fit_A, update)
AB <- lapply(A_sims, fit_B, update)
BA <- lapply(B_sims, fit_A, update)
BB <- lapply(B_sims, fit_B, update)

## which create 2 distributions
null_dist = sapply(AB, logLik) - sapply(AA, logLik)
test_dist = sapply(BB, logLik) - sapply(BA, logLik)

in which the exact same code could be used regardless of whether my_fitting_function was brownie.lite, fitContinuous, fitDiscrete, hansen, OUwie, any of the laser fns, etc etc, and another_fitting_function could be the same call with different parameters (e.g. call to hansen with a different painting) or a call to some other function entirely (e.g. brownie.lite). Instead, OUCH is the only one that does this, and it insists on using a different format for “data” and “phy” which makes it annoying to have it play nicely with other fitting routines.

The pmc package simply writes wrappers that will provide these generic methods to some of these functions. Because different packages (e.g. ouch, geiger) handle data differently, the wrappers must also be able to convert back and forth between these formats if we want to compare methods across different packages in this way. Though straight forward, it is not good practice to have the methods such as simulate and update provided in a different package than the actual fitting routines, but the number of package fitting methods continues to grow and I haven’t persuaded anyone to consider including such routines. With my postdoctoral work outside the area of phylogenetics, updating pmc each time is impractical. If packages provided such generic routines, code such as the above would work out of the box without the need for a specialized pmc package, which appears to make this very intuitive technique more opaque then necessary.

There are many other good statistical tests that could benefit from such methods (particularly simulation methods which could assist in many ways). I hope one day they may be more popular in our phylogenetics community.

Bayesian Parametric Uncertainty Sdp

2013-04-19T00:00:00-07:00

Adapting the parametric uncertainty represented by the posterior distributions of the Bayesian estimate (see earlier notes) to the stochastic dynamic programming solution for the optimal policy. Simply requires evaluating the expectation over the distribution, but is computationally intensive given the spread and the three parameters.

f_transition_matrix <- function(f, p, x_grid, h_grid, sigma_g, pardist){
  lapply(h_grid, par_F, f, p, x_grid, sigma_g, pardist)
}


par_F <- function(h, f, p, x_grid, sigma_g, pardist, n_mc = 100){
  
  # Set up monte carlo sampling 
  d <- dim(pardist)
  indices <- round(runif(n_mc,1, d[1]))
  
  # Compute the matrix
  F_true <- 
    sapply(x_grid, function(x_t){           # For each x_t  

      bypar <- sapply(indices, function(i){ # For each parameter value
        p <- unname(pardist[i,c(4,1,5)])    # parameters for the mean, at current sample 
        mu <- f(x_t,h,p)
        est_sigma_g <- pardist[i,2]         # Variance parameter 

        if(snap_to_grid(mu,x_grid) < x_grid[2]){ # handle the degenerate case 
          out <- numeric(length(x_grid))
          out[1] <- 1
          out
        } else {
          out <- dlnorm(x_grid/mu, 0, est_sigma_g) 
        }
      })  
      ave_over_pars <- apply(bypar, 1, sum) # collapse by weighted average over possible parameters    
      ave_over_pars / sum(ave_over_pars)
      })
  F_true <- t(F_true)
        
}


snap_to_grid <- function(x, grid) sapply(x, function(x) grid[which.min(abs(grid - x))])   


# internal helper function
rownorm <- function(M) 
  t(apply(M, 1, function(x){ 
    if(sum(x)>0){
      x/sum(x)
    } else {
      out <- numeric(length(x))
      out[1] <- 1
      out
    }
  }))

Run the Bayesian analysis to obtain posterior distributions for parameters.

Test case using perturbed parameters

First, sanity test. Use the correct parameter values (slightly perturbed).

pardist <- mcmcall
pardist[,1] = p[2] + rnorm(100, 0, 0.000001)
pardist[,4] = p[1] + rnorm(100, 0, 0.000001)
pardist[,2] = sigma_g + rnorm(100, 0, 0.000001)
pardist[,5] = p[3] + rnorm(100, 0, 0.000001)

Compute optimal policy

sdp = f_transition_matrix(f, p, x_grid, h_grid, sigma_g, pardist)
s_opt <- value_iteration(sdp, x_grid, h_grid, OptTime=1000, xT, profit, delta)

Compare to the case without parameter uncertainty (growth noise only)

SDP_Mat <- determine_SDP_matrix(f, p, x_grid, h_grid, sigma_g)
pars_fixed <- value_iteration(SDP_Mat, x_grid, h_grid, OptTime=1000, xT, profit, delta)

Plot results

require(reshape2)
policies <- melt(data.frame(stock=x_grid, pars.uncert = x_grid[s_opt$D], pars.fixed = x_grid[pars_fixed$D]), id="stock")
ggplot(policies, aes(stock, stock - value, color=variable)) + geom_line(alpha=1) + xlab("stock size") + ylab("escapement")

plot of chunk unnamed-chunk-6

Using actual estimates

pardist <- mcmcall

Transform parameters back

pardist[,4] = exp(pardist[,4])
pardist[,5] = exp(pardist[,5])

Compute optimal policy

sdp = f_transition_matrix(f, p, x_grid, h_grid, sigma_g, pardist)
s_opt <- value_iteration(sdp, x_grid, h_grid, OptTime=1000, xT, profit, delta)

Compare to the case without parameter uncertainty (growth noise only)

SDP_Mat <- determine_SDP_matrix(f, p, x_grid, h_grid, sigma_g)
pars_fixed <- value_iteration(SDP_Mat, x_grid, h_grid, OptTime=1000, xT, profit, delta)

Plot results

require(reshape2)
policies <- melt(data.frame(stock=x_grid, pars.uncert = x_grid[s_opt$D], pars.fixed = x_grid[pars_fixed$D]), id="stock")
ggplot(policies, aes(stock, stock - value, color=variable)) + geom_line(alpha=1) + xlab("stock size") + ylab("escapement")

plot of chunk unnamed-chunk-11

To Do

So far this is just a proof of principle example.

Needs to be adjusted to account for uncertainty in the estimates of the noise processes as well.
Needs to be written as generic and documented functions, add to nonparametric-bayes package routines.

Notes

2013-04-18T00:00:00-07:00

Misc coding

knitcitations handling of html formatting in tooltip: html inside the tag must needs be escaped. knitcitations/#37
pmc functions

hmm.. how do we easily wrap a function such that it returns its arguments (given and default) in a named list (appropriate for do.call(function_name, args_list))? match.call() inside the function will give the literal string of how the function was called, which, if evaluated in the same environment with same variables in working space, will return the same results, but doesn’t lend itself to updating any one of those arguments. Hoped to extract variable values with some combination of formals and match.arg() but no luck.

Mendeley API authentication solved using Michal’s clever implementation for OAuth, see my original query on SO

nonparametric-bayes

Implement the SDP calculation for the posterior distributions returned in the parametric comparison case nonparametric-bayes/allen.Rmd.
Working out the integration over uncertainty for the parameter distribution. Brute force approach clearly doesn’t scale, even on coarse grid, to many parameters. After discussion with Marc, implementing as Monte Carlo instead. As we get samples from the posterior out of the BUGs fit in the first place, this is much more efficient. Should of tried that the first time. nonparametric-bayes/par_uncertainty_sdp.Rmd

warning-signals

Analytic calculation of eigenvalues for managed vs unmanaged system
finish fallacy reply
finish ews-review reply

Geospatial coordinates from plain text

Discussed the problem of geolocation from plain text discriptions with Simon a few days ago. I’ve written a simple iR function to query place names against the Google Map API, but as Simon has demonstrated, this will often return the coordinates of a cafe in San Fransisco rather than the lake in Italy you meant to get. At DataCite 2011 I had met some folks who mentioned developing a machine learning algorithm for this, though stupidly it looks like I did not record this in my notes. Oh well, Google turned up the following interface: http://geoparser.digmap.eu/advanced.jsp

Click the geoparse example and scroll down in the XML; they assign probabilities of possible spatial matches. For instance, Hiroshima as the name of a “railroad station” gets 0.05 probability, whereas the majority of the weight falls on “populated place” in Japan.

It looks like you could query against the service directly, haven’t really explored.

They have a paper out on this too, http://oa.upm.es/4367/1/INVE_MEM_2008_60075.pdf

Reading

Shelton et. al. (2013) provide a rather elegant example of how accounting for individual variation improves modeling efforts in growth dynamics.
Measurement error is important but difficult to deal with; a proper state-space model should use all the replicates to estimate the measurement error process. Knape et. al. (2013) argue that you can do pretty well with just the standard errors from the replicates. – Um, but don’t we know this? We expect the central limit theorem to be pretty good a lot of the time. The demonstration that it can work is not quite as useful as a demonstration of when we might expect it not to; but my quick read may be missing something. Certainly an interesting point to wrestle with before going off the deep end with the state-space modeling each time…
Looking over an early draft of Duncan and Deborah’s “XML and Web Technologies for Data Sciences with R”. Looks fantastic – where was this 3 years ago?
Just launched: http://bison.usgs.ornl.gov (US only). Individual sample occurances, but no morphological measurements available.

References

Jonas Knape, Panagiotis Besbeas, Perry de Valpine, (2013) Using Uncertainty Estimates in Analyses of Population Time Series. Ecology 130417174952006-NA 10.1890/12-0712.1
Andrew O. Shelton, William H. Satterthwaite, Michael P. Beakes, Stephan B. Munch, Susan M. Sogard, Marc Mangel, (2013) Separating Intrinsic And Environmental Contributions to Growth And Their Population Consequences. The American Naturalist 000-000 10.1086/670198

Notes

2013-04-17T00:00:00-07:00

warning-signals

Finishing up edits.
Writing PRSB reply

Logistics

In tomorrow’s Marc meeting, check in about what to include for the CSTAR report
manuscript / analysis updates
Comments on recent lit?

ropensci

treebase/#4

Reading

Dai et. al. (2013) , with introductory piece by Carpenter (2013) . In this they look only at the spatial extent of a perturbation. Presumbably one might attempt to compare the spatial extent over time to establish a baseline, or otherwise identify a control system. Very difficult to see how this can be applied without appropriate references.

Also provides an elegant and clear, if rather intuitive, demonstration that spatially connected / mixing populations will dampen out the signal. Perhaps the interpretation of this misplaces the emphasis – for the connectedness is allowing the insult (dilution of nutrients) to diffuse among neighbors may be strengthening the resilience relative to an isolated population, and therefore it is only appropriate that the signal would weaken.

Hehe. Nice to see our Prosecutor’s Fallacy paper (Boettiger & Hastings, 2012 ) being used as further evidence for the value of experimental confirmation: “tests of these indicators in replicated experiments, which avoid the bias intro- duced by selective sampling (23) are lacking.”

Glaser et. al. (2013) . Nonlinear forecasting doesn’t perform well across the board on these various fisheries data. Authors conclude that this suggests these systems are inherently unpredictable, though seems one could do more to rule out methodological weaknesses and make a more precise statement (perhaps a mutual information claim). Their approach having a large suite of data sets against which to test the performance of a method is nevertheless an interesting idea that’s had some popularity in the machine learning world. Hopefully more researchers will attempt a parallel analysis of these data.

References

C. Boettiger, A. Hastings, (2012) Early Warning Signals And The Prosecutor’S Fallacy. Proceedings of The Royal Society B: Biological Sciences 279 4734-4739 10.1098/rspb.2012.2085
Stephen R. Carpenter, (2013) Complex Systems: Spatial Signatures of Resilience. Nature 10.1038/nature12092
Lei Dai, Kirill S. Korolev, Jeff Gore, (2013) Slower Recovery in Space Before Collapse of Connected Populations. Nature 10.1038/nature12071
Sarah M Glaser, Michael J Fogarty, Hui Liu, Irit Altman, Chih-Hao Hsieh, Les Kaufman, Alec D MacCall, Andrew A Rosenberg, Hao Ye, George Sugihara, (2013) Complex Dynamics May Limit Prediction in Marine Fisheries. Fish And Fisheries n/a-n/a 10.1111/faf.12037

a few notes on various projects

2013-04-16T00:00:00-07:00

warning-signals

Working on revisions for ews-review
Grr, the challenges of open science – can’t stick confidential reviews in ews-review repo where they belong, or track them on the issues tracker, without disclosing their contents. What are the terms of use for these things anyhow>

phylogentics / pmc

Apparently pmc does have users – at least one recognized that my Vignette is broken following the latest Gieger update. Need to add simulate and update methods for the new fitContinuous, and then integrate these. pmc/#3
Another pmc request. Sounds like I should also add support for OUwie fits. See if I can get Jeremy to add simulate and update methods. pmc/#4

Dear phylogeneticists: please define a methods API for your functions. Accessing coefficients using something like output[[1]][[2]]$somevar$name[4] is not only cumbersome, but bound to break when you modify your function, so provide a coef method. Likewise provide simulate and update methods so we can all raise the bar on our statistics a little bit.

Alternatively put the code on Github, where sensible folks can fork it, add some common-sense classes and give you a pull request.

Code tricks

Because I’m always wondering how to do this one: let’s see what methods are defined for Geiger’s new gfit class:

methods(class="gfit")

Returns:

[1] coef.gfit*   logLik.gfit*

   Non-visible functions are asterisked

ht Michael Hannon for that one, and a good way to confuse some folks on SO.

knitcitations

Fixed a minor issue in tooltip formatting, knitcitations/#37. Unfortunately not updated in earlier notebook entires, but should be fixed from here on).
Interesting ideas from some folks about adding CSL support, and thinking about being able to pull data into knitcitations from personal zotero libraries, knitcitations/#38. Zotero API seems to have some nice features for listing entries of a collection in a variety of file formats, including CSL styling for the returns.
Nearly a month since last push (0.4-4). Should update CRAN to current development version (0.4-6) sometime this week…

Reading

The debates continue: Szuwalski (2013) , and the reply Vert-pre et. al. (2013) , reflecting confusion over the article earlier this year, Vert-pre et. al. (2013b) .
Disupting the claim of Ditlevsen & Johnsen (2010) and others, Cimatoribus et. al. (2013) argue that DO events do show warning signals. But are they chance transitions then?
Looking over Lenton (2011) , which has a review of some of the evidence against warning signals for noise-induced transitions.
An even earlier result from Ludwig, in Ludwig (1975) .

References

A. A. Cimatoribus, S. S. Drijfhout, V. Livina, G. van der Schrier, (2013) Dansgaard–Oeschger Events: Bifurcation Points in The Climate System. Climate of The Past 9 323-333 10.5194/cp-9-323-2013
Peter D. Ditlevsen, Sigfus J. Johnsen, (2010) Tipping Points: Early Warning And Wishful Thinking. Geophysical Research Letters 37 n/a-n/a 10.1029/2010GL044486
Timothy M. Lenton, (2011) Early Warning of Climate Tipping Points. Nature Climate Change 1 201-209 10.1038/nclimate1143
Donald Ludwig, (1975) Persistence of Dynamical Systems Under Random Perturbations. Siam Review 17 605-640 10.1137/1017070
C. Szuwalski, (2013) Production is A Poor Metric For Identifying Regime-Like Behavior in Marine Stocks. Proceedings of The National Academy of Sciences 110 E1436-E1436 10.1073/pnas.1301759110
K. A. Vert-pre, R. O. Amoroso, O. P. Jensen, R. Hilborn, (2013) Reply to Szuwalski: Policies Robust to Uncertainty in Causes of Productivity Changes Are Needed. Proceedings of The National Academy of Sciences 110 E1437-E1437 10.1073/pnas.1302181110
K. A. Vert-pre, R. O. Amoroso, O. P. Jensen, R. Hilborn, (2013) Frequency And Intensity of Productivity Regime Shifts in Marine Fish Stocks. Proceedings of The National Academy of Sciences 110 1779-1784 10.1073/pnas.1214879110

Parametric Bayesian Example

2013-04-12T00:00:00-07:00

As discussed with Marc in yesterday’s meeting, it would be useful to compare the Gaussian process, as a nonparametric Bayesian estimate, to the optimal management solution under a parametric Bayesian case, e.g. with the correct underlying model. This is implemented in BUGS (jags) through the R interface as described in Bolker et. al. (2013) , and follows the state-space model analysis along the lines outlined in Pedersen et. al. (2011) .

We base the estimate on the same simulated data as for the GP and MLE approaches,

f <- RickerAllee
p <- c(2, 10, 5)
K <- 10
allee <- 5

with the same parameters (many of which we won’t need until the SDP value iteration),

sigma_g <- 0.05
sigma_m <- 0
z_g <- function() rlnorm(1, 0, sigma_g)
z_m <- function() 1 + (2 * runif(1, 0, 1) - 1) * sigma_m
x_grid <- seq(0, 1.5 * K, length = 101)
h_grid <- x_grid
profit <- function(x, h) pmin(x, h)
delta <- 0.01
OptTime <- 20  # stationarity with unstable models is tricky thing
reward <- 0
xT <- 0
seed_i <- 1
Xo <- K  # observations start from
x0 <- Xo  # simulation under policy starts from
Tobs <- 35

Generate the simulated data

obs <- sim_obs(Xo, z_g, f, p, Tobs = Tobs, nz = 1, harvest = sort(rep(seq(0, 
    0.5, length = 7), 5)), seed = seed_i)

And fit the MLE models as before

alt <- par_est(obs, init = c(r = p[1], K = mean(obs$x[obs$x > 0]), 
    s = sigma_g))
est <- par_est_allee(obs, f, p, init = c(r = p[1] + 1, K = p[2] + 
    2, C = p[3] + 2, s = sigma_g))

Parametric Bayesian model fit

We will use the jags interface from R,

library(R2jags)

initialize parameters. The parametric equation (in this case, matching the true dynamics) and priors are defined in BUGS notation in text file.

init_p = est$p
names(init_p) = c("r0", "K", "theta")
y <- obs$y[-1]
N = length(y)

jags.data <- list("N", "y")
n.chains = 1
n.iter = 20000
n.burnin = floor(n.iter/2)
n.thin = max(1, floor(n.chains * (n.iter - n.burnin)/1000))

JAGS using gamma priors on precision

jags.params=c("K","logr0","logtheta","iR","iQ"); # Don't need to save "x"
jags.inits <- function(){
  list("K"=init_p["K"],"logr0"=log(init_p["r0"]),"logtheta"=log(init_p["theta"]),"iQ"=1/0.05,"iR"=1/0.1,"x"=y,.RNG.name="base::Wichmann-Hill", .RNG.seed=123)
}
set.seed(12345)
time<-system.time(       
  jagsfit <- jags(data=jags.data, inits=jags.inits, jags.params, n.chains=n.chains, 
                  n.iter=n.iter, n.thin=n.thin, n.burnin=n.burnin,model.file="bugmodel-GammaPrior.txt")
)

time <- unname(time["elapsed"]);

Summarize the results

library(emdbook)
library(coda)
tfit_jags_m <- as.mcmc.bugs(jagsfit$BUGSoutput)
print(xyplot(tfit_jags_m))

Parameter traces provide a visual tool to assess performance and convergence of the MCMC routine

print(densityplot(tfit_jags_m))

The resulting posterior distributions for the model parameters inferred

We collect model parameters over the replicate runs,

mcmc <- as.mcmc(jagsfit)
mcmcall <- mcmc[, -2]
who <- colnames(mcmcall)
mcmcall <- cbind(mcmcall[, 1], mcmcall[, 2], mcmcall[, 3], mcmcall[, 
    4], mcmcall[, 5])
colnames(mcmcall) <- who

Further parameter exploration

We will need to integrate over parameter uncertainty. Can consider the discritized binning

theta <- exp(mcmcall[, "logtheta"])
theta_dist <- hist(theta, freq = FALSE)

theta_dist$mids

##  [1] 0.25 0.75 1.25 1.75 2.25 2.75 3.25 3.75 4.25 4.75 5.25 5.75 6.25 6.75
## [15] 7.25

theta_dist$density

##  [1] 1.154 0.256 0.130 0.092 0.068 0.058 0.044 0.042 0.042 0.018 0.022
## [12] 0.026 0.022 0.014 0.012

delta <- theta_dist$mids[2] - theta_dist$mids[1]

Evaluating the value function given fixed at each of theta_dist$mids multiplied by theta_dist$density * delta and summed over each is going to be slow for a single parameter but simply unrealistic for higher dimensions.

References

Benjamin M. Bolker, Beth Gardner, Mark Maunder, Casper W. Berg, Mollie Brooks, Liza Comita, Elizabeth Crone, Sarah Cubaynes, Trevor Davies, Perry de Valpine, Jessica Ford, Olivier Gimenez, Marc Kery, Eun Jung Kim, Cleridy Lennert-Cody, Arni Magnusson, Steve Martell, John Nash, Anders Nielsen, Jim Regetz, Hans Skaug, Elise Zipkin, (2013) Strategies For Fitting Nonlinear Ecological Models in R, ad Model Builder, And Bugs. Methods in Ecology And Evolution n/a-n/a 10.1111/2041-210X.12044
M.W. Pedersen, C.W. Berg, U.H. Thygesen, A. Nielsen, H. Madsen, (2011) Estimation Methods For Nonlinear State-Space Models in Ecology. Ecological Modelling 222 1394-1400 10.1016/j.ecolmodel.2011.01.007

Tutorial On Likelihood And Roc Methods For Early Warning Signals

2013-04-11T00:00:00-07:00

In this tutorial we walk through how to use the likelihood-based approach of Boettiger & Hastings (2012) to provide an early warning signal for a critical transition, and how to generate ROC curves discussed in the same paper for comparing different early warning signals. Context and details about the methodology are presented in the paper; here we focus on using the methodology provided in the accompanying R package. All of the examples shown here can be reproduced by pasting the code provided into an R terminal. We will need the following libraries to run this example. If you don’t have them, they can be installed from CRAN using the install.packages function.

library(ggplot2)
library(reshape2)
library(multicore)
library(devtools)

We will be using custom functions provided in the R package Boettiger, (2012) that accompanies Boettiger & Hastings, (2012), which can be installed directly from the development repository on GitHub:

install_github("earlywarning", "cboettig")
library(earlywarning)

We begin by loading the ibm_critical data provided in the package (from an individual-based model simulation of a critical transition). We plot the raw data to take a look at it.

data("ibms")
plot(ibm_critical, type="b")

Fit both models to the original data, record the observed likelihood ratio of the original data

A <- stability_model(ibm_critical, "OU")
B <- stability_model(ibm_critical, "LSN")
observed <- -2 * (logLik(A) - logLik(B))

Perform the bootstrapped model comparison on the parallel cluster.

runtime <- system.time(
reps <- mclapply(1:200, function(i) compare(A, B)))

Which took 4176.78 seconds to run for the example shown here.

A helper function extracts the likelihood ratios under each of the pairwise comparisons (The null distribution: -2 times the log likelihood of data fit under A that had been simulated under A, minus the log likelihood of fits under A simulated under B; and the test distribution: fit under B when simulated under A, minus the loglikehood of being fit under B, simulated under B) as a data frame. we show the top of the data frame as output below.

lr <- lik_ratios(reps)
head(lr)

##   simulation rep     value
## 1       null   1  0.001659
## 2       test   1  2.255139
## 3       null   2  1.892615
## 4       test   2  9.804084
## 5       null   3  4.795269
## 6       test   3 22.696982

We use this data to generate the overlapping distributions shown in Boettiger & Hastings, (2012),

ggplot(lr) + geom_density(aes(value, fill=simulation), alpha=0.6) + geom_vline(aes(xintercept=observed))

A second helper function generates the ROC curve from the likelihood ratio data, providing an alternative way to visualize this overlap:

roc <- roc_data(lr)
ggplot(roc) + geom_line(aes(False.positives, True.positives))

ROC curves are particularly useful in comparing the power of a variety of approaches. For instance, we can compare the performance of the likelihood-based signal shown above to the traditional approach of using a correlation coefficient to detect the increase in some summary statistic such as variance or autocorrelation. To obtain replicates, we will still simulate under the null (OU) and test (LSN) models estimated above, but instead of computing the likelihood of these data, we will estimate the commonly used rank correlation coefficient, Kendall’s tau, to quantify the increase in variance, autcorrelation, and skew observed in a moving window over the data.

var <- bootstrap_trend(ibm_critical, window_var, method="kendall", rep=200)
acor <- bootstrap_trend(ibm_critical, window_autocorr, method="kendall", rep=200)
skew <- bootstrap_trend(ibm_critical, window_skew, method="kendall", rep=200)

These data are formatted like the likeihood ratio data above, only that the statistic is now Kendall’s tau.

ggplot(var) + geom_density(aes(value, fill=simulation), alpha=0.6) + geom_vline(aes(xintercept=observed))

We can combine the data for the ROC curves of each indicator to compare:

indicators <- melt(list(var = roc_data(var), acor = roc_data(acor), skew = roc_data(skew), lr = roc), id = c("Threshold", "False.positives", "True.positives"))
ggplot(indicators) + geom_line(aes(False.positives, True.positives, color=L1))

C. Boettiger, A. Hastings, (2012) Quantifying Limits to Detection of Early Warning For Critical Transitions. Journal of The Royal Society Interface 9 2527-2539 10.1098/rsif.2012.0125

nonparametric-bayes: framing the issues for the manuscript

2013-04-11T00:00:00-07:00

Marc meeting: nonparametric-bayes manuscript

Very good meeting today going over some big picture considerations in the presentation of the nonparametric-bayes manuscript with Marc. (Also see new entries in issues log on nonparametric-bayes.) Make it clear that we are making a point rather than providing a tool. Focus on the tipping point context rather than the general management context.

Comparison against a parametric Bayesian model (40)
Comparison against an active learning parametric model facing a tipping point (41)

Parametric Bayesian model

Drafting parametric Bayesian case for comparison, following Marc’s suggestion. So far, code as implemented in BUGS

Warning signals Reading

Alan pointed out Boerlijst et. al. (2013) on warning signals in PLoS ONE today. A very nice piece! It’s quite useful how it show that the early warning signal can appear in unmonitored variables (e.g. juvenile dynamics) or in variables without adequate noise to provide a signal. Kind of funny that they pitch it as a counterexample to the conjecture that early warning signals are universal, since even Scheffer has avoided that word, and it’s not really a counterexample given the caveat of some process noise and complete observability of the system people have so far assumed (while your work with Derin is a more conclusive counter-example by going beyond fold bifurcations)… Anyway, good examples. We need a better understanding of warning signals in this multi-dimensional context at large.
Recently Noam pointed out Streeter & Dugmore (2013) , which looks like a nice spatial warning signal example (with the classic patterns, rather than the Bel et. al. (2012) pattern), but haven’t had time for a closer read yet.

References

Golan Bel, Aric Hagberg, Ehud Meron, (2012) Gradual Regime Shifts in Spatially Extended Ecosystems. Theoretical Ecology 5 591-604 10.1007/s12080-011-0149-6
Maarten C. Boerlijst, Thomas Oudman, André M. de Roos, Ricard V. Solé, (2013) Catastrophic Collapse Can Occur Without Early Warning: Examples of Silent Catastrophes in Structured Ecological Models. Plos One 8 e62033-NA 10.1371/journal.pone.0062033
Donald Ludwig, Carl J. Walters, (1981) Measurement Errors And Uncertainty in Parameter Estimates For Stock And Recruitment. Canadian Journal of Fisheries And Aquatic Sciences 38 711-720 10.1139/f81-094
Zeev Schuss, Bernard J. Matkowsky, (1979) The Exit Problem: A New Approach to Diffusion Across Potential Barriers. Siam Journal on Applied Mathematics 36 604-623 10.1137/0136043
R. Streeter, A. J. Dugmore, (2013) Anticipating Land Surface Change. Proceedings of The National Academy of Sciences 110 5779-5784 10.1073/pnas.1220161110
Ethan White, Elita Baldridge, Zachary Brym, Kenneth Locey, Daniel McGlinn, Sarah Supp, (2013) Nine simple ways to make it easier to (re)use your data. PeerJ PrePrints http://peerj.com/preprints/7
Donald Ludwig, (1981). Escape from Domains of Attraction for Systems Perturbed by Noise, Nonlinear Phenomena in Physics and Biology.

A Clever Derivation On Realized Escape Times

2013-04-05T00:00:00-07:00

I have often observed (e.g. Boettiger & Hastings (2012) ) that if purely stochastic transitions out of a potential well happen, then they happen fast (dispite the fact that waiting time for them to happen is very long). Yesterday Marc mentioned an interesting proof of this from Don Ludwig, who was able to show that the waiting time to reach a large deviation in a time , given that it reaches it at all, is of the same order as the return time from the large deviation to the equilibrium, . That’s an absolutely fascinating conclusion, indeed a bit stronger than even I had expected (the waiting time until an escape is observed rather different! – of order .

Don’s derivation is repeated in pages 302-311 Marc’s “Theoretical Biologist’s Toolbox,” rather delightfully too (e.g. the “Kumura Maneuver”, or sidenotes such as “it is also kind of fun to do a web search with key words ‘Dawson’s Integral’”). Lande (1985) provides a somewhat different derivation of the same result.

Return time

The return time calculation is straight forward. Consider our OU model

We ask for the return time into a neighborhood of the origin, starting at distant from the origin. The deterministic solution

implies , so setting and solving for , we find that is within of in time , or , as claimed.

Waiting time

The time we must wait before any trajectory can reach a given boundary is a classic hitting-time problem for the OU process that can be solved in several ways. Any of these basically start from PDE that must be satisfied for the hitting time (e.g. see Crispin Gardner, Stochastic Methods) for the OU process at stationary state (),

With the appropriate boundary conditions . A little integration (see Dynkin’s formula) shows this scales as (Compare to the Brownian walk that doesn’t have to escape any pull, which escapes at rate .

Exit time

Most trajectories will thus return many times to the origin (stable point) before finally escaping. But if we focus just on the part of the trajectory from some value near the origin, that eventually escapes without ever crossing the origin again,

We are interested in comparing the average time taken by trajectories exiting by time relative to the probability of exiting at all (i.e. as ),

From here we set up the classic hitting time solution for the OU for the time-dependent PDE,

and with initial condition , , and . A bit of effort and an approximation to Dawson’s integral for leads Don to bound the mean time to exit,

as claimed.

Results from Marc’s other suggestion

Marc also suggested exploring this by directly comparing steps left and right. A related way to measure the relative number of steps in the positive direction relative to the negative direction is Kendall’s tau rank correlation coefficient, which also looks at the number of increases relative to the number of decreases.

I’ve computed this statistic for my sample trajectories and plotted as a histogram here. The negative skew shows the tendancy to decrease (which is the direction of the deviation I selected – smaller population sizes)

I’ve also computed the ratio of steps up to steps down and histogrammed that: like the above, it shows more steps down than up, though not as dramatic as I might think by any means. Have to think about the role of the step size between observations in formulating this perhaps.

For the sake of completeness:

I may try and add a demonstration of Ludwig’s result above when I get some more free time to fiddle; will just need to add some additional simulations of return trajectories.

References

C. Boettiger, A. Hastings, (2012) Early Warning Signals And The Prosecutor’S Fallacy. Proceedings of The Royal Society B: Biological Sciences 279 4734-4739 10.1098/rspb.2012.2085
R Lande, (1985) Expected time for random genetic drift of a population between stable phenotypic states. Proceedings of the National Academy of Sciences http://www.pnas.org/content/82/22/7641.abstract

Notebook Features: parsing linked data in the semantic notebook

2013-04-04T00:00:00-07:00

In a post a while back I originally put forward the idea of a semantic lab notebook. Semantics, or linked data, are among the most powerful concepts to emerge in online science for scholarly data organization and communication. I have slowly been adding and exploring new ways to introduce semantic concepts into the notebook, which I have documented along the way under my #semantics tag. In this post, rather than discuss how to generate the semantic data, I try to focus on some of the things we can do with it. This really just scratches the surface of possibilities, but should at least illustrate the general idea.

We will start with some very simple examples exploiting the semantics inherent in HTML5. We can then work up to richer examples that rely on XML-based parsing. The richest potential of the semantic notebook lies in leveraging the RDFa content, whose terms are defined as ontologies over which a machine can apply reasoning and formal logic against other web content (see, e.g. Shadbolt et. al. (2006) for further unformation). Though we show how to extract the and parse the RDF here, exploiting the RDF in the last example must wait to a later post.

Parsing semantic HTML

Our first set of examples will address parsing the semantic HTML directly. For background on how these elements are added to the notebook, see this entry. We will use R with it’s excellent collection of web, XML parsing, and text-mining tools to take advantage of the semantic structure. First we load the required packages,

library(RCurl)
library(XML)
library(tm)
library(wordcloud)
library(RColorBrewer)

We can get a list of pages to download from the sitemap

pagelist <- readLines("http://carlboettiger.info/sitemap.txt")
pagelist <- pagelist[grep("/201\\d/", pagelist)]  # drop non-posts)
pages <- lapply(pagelist, getURLContent, followlocation = TRUE)

Or, if an archive is available locally, (e.g. from figshare cache), we can read the files in directly.

pages <- system("ls ~/Documents/labnotebook/_site/2***/*/*/*.html", 
    intern = TRUE)

We parse each of the pages as HTML so that we can manipulate them with XML tools

html <- lapply(pages, htmlParse)

For instance, we can easily get the content of all entries:

text <- sapply(html, xpathSApply, "//article", xmlValue)

We can also extract metadata based on the semantic markup.

titles <- sapply(html, xpathSApply, "//title", xmlValue)
categories <- sapply(html, xpathSApply, "//node()[@class='category']", 
    xmlValue)
tags <- sapply(html, xpathSApply, "//node()[@class='tag']", xmlValue)

R makes it easy to summarize this data, e.g. by generating a table of the number of entries in each category, or a wordcloud of the tags.

table(unlist(categories))


 computation      ecology    evolution open-science     teaching 
          40          376          287           85           17

wordcloud(Corpus(VectorSource(tags)))

Extracting citations

Citation information can be encoded

We can perform more direct text mining as well. For instance, we extract all DOIs found in the text:

doi_pattern = "\\b(10[.][0-9]{4,}(?:[.][0-9]+)*/(?:(?![\"&'<>])\\S)+)\\b"
require(gsubfn)
dois <- strapply(text, doi_pattern, perl = TRUE)  #text[-462]
head(sort(table(unlist(dois))))


10.1002/(SICI)1520-6602(1998)1:1                 10.1002/bjs.6880 
                               1                                1 
                10.1002/etc.2140           10.1006/jtbi.1998.0660 
                               1                                1 
          10.1006/jtbi.2000.1080           10.1006/jtbi.2001.2299 
                               1                                1

Or generate a wordcloud of the full text


carl <- Corpus(VectorSource(text))
carl <- tm_map(carl, removePunctuation)
carl <- tm_map(carl, tolower)
carl <- tm_map(carl, function(x) removeWords(x, stopwords()))

carl.tdm <- TermDocumentMatrix(carl)
carl.m <- as.matrix(carl.tdm)
carl.v <- sort(rowSums(carl.m), decreasing = TRUE)
carl.d <- data.frame(word = names(carl.v), freq = carl.v)


wordcloud(carl.d$word, carl.d$freq, scale = c(8, 0.4), min.freq = 3, 
    max.words = 100, random.order = FALSE, rot.per = 0.15, colors = brewer.pal(8, 
        "Dark2"))

RDFa parsing

RDF triples are the mainstay of semantic, linked data. Unlike the more text-mining oriented examples above, data in this format follows a strict and universal standard which allows a machine to identify meaning rather precisely. Critically, this allows one to automatically link data appearing in the notebook to data elsewhere on the web without the ambiguities of natural language that for instance, might confuse the animal jaguar with the car.

RDFa is a way of adding these precise statements to HTML, again see the earlier entry on how this is done. The technically inclined will note that the namespaces of the RDFa itself are not accessible in the XML parsing we used above, since they do not correspond to nodes or attributes, but appear only in the values of attributes. Fortunately, there are many excellent tools to extract this RDFa data, turning it into the XML formatted RDF triples we need. e can perform this using the Any23 API

download.file(paste("http://any23.org/rdfxml", "http://carlboettiger.info", 
    sep = "/"), "temp.xml")
doc <- xmlParse("temp.xml")

Which creates a beautiful RDF XML file of all linked data found in the entry.

doc

<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF xmlns:xhtml="http://www.w3.org/1999/xhtml/vocab#" xmlns:dcterms="http://purl.org/dc/terms/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about="http://www.carlboettiger.info/">
    <dcterms:title xml:lang="en">Carl Boettiger</dcterms:title>
    <xhtml:license rdf:resource="http://creativecommons.org/publicdomain/zero/1.0/"/>
    <dcterms:title xml:lang="en">Carl Boettiger</dcterms:title>
    <dcterms:creator xml:lang="en">Carl Boettiger</dcterms:creator>
    <dcterms:date xml:lang="en">2013-04-04T11:07:14-07:00</dcterms:date>
    <dcterms:format xml:lang="en">text/html</dcterms:format>
    <dcterms:language xml:lang="en">en</dcterms:language>
    <dcterms:identifier xml:lang="en">/index.html</dcterms:identifier>
    <dcterms:rights xml:lang="en">CC0</dcterms:rights>
    <dcterms:source xml:lang="en">Lab Notebook</dcterms:source>
    <dcterms:subject xml:lang="en">Ecology</dcterms:subject>
    <dcterms:type xml:lang="en">website</dcterms:type>
    <title xmlns="http://ogp.me/ns#" xml:lang="en">Carl Boettiger</title>
    <author xmlns="http://ogp.me/ns#" xml:lang="en">http://www.carlboettiger.info/index.html#me</author>
    <first_name xmlns="http://ogp.me/ns/profile#" xml:lang="en">Carl</first_name>
    <last_name xmlns="http://ogp.me/ns/profile#" xml:lang="en">Boettiger</last_name>
    <published_time xmlns="http://ogp.me/ns/article#" xml:lang="en">2013-04-04T11:07:14-07:00</published_time>
    <site_name xmlns="http://ogp.me/ns#" xml:lang="en">Lab Notebook</site_name>
    <url xmlns="http://ogp.me/ns#" xml:lang="en">http://www.carlboettiger.info/index.html</url>
    <type xmlns="http://ogp.me/ns#" xml:lang="en">website</type>
  </rdf:Description>
  <rdf:Description rdf:about="http://carlboettiger.info#me">
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
    <rdf:type rdf:resource="http://schema.org/Person#Person"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://www.carlboettiger.info/assets/img/carlboettiger.png">
    <depiction xmlns="http://xmlns.com/foaf/0.1/" xml:lang="en"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://carlboettiger.info">
    <homepage xmlns="http://xmlns.com/foaf/0.1/" rdf:resource="http://carlboettiger.info"/>
    <url xmlns="http://schema.org/Person#" rdf:resource="http://carlboettiger.info"/>
    <name xmlns="http://xmlns.com/foaf/0.1/" xml:lang="en">Carl Boettiger</name>
  </rdf:Description>
  <rdf:Description rdf:about="http://carlboettiger.info#me">
    <jobTitle xmlns="http://schema.org/Person#" xml:lang="en">a post-doctoral researcher</jobTitle>
  </rdf:Description>
  <rdf:Description rdf:nodeID="node17eprp1n4x899515">
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://carlboettiger.info#me">
    <knows xmlns="http://xmlns.com/foaf/0.1/" rdf:nodeID="node17eprp1n4x899515"/>
  </rdf:Description>
  <rdf:Description rdf:nodeID="node17eprp1n4x899515">
    <homepage xmlns="http://xmlns.com/foaf/0.1/" rdf:resource="http://users.soe.ucsc.edu/~msmangel/"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://carlboettiger.info#me">
    <knows xmlns="http://xmlns.com/foaf/0.1/" rdf:nodeID="node17eprp1n4x899515"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://users.soe.ucsc.edu/~msmangel/">
    <name xmlns="http://xmlns.com/foaf/0.1/" xml:lang="en">Marc Mangel</name>
  </rdf:Description>
  <rdf:Description rdf:nodeID="node17eprp1n4x899516">
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
    <homepage xmlns="http://xmlns.com/foaf/0.1/" rdf:resource="http://swfsc.noaa.gov/staff.aspx?&amp;id=17294"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://swfsc.noaa.gov/staff.aspx?&amp;id=17294">
    <name xmlns="http://xmlns.com/foaf/0.1/" xml:lang="en">Steve Munch</name>
  </rdf:Description>
  <rdf:Description rdf:about="http://carlboettiger.info#me">
    <affiliation xmlns="http://schema.org/Person#" rdf:resource="http://boe.ucsc.edu/~msmangel/CSTAR.html"/>
    <workplaceHomepage xmlns="http://xmlns.com/foaf/0.1/" rdf:resource="http://www.ucsc.edu/"/>
    <weblog xmlns="http://xmlns.com/foaf/0.1/" rdf:resource="http://www.carlboettiger.info/lab-notebook.html"/>
  </rdf:Description>
  <rdf:Description rdf:nodeID="node17eprp1n4x899517">
    <rdf:type rdf:resource="http://schema.org/PostalAddress"/>
    <address xmlns="http://schema.org/Person#" xml:lang="en">Center for Stock Assessment Research, 110 Shaffer Rd, Santa Cruz, CA 95050, USA</address>
    <streetAddress xmlns="http://schema.org/PostalAddress/" xml:lang="en">Center for Stock Assessment Research, 110 Shaffer Rd</streetAddress>
    <addressLocality xmlns="http://schema.org/PostalAddress/" xml:lang="en">Santa Cruz</addressLocality>
    <addressRegion xmlns="http://schema.org/PostalAddress/" xml:lang="en">CA</addressRegion>
    <postalCode xmlns="http://schema.org/PostalAddress/" xml:lang="en">95050</postalCode>
    <addressCountry xmlns="http://schema.org/PostalAddress/" xml:lang="en">USA</addressCountry>
  </rdf:Description>
  <rdf:Description rdf:about="https://orcid.org/0000-0002-1642-628X">
    <orcid xmlns="http://purl.org/spar/datacite/" rdf:resource="https://orcid.org/0000-0002-1642-628X"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://www.carlboettiger.info#me">
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
    <account xmlns="http://xmlns.com/foaf/0.1/" rdf:resource="http://www.cloudflare.com/email-protection#f497969b9180809d93b49399959d98da979b99"/>
    <account xmlns="http://xmlns.com/foaf/0.1/" rdf:resource="https://twitter.com/cboettig"/>
    <account xmlns="http://xmlns.com/foaf/0.1/" rdf:resource="https://github.com/cboettig"/>
  </rdf:Description>
  <rdf:Description rdf:about="http://www.carlboettiger.info/">
    <license xmlns="http://www.carlboettiger.info/" rdf:resource="http://creativecommons.org/publicdomain/zero/1.0/"/>
    <license xmlns="http://creativecommons.org/ns#" rdf:resource="http://creativecommons.org/publicdomain/zero/1.0/"/>
  </rdf:Description>
</rdf:RDF>

We can now explore this data using the XML tools illustrated above. The rigidity of the XML rather than HTML parsing and the use of namespaces gives us greater precision.

Further Examples Managing Tipping Points With Gp

2013-04-03T00:00:00-07:00

Monday Tuesday

2013-04-02T00:00:00-07:00

Writing

still working on non-parametric bayes draft today..

Reviewing

review sent to PRSB

Reading

Came across this rather nice summary of big data in Ecology through some buzz in the twittersphereHampton et. al. (2013) .
More fishery managment controversy with Halpern et. al. (2013) critiquing the ‘ocean food provision index’ from Nature last year, and a subset of that team replying Halpern et. al. (2013) . Seems to fall along the usual divisions of whether to use or ignore imperfect data; no doubt the truth lies somewhere in between.
Looks like Cavanaugh et. al. (2013) have a nice synchrony paper in kelp forests which I’m looking forward to reading.
Nonlinear forecasting strikes again in PNAS, Deyle et. al. (2013) Predicting climate effects on sardine with the usual suspects. May disentangle physical forcing from ecological dynamics.
Halpern et. al. (2013b) discuss the classic problem of optimization in conservation: balancing social equity, economic return and conservation needs.
ROpenSci is mentioned in a letter to the editor published in Ecotoxicology, (Schäfer et. al. 2013 ).
Kyle C. Cavanaugh, Bruce E. Kendall, David A. Siegel, Daniel C. Reed, Filipe Alberto, Jorge Assis, (2013) Synchrony in Dynamics of Giant Kelp Forests is Driven by Both Local Recruitment And Regional Environmental Controls. Ecology 94 499-509 10.1890/12-0268.1
E. R. Deyle, M. Fogarty, C.-h. Hsieh, L. Kaufman, A. D. MacCall, S. B. Munch, C. T. Perretti, H. Ye, G. Sugihara, (2013) Predicting Climate Effects on Pacific Sardine. Proceedings of The National Academy of Sciences 10.1073/pnas.1215506110
Benjamin S. Halpern, Steven D. Gaines, Kristin Kleisner, Catherine Longo, Daniel Pauly, Andrew A. Rosenberg, Jameal F. Samhouri, Dirk Zeller, (2013) Halpern et al. Reply. Nature 495 E7-E7 10.1038/nature11975
B. S. Halpern, C. J. Klein, C. J. Brown, M. Beger, H. S. Grantham, S. Mangubhai, M. Ruckelshaus, V. J. Tulloch, M. Watts, C. White, H. P. Possingham, (2013) Achieving The Triple Bottom Line in The Face of Inherent Trade-Offs Among Social Equity, Economic Return, And Conservation. Proceedings of The National Academy of Sciences 10.1073/pnas.1217689110
Stephanie E Hampton, Carly A Strasser, Joshua J Tewksbury, Wendy K Gram, Amber E Budden, Archer L Batcheller, Clifford S Duke, John H Porter, (2013) Big Data And The Future of Ecology. Frontiers in Ecology And The Environment 11 156-162 10.1890/120103
Ralf B. Schäfer, Mirco Bundschuh, Andreas Focks, Peter C. von der Ohe, (2013) to The Editor. Environmental Toxicology And Chemistry 32 734-735 10.1002/etc.2140

Research

2013-03-26T00:00:00-07:00

nonparametric-bayes

writing writing …

multiple-uncertainty

Fixed matlab implementation. (Some fun with matrix transposes in translating from R. #4) See:

Compare to

Matlab code shows the same pattern in which growth noise results in constant escapement, but either implementation or measurement noise cause deviations from this rule (most dramatic in the case of implementation uncertainty).

Notebook

Some feedback and ideas from Noam, such as landing pages #77 and inline intro #78. Also fixed history links and SHA links are now unique to their pages #67

Research

2013-03-25T00:00:00-07:00

Reading

(Schreiber & Killingback, 2013 )
(Warren, 2013 )

Notebook

Added SHA hashes. See notebook Issues #67 for details.
modification date. #74, also involved resolving page path robustly, #73
analytics on history button, layout #72
push dreamhost-based images to flickr See #75

prosecutor

Run with OU based data. Draw from sde package.

Packages

Knitcitations

Feature request for additional markup for page ranges #32

rfigshare

Handle pull request for rfigshare

code-tricks: notes on merging select files from pull request

Pull request included changes to files we don’t want to change. To address this, we separate fetch and merge with --no-commit. First, fetch the remote repository:

git fetch https://github.com/jcolomb/rfigshare master

The fetch is at FETCH_HEAD as it tells us in the console.

Now we merge the changes without committing them,

git merge --no-commit FETCH_HEAD

So the changes are all made, but not committed. We can simply unstage the changes we do not want. Since we only want one file, lets start by unstaging all commits

git reset --

So all changes are in our working directory as if we had just made them, but not told git to do anything yet. We can now drop any of the files we did not want to change back to their state at last commit (HEAD), for instance,

git checkout HEAD inst/examples/tutorial.Rmd

while staging the changes to the file we want changed:

git add R/fs_update.R

and finally commit and push the changes.

References

Sebastian J. Schreiber, Timothy P. Killingback, (2013) Spatial Heterogeneity Promotes Coexistence of Rock–Paper–Scissors Metacommunities. Theoretical Population Biology 10.1016/j.tpb.2013.02.004
Dan L. Warren, (2013) ‘Niche Modeling’: That Uncomfortable Sensation Means it’S Working. A Reply to Mcinerny And Etienne. Trends in Ecology & Evolution 28 10.1016/j.tree.2013.02.003

Research

2013-03-22T00:00:00-07:00

multiple-uncertainty

Fixed issues with translation to matlab and applyfun. Element-wise calls can use meshgrid arguments directly, and one can force functions to use element-wise operations with bxfun. So the following are equivalent: :

[X, Y] = meshgrid(x,y)
applyfun(@min, X, Y)

function out = element_min(x,y) bxfun(@min, x, y)
element_min(X, Y)

Meanwhile my challenge with additional arguments to applyfun can be circumvented with an inline function,

applyfun(@(x,y) ssum(x,y, additional_arg), X, Y)

In other developments, may have missed a rownormalization on growth noise.
Time to run and compare R and matlab codes and send to Jim.

prosecutors-fallacy

Check status of runs on farm. Appears that a type error blocked calculations of null distribution. Also, may need further threshold, a threshold of 220 for deviation occurs in 75% of (2000) runs (though depends on run length whether that is sufficiently rare). Threshold of 200 is reached in about 2%. re-running now.

tweaked my original code a bit to run the case with a single stable state and condition on observing large deviations, and see if the observations prior to the deviation show the ‘spurious’ increase in autocorrelation and variance observed in the original paper.

Looks like there is still a bias to me.

ropensci

write coversheet
Research Data Alliance launch. Interesting working groups.

Research

2013-03-20T00:00:00-07:00

pomp

Exploring pomp for examples tomorrow. See pomp-example

To access help on S4 methods
print method source code: getMethod("simulate", "pomp")
access help file on available methods: method?show("pomp")
pomp simulate method returns object of class pomp with data in object@data, unless user sets either/both of obs=TRUE and state=TRUE to return the observed and true state data (in list objects). Seems to be the only way to get the true state data out of the simulate routine (other than shortcutting the observation method rmeasure). Poor form to change return type based on input options, but oh well.

prosecutor’s fallacy

troubleshooting comment runs setup. Some replicates realize their extrema within the first 200 points, causing negative indexing (if not handled) and mismatched length of data objects (if not handled). Improve exception handling and restart runs.

nonparametric-bayes

Writing writing writing. see repo.

multiple-uncertainty

Troubleshooting the matlab translation

Cannot seem to pass an array in place of a variable to matlab’s (okay, actually octave’s) applyfun function:

function out = ssum(x,y, a)
  out = (x + y);
end
[X, Y] = meshgrid([1:5], [1:5])
b = arrayfun(@ssum, X, Y, 1)  %Works fine

But if the extra argument is a vector this fails.

b = arrayfun(@ssum, X, Y, [1:6])

Need a convenient way to generate probability densities from continuous distributions in which parameter values and values are restricted to a grid. Matlab translation of this didn’t work.

I want to get the probability density of value from uniform distribution between and . If is zero, I want to return unity if is equal to . The trick is that I want all values discretized to a grid, so P == mu really means if and fall in the same bin of the discrete grid.

I thought this function would do it:

    function out = snap_to_grid(x, grid)
      [v,i] = min(abs(grid-x));
      out = grid(i);  
    end
     
    %% generate various sources of noise, or delta fns if noise is zero
    function out = pdfn(P, mu, s)
      grid = [1:5]; % dumb grid, ideally an argument to pdfn but doesn't work
      if mu == 0
        out = (P == 0);
      elseif s > 0
        if mu > 0
          out = unifpdf(P, mu * (1 - s), mu * (1 + s));
        end
      else  % delta spike
        P = snap_to_grid(P, grid);
        mu = snap_to_grid(mu, grid);
        out = (P == mu);
      end
    end

But clearly I’ve gone wrong somewhere.

ropensci

final revisions, reread draft, close outstanding issues.
troubleshooting SSOAP example. Added checkCircular enables successful parsing, but now no luck generating functions:

library(SSOAP)
wsdl = processWSDL("http://www.nbnws.net/ws_3_5/GatewayWebService?wsdl", checkCircular=FALSE)
iface = genSOAPClientInterface(def = wsdl)

Prosecutor Fallacy As Large Deviations

2013-03-19T00:00:00-07:00

(Corresponds to commit 4bd8422107 in earlywarning repo)

For the individual-based simulation, the population dynamics are given by

\[\begin{align} \frac{dP(n,t)}{dt} &= b_{n-1} P(n-1,t) + d_{n+1}P(n+1,t) - (b_n+d_n) P(n,t), \\ b_n &= \frac{e K n^2}{n^2 + h^2}, \\ d_n &= e n + a, \end{align}\]

which is provided by the saddle_node_ibm model in populationdynamics.

For each of the warning signal statistics in question, we need to generate the distibution over all replicates and then over replicates which have been selected conditional on having experienced a crash.

We begin by running the simulation of the process for all replicates.

Load the required libraries

library(populationdynamics)
library(earlywarning)
library(reshape2)   # data manipulation
library(data.table) # data manipulation
library(ggplot2)    # graphics
library(snowfall)   # parallel

theme_publish <- theme_set(theme_bw(12))
theme_publish <- 
  theme_update(legend.key=theme_blank(),
        panel.grid.major=theme_blank(),panel.grid.minor=theme_blank(),
        plot.background=theme_blank(), legend.title=theme_blank())

Conditional distribution

Then we fix a set of paramaters we will use for the simulation function. Since we will simulate 20,000 replicates with 50,000 pts a piece, we can save memory by performing the conditional selection on the ones that cross a threshold we go along and disgard the others. (We will create a null distribution in which we ignore this conditional selection later).

threshold <- 250
select_crashes <- function(n){
  T<- 5000
  n_pts <- n
  pars = c(Xo = 500, e = 0.5, a = 180, K = 1000, h = 200,
    i = 0, Da = 0, Dt = 0, p = 2)
  sn <- saddle_node_ibm(pars, times=seq(0,T, length=n_pts), reps=1000)
  d <- dim(sn$x1)
  crashed <- which(sn$x1[d[1],] <= threshold)
# crashed <- which(sn$x1[d[1],]==0)
  sn$x1[,crashed] 
}

A few extra commands format the data into a table with columns of times, replicate id number, and population value at the given time.

sfInit(parallel=FALSE)
sfLibrary(populationdynamics)

sfExportAll()
examples <- sfLapply(1:20, function(i) select_crashes(50000))
dat <- melt(as.matrix(as.data.frame(examples, check.names=FALSE)))
names(dat) = c("time", "reps", "value")
levels(dat$reps) <- 1:length(levels(dat$reps)) # use numbers for reps

ggplot(subset(dat, reps %in% levels(dat$reps)[1:9])) + geom_line(aes(time, value)) + facet_wrap(~reps, scales="free")

plot of chunk testing

Zoom in on the relevant area of data near the crash

require(plyr)
zoom <- ddply(dat, "reps", function(X){
    tip <- min(which(X$value<threshold))
    index <- max(tip-200,1):tip
    data.frame(time=X$time[index], value=X$value[index])
    })

ggplot(subset(zoom, reps %in% levels(zoom$reps)[1:9])) + geom_line(aes(time, value)) + facet_wrap(~reps, scales="free")

plot of chunk example-trajectories

Compute model-based warning signals on all each of these.

dt <- data.table(subset(zoom, value>threshold))
var <- dt[, warningtrend(data.frame(time=time, value=value), window_var), by=reps]$V1
acor <- dt[, warningtrend(data.frame(time=time, value=value), window_autocorr), by=reps]$V1
dat <- melt(data.frame(Variance=var, Autocorrelation=acor))

Null distribution

To compare against the expected distribution of these statistics, we create another set of simulations without conditioning on having experienced a chance transition, on which we perform the identical analysis.

threshold <- 250
select_crashes <- function(n){
  T<- 5000
  n_pts <- n
  pars = c(Xo = 500, e = 0.5, a = 180, K = 1000, h = 200,
    i = 0, Da = 0, Dt = 0, p = 2)
  sn <- saddle_node_ibm(pars, times=seq(0,T, length=n_pts), reps=500)
  d <- dim(sn$x1)
  sn$x1[1:501,]
}

sfExportAll()
examples <-  sfLapply(1:10, function(i) select_crashes(50000))
nulldat <- melt(as.matrix(as.data.frame(examples, check.names=FALSE)))
nulldat <- melt(examples)
names(nulldat) = c("time", "reps", "value")
levels(nulldat$reps) <- 1:length(levels(dat$reps))

Zoom in on the relevant area of data near the crash

require(plyr)
nullzoom <- ddply(nulldat, "reps", function(X){
    data.frame(time=X$time, value=X$value)
    })

nulldt <- data.table(nullzoom)
nullvar <- nulldt[, warningtrend(data.frame(time=time, value=value), window_var), by=reps]$V1
nullacor <- nulldt[, warningtrend(data.frame(time=time, value=value), window_autocorr), by=reps]$V1
nulldat <- melt(data.frame(Variance=nullvar, Autocorrelation=nullacor))

ggplot(dat) + geom_histogram(aes(value, y=..density..), binwidth=0.3, alpha=.5) +
 facet_wrap(~variable) + xlim(c(-1, 1)) + 
 geom_density(data=nulldat, aes(value), adjust=2) + xlab("Kendall's tau") + theme_bw()

plot of chunk fig

Collected notes from past few days

2013-03-19T00:00:00-07:00

Friday

some ropensci coordination
Adapt basic science abstract and broader impacts statement
some writing on notebook features draft
nonparametric writing!

Monday

nonparametric-bayes writing. Combining my generic position piece (beyond-mechanistic-models.Rmd) with the technical nonparametric-bayes writeup (nonparametric-bayes.Rmd) to get a more general-audience paper.
ropensci reviewing
Note: annoying feature of knitr’s purl: appears to generate valid “external” knitr code, but includes all options in comment headers instead of just the chunk names. This prevents “external” inclusion from identifying these chunks currectly. tangle (option in knit) appears not to generate any header at all. Maybe I’m missing something.

Prosecutor’s Fallacy Comments

Further exchanges with John over prosecutors fallacy. Some scratch writing for a potential reply piece.

Our discussion highlighted two potential test cases that would further demonstrate (or refute) that the critical slowing down we observed is an artifact – our ability to detect it in cases that do not transition (e.g. population of all cases reaching a certain excursion) in the original dynamics, and our ability to detect it in scenarios that do not involve transition dynamics at all. Running these two cases as

comment.Rmd – repeat the analysis using a generic threshold deviation rather than only populations that have crashed.
beer.Rmd – repeat analysis in which there is only a single stable point.

ugh, looks like libraries on zero haven’t been kept up. Some delays before production code will be running.

Tuesday

More writing
Assemble material for applied math club. Going with particle filtering (Ionides et. al. 2006) and the pomp vignette
ropensci reviewing

Misc

Sounds like exciting stuff happening a the #btpdf2 conference

Notes

2013-03-14T00:00:00-07:00

Writing

Meeting with Steve over outline
Writing on nonparametric-bayes manuscript
A few notes on notebook-features draft

Reading

EasyABC by Jabot et. al. (2013). While it’s great that MEE publishes this stuff, this is unique in that the authors are updating and improving upon an existing package published by MEE the year before, Brook et. al. (2013). The authors of the recent manuscript have certainly made an important contribution which deserves the traditional recognition provided by a separate publication, but one must wonder why we need another package instead of building on the existing one. Is that necessary for the publication? If only there was a way to fork the original package, make their additions, issue a pull request instead… Could more research follow such a model?
The rise of MOOCs may be old news, but the Nature column has a pretty excellent write-up on them all the same Halpern et. al. (2012), including a discussion of the challenges they face (e.g. completion rates) and how they might address them. Only passingly addresses the challenges these pose for universities seeking to get on board – such as the substantial upfront costs in providing content in a different medium and structure (see this recent Inside HigherEd piece.
Nice column in Nature on job opportunities in risk management, Nelson (2013) which highlights the interdisciplinary nature of the work. A pity they find the greatest market for jobs of this type in reinsurance companies and the smallest in academia…
TREE debates the existence and relevance of tipping points in global context and Hughes et. al. (2013), seemingly on the grounds that the globe isn’t a very connected place.
In the same issue, the usual suspects emphasize the idea that transitions due to saddle-node bifurcations need not be fast relative to the timescale of management Brook et. al. (2013). The “fast” transition represents a return to equilibrium at the new laternative stable state. The authors remind us that this transient phase can take a long time (and amazingly failing to cite Hastings & Higgins (1994), Hastings (2001), or Hastings (2004) in the process).

References

Barry W. Brook, Erle C. Ellis, Michael P. Perring, Anson W. Mackay, Linus Blomqvist, (2013) Does The Terrestrial Biosphere Have Planetary Tipping Points?. Trends in Ecology & Evolution 10.1016/j.tree.2013.01.016
Benjamin S. Halpern, Catherine Longo, Darren Hardy, Karen L. McLeod, Jameal F. Samhouri, Steven K. Katona, Kristin Kleisner, Sarah E. Lester, Jennifer O’Leary, Marla Ranelletti, Andrew A. Rosenberg, Courtney Scarborough, Elizabeth R. Selig, Benjamin D. Best, Daniel R. Brumbaugh, F. Stuart Chapin, Larry B. Crowder, Kendra L. Daly, Scott C. Doney, Cristiane Elfes, Michael J. Fogarty, Steven D. Gaines, Kelsey I. Jacobsen, Leah Bunce Karrer, Heather M. Leslie, Elizabeth Neeley, Daniel Pauly, Stephen Polasky, Bud Ris, Kevin St Martin, Gregory S. Stone, U. Rashid Sumaila, Dirk Zeller, (2012) an Index to Assess The Health And Benefits of The Global Ocean. Nature 488 10.1038/nature11397
A. Hastings, K. Higgins, (1994) Persistence of Transients in Spatially Structured Ecological Models. Science 263 10.1126/science.263.5150.1133
Alan Hastings, (2001) Transient Dynamics And Persistence of Ecological Systems. Ecology Letters 4 10.1046/j.1461-0248.2001.00220.x
Alan Hastings, (2004) Transients: The Key to Long-Term Ecological Understanding?. Trends in Ecology & Evolution 19 10.1016/j.tree.2003.09.007
Terry P. Hughes, Cristina Linares, Vasilis Dakos, Ingrid A. van de Leemput, Egbert H. van Nes, (2013) Living Dangerously on Borrowed Time During Slow, Unrecognized Regime Shifts. Trends in Ecology & Evolution 28 10.1016/j.tree.2012.08.022
Franck Jabot, Thierry Faure, Nicolas Dumoulin, (2013) Easyabc: Performing Efficient Approximate Bayesian Computation Sampling Schemes Using R. Methods in Ecology And Evolution 10.1111/2041-210X.12050
Bryn Nelson, (2013) Natural Disasters: A Calculated Risk. Nature 495 10.1038/nj7440-271a

Notes

2013-03-13T00:00:00-07:00

Multiple-uncertainty

Created smaller version of table to simplify comparisons
Switched to stationary policy comparisons only. (Running out over longer time shifts total value a bit but doesn’t amplify scale of the differences much)
Larger noise amplifies the effects (compare 0.3 levels to 0.5 levels)
note that averaging over replicates gives rather consistent means. The differences between successive runs of the algorithm give results agreeing to the tenths place at least.

Example escapement, code

the distribution of profits by scenario

Tables of results

Note that columns represent the decision-maker’s beliefs about uncertainty and rows represent the true uncertainty present in the simulation.

	det	g	m	all
det	19.11	19.11	18.81	18.81
g	18.92	18.63	18.74	18.56
m	16.17	16.76	17.38	18.08
all	15.54	14.65	16.50	17.15

	det	g	m	all
det	0.00	0.00	0.00	0.00
g	4.41	4.31	3.59	3.28
m	3.10	2.96	1.60	1.12
all	5.06	4.89	4.20	4.59

logistic case

Overcompensatory density dependence is tough on stock. With given discounting (5%) and finite horizon (15 cycles) strategies act conservatively. Hmm, strange that deterministic case is equally effected..
running with much weaker r to reduce overcompensatory impacts…

Coding

Attempting to translate algorithm into Jim’s native tongue (matlab) for feedback. R version vs matlab not there yet…

Reviewing

Another review request, another review done.

Misc

With Alistair, playing around with this HMM EM algorithm gui11aume/HMMt. Also threw in a pull request with documentation. grr, took a bit to understand what was going on with that paper.

Notes

2013-03-12T00:00:00-07:00

Monday

multiple_uncertainty abstract
Meeting with Robin Snyder
application writing

Tuesday

Morning

applications
Coordinating summer conference travel
ropensci proposals
EML is not an ontology, not analgous to Dublin Core, etc. Just an XML schema. see OBOE for an ecological ontology. to explore later.

Afternoon

multiple-uncertainty

Revisiting multiple-uncertainty runs. Running smaller table example. Recoded script to allow arbitrary set of noise configurations to be consistently compared.
Some debugging. Shared-memory parallelization prohibitive at current memory requirements.
See: small table example (for issue #3) 04:48 pm 2013/03/12

nonparametric-bayes

Exploring options for approximate dynamic programming. For the moment, working through the basic ADP algorithm presented in Powell (2006).
in-progress notes: latest version version-stable link

Notes From The Week

2013-03-08T00:00:00-08:00

(Working from Galveston this week while at Louise’s conference, looking after little one. some progress below.)

Monday

Alan Skype meeting

Comment piece / reply
ews-review
Comments on TE review?
IARPA Conference, funding options?

Tasks

On Marc’s suggestion, writing comparison of pattern-based and mechanism based approaches to modeling, particularly in decision theory context. draft in nonparametric-bayes
Review edits and writing on ews-review

Tuesday

Kathy Skype

Decision theory in early warning signals?
NCEAS / SESYNC working group?
Long term lake monitoring data, including some sedimentary analysis (see Wang et. al. (2012)).
Send Kathy TREE articles: Polasky et. al. (2011) and Fischer et. al. (2009).

Tasks

Writing comparison of pattern-based and mechanism based approaches to modeling, particularly in decision theory context
Review edits and writing on ews-review

Wednesday

more of the same.

Friday

Finished and submitted ews-review.
See commit log and issues tracker for details.

Notebook: Semantics,

stemming-based search seems to perform poorly – i.e. cannot find phrases, weight terms in titles, etc, just most frequently occuring terms in a post. Trying lunr, loads full text into memory but should do so only on selecting search bar? See issue #7
Modified google-analytics plugin to make single call to the API see issue #66
Tuning RDFa, primarily on vita. #22 #63. Tuned FOAF referencing to common nodes. Note that rel always applys to links while property applies to the content of the tag (e.g. anchor text, span context). Use rel in span and div elements, etc, to apply to multiple links.

References

Joern Fischer, Garry D. Peterson, Toby A. Gardner, Line J. Gordon, Ioan Fazey, Thomas Elmqvist, Adam Felton, Carl Folke, Stephen Dovers, (2009) Integrating Resilience Thinking And Optimisation For Conservation. Trends in Ecology & Evolution 24 10.1016/j.tree.2009.03.020
Stephen Polasky, Stephen R. Carpenter, Carl Folke, Bonnie Keeler, (2011) Decision-Making Under Great Uncertainty: Environmental Management in an Era of Global Change. Trends in Ecology & Evolution 26 10.1016/j.tree.2011.04.007
Rong Wang, John A. Dearing, Peter G. Langdon, Enlou Zhang, Xiangdong Yang, Vasilis Dakos, Marten Scheffer, (2012) Flickering Gives Early Warning Signals of A Critical Transition to A Eutrophic Lake State. Nature 492 10.1038/nature11655

Delayed Release Archives

2013-03-02T00:00:00-08:00

Today I dug out a handful (37) posts from the past two years that were published as #delayed-release content (and have all been tagged as such to indicate they were not public at their publication date). Many of these were connected to the #warning-signals project but involved results that I or Alan weren’t ready to provide ahead of publication, as I discussed in a post when I first began marking some posts for delayed release, challenges with collaboration in open science These posts were created while on the Wordpress platform and had been marked private, so that they would be invisible to public browsing. Wordpress makes this easy to do, as I discuss in the earlier post. When I switched to a Jekyll platform, these posts all received the yaml header published: false and were also excluded from the Github repository until I could revisit and decide which to release. Managing these posts for release in the Jekyll framework just requires a little command-line fu, grep -l 'published: false' *.md grabs the relevant posts for inspection and release.

The notebook is currently back to an ONS All-content, immediate, as I have been able to okay the practice with all current major collaborations. (Sure, there’s always stuff that doesn’t make it into the notebook or is not appropriate for the notebook – the most common case of the latter being peer reviews I write for various journals. Though I make a practice of signing my reviews, I believe they still represent private communication unless the authors, editors and journals were to agree to publishing them.) All of these posts would have been released sooner but for finding a minute to remember to do so – perhaps a cautionary example for the delayed release practice.

Looking back on these posts, I probably would not have delayed most of them today, as I have gotten more comfortable not only with the open notebook, but with discussing the practice with collaborators. My own feelings of not wanting to push my adviser in a discussion of the notebook was as much responsible for these delays as other factors that I discuss in the earlier post. One key thing I wondered about in writing delayed-release posts was whether I would (perhaps subconsciously) write in a different or more candid style than I do for my regular notes. I was glad not to notice any glaring examples (though perhaps a diligent or very bored reader might find some!) of expletives and insulting commentary in the delayed posts. I do notice the overall tone and volume of the textual content changes slightly. However, that trend is actually much more dominant in comparing my very early posts (mostly from the OWW days), which were quite discursive, to the current posts which tend to be much more terse.

So have a look at the #delayed-release posts and let me know what you think!

Notes

2013-02-28T00:00:00-08:00

Meeting with Marc

Selection of problem: consider simpler models only. Sep through complexity hierarchy at the beginning:

classical approaches are to fix relationship of stock recruitment.
parameter uncertainty
Gaussian processes for model uncertainty

The Discussion section can deal with model uncertainty model choice approaches.

A method where we don’t have to assume the stock recuritment relationship
How well we can do with that.
How much data we need.

discussion of mathematical presentation, clarity

Misc

Finished review. I should stop signing reviews, then I could write more careless quick impressions.
discussed potential of a GSOC proposal for an R NEXML parser
requested to consider presenting at ISI Conference session on early warning.

some reading on semantics

David Shotton and Tanya Gray have release a CiTO tool for adding semantic data to many major publication platforms, taking a clever approach to annotation that I’ve never considered – croudsourcing to readers (davidshotton, 2013). The tool isn’t perfect but is very intuitive. The clever part is that all annotations the reader adds are collated in a database and cand be retrieved through their API. Not sure if many readers will make use of this, but time will tell. Meanwhile authors or publishers might pay attention.

In that vein, David has also written a call of Ten Next Steps for publishers, (davidshotton, 2013b). Some may be redundant, but certainly shows how publishers could go a lot further. If only there was more incentive for publishers to innovate like this instead of crafting clever and questionable strategies to boost impact factors.

Reading

Very interesting paper on implementing/teaching nonlinear model inference to ecologists out today (Bolker et. al. 2013), and the code and data for each of the examples they discuss. Lessons in the mansucript may be familiar to many, but should still be taught more widely in ecology. Should be useful for teaching this stuff in graduate courses too.

Benjamin M. Bolker, Beth Gardner, Mark Maunder, Casper W. Berg, Mollie Brooks, Liza Comita, Elizabeth Crone, Sarah Cubaynes, Trevor Davies, Perry de Valpine, Jessica Ford, Olivier Gimenez, Marc Kery, Eun Jung Kim, Cleridy Lennert-Cody, Arni Magnusson, Steve Martell, John Nash, Anders Nielsen, Jim Regetz, Hans Skaug, Elise Zipkin, (2013) Strategies For Fitting Nonlinear Ecological Models in R, ad Model Builder, And Bugs. Methods in Ecology And Evolution 10.1111/2041-210X.12044
davidshotton, (2013) CiTO Reference Annotation Tools. Semantic Publishing http://semanticpublishing.wordpress.com/2013/02/26/cito-tools/
davidshotton, (2013) Ten next steps for semantic authors and publishers. Semantic Publishing http://semanticpublishing.wordpress.com/2013/02/26/ten-next-steps/

Notes

2013-02-27T00:00:00-08:00

Reviewing

arg, reviewing. so much time.

Seminar

Kate Richerson gave a fanstastic presentation to mega group yesterday showing her integration of behavior and dispersal in krill dynamics. She derives the krill’s behavioral movement patterns through a stochastic dynamic programming solution rather than just dictate it to the dispersal model directly.

Reading

Provicative stuff in PNAS Perretti et. al. (2013). It would be nice to see the comparison against the generating model. The forecasting errors of the fully Bayesian inference using the correct model shouldn’t necessary be small, but they should be just as big as the Bayesian model expects them to be (e.g. over replicates, the Bayesian forecast error reflects the widths of the posteriors (modulo some statement about priors)).

My worry with any approach doing better than the expected error is that it does so by generating biased estimation – e.g. something in the forecasting method, rather than in the data, happens to be biased towards the ‘right answer’. In this example I don’t think that is the case.

We already know that the modes of the posteriors don’t recover the correct model parameters of the generating model from an MCMC of such a complex model with limited data (though perhaps it is good to remind folks of this!), so it is not surprising that a heuristic approach can outperform this in cases.

It certainly is good to wrestle with these examples in any case. Steve’s one of my post-doc advisors now so no doubt I’m biased. Noam also pointed out this earlier article in Ecology Perretti et. al. (2012).

In seminar, Marc mentioned the great paper by Wiedenmann et. al. (2011) that uses the comparison of the optimal foraging strategy evolved by whales in their evolutionary environment to how that strategy performs today (similar spirit to the kinds of comparisons I am interested in with #multiple-uncertainty project.

Reading an old Schaffer paper Schaffer (1984).

Though Nature says the National Academy of Sciences needs an overhaul to stay relevant. Probably true but they still do some excellent reports that could deserve more eyes, like this on model fitting (). We could do more VVUQ in ecology and evolution, and the value of presenting benchmark examples of this in simple (trivial) cases cannot be overstated, e.g. Fig 5.2.1.
GP methods in Biometrika Banerjee et. al. (2012)
Luminaries of evolution forecast the future of the field, with an eye towards data and semantics in Losos et. al. (2013)

Misc

wrote a quick CSL format for knitcitations. (Generate inline citations using the copy function in Mendeley library. A bit silly since normally that would already generate the entire bibliographic citation.)
Should really make a CSL format for Mendeley to paste in a HTML citation (with full metadata in tooltip). Might then use an existing format to copy-paste bib info at end.

bibliography()

A. Banerjee, D. B. Dunson, S. T. Tokdar, (2012) Efficient Gaussian Process Regression For Large Datasets. Biometrika 100 10.1093/biomet/ass068
Jonathan B. Losos, Stevan J. Arnold, Gill Bejerano, E. D. Brodie, David Hibbett, Hopi E. Hoekstra, David P. Mindell, Antónia Monteiro, Craig Moritz, H. Allen Orr, Dmitri A. Petrov, Susanne S. Renner, Robert E. Ricklefs, Pamela S. Soltis, Thomas L. Turner, (2013) Evolutionary Biology For The 21st Century. Plos Biology 11 10.1371/journal.pbio.1001466
Charles Thomas Perretti, George Sugihara, Stephan B. Munch, (2012) Nonparametric Forecasting Outperforms Parametric Methods For A Simulated Multi-Species System. Ecology 10.1890/12-0904.1
C. T. Perretti, S. B. Munch, G. Sugihara, (2013) Model-Free Forecasting Outperforms The Correct Mechanistic Model For Simulated And Experimental Data. Proceedings of The National Academy of Sciences 10.1073/pnas.1216076110
William M. Schaffer, (1984) Stretching And Folding in Lynx Fur Returns: Evidence For A Strange Attractor in Nature?. The American Naturalist 124 10.1086/284318
John Wiedenmann, Katherine A. Cresswell, Jeremy Goldbogen, Jean Potvin, Marc Mangel, (2011) Exploring The Effects of Reductions in Krill Biomass in The Southern Ocean on Blue Whales Using A State-Dependent Foraging Model. Ecological Modelling 222 10.1016/j.ecolmodel.2011.07.013
Assessing the Reliability of Complex Models: Mathematical and Statistical Foundations of Verification, Validation, and Uncertainty Quantification. http://www.nap.edu/catalog.php?record_id=13395

Semantic Citations For The Notebook And Knitr

2013-02-22T00:00:00-08:00

I have on ocassion been exploring the use of semantic markup in the notebook. In this post I illustrate how I am handling semantic citations. One of the more intriguing ideas is the ability to add semantic meaning to citations through the CITO ontology of Shotton (2010). Citation counts form a central part of academic discourse, but contain very little information regarding the reason for the citation. Most notably, ‘negative’ citations refuting a claim carry just the same weight as those confirming or relying upon a claim. Given the scale and expansion of academic literature, it is rarely reasonable to explore this citation graph manually. CITO provides a language for embedding the meaning of the citation, such as “discusses”, “refutes”, or “usesMethodIn”, to the citation. (For instance, my earlier citation to Shotton identifies itself as “usesMethodIn”, as I will explain).

The main barrier to this approach is a lack of adoption. One of the primary concerns is the burden it places on authors of adding the extra data. On one hand, authors already bother formatting and reformatting layout, spelling, and reference order to the arcane specifications of different journals, which suggests authors can be persuaded to do some pretty tedious tasks if the publishers would require it. After all, the task of adding citations is already much easier than it was in the days of paper journals. Still, it is much simpler to remove a tedious requirement than to add a new one. My hope is that intelligent tools can simplify this process, as they already have with other aspects of managing citations, and encourage the use of CITO. In this spirit, I have recently started trying to consistently use the CITO ontology in my notebook entries as a test case, using some tools of my own design.

Semantics in knitcitations

Several months ago I created the R package knitcitations to provide a citation platform for knitr dynamic documents, which provide executable code and automatic inclusion of results inside plain-text (markdown) descriptions. I write most of my research scripts and many of my notebook entries in this manner. The package can generate citations by DOI, circumventing the need for maintaining bibtex or similar database of citation information, using commands such as

citet("10.1186/2041-1480-1-S1-S6")

Extending the package to support CITO was rather straight forward. Using the latest version of knitcitations, one can generate in-line citations with CITO semantics simply by passing the reason for the citation as well, such as

citet("10.1186/2041-1480-1-S1-S6", cito="usesMethodIn")

which generates the following HTML:

<a href='http://dx.doi.org/10.1186/2041-1480-1-S1-S6' property='http://purl.org/spar/cito/usesMethodIn' >Shotton (2010)</a>

This provides a convient platform to generate semantic citations in this lab notebook. As before, knitcitations will also generate a complete reference list at the end of the document by calling the bibliography function at the end.

Semantic overkill?

It is possible to add far more semantic data to this reference list at the end of an article. Invisible semantic markup can identify to a machine what value corresponds to the volume number or issue number, or journal name, e,g, using the BIBO ontology. I have added support for ths kind of markup to knitcitations as well, and several of my posts provide examples. The raw markup looks like this:

<div prefix="dc: http://purl.org/dc/terms/,
                      bibo: http://purl.org/ontology/bibo/,
                      foaf: http://xmlns.com/foaf/spec/,
                      biro: http://purl.org/spar/biro/"
        rel="http://purl.org/spar/biro/ReferenceList"> <ul class='bibliography'> 
<li> <span property="dc:title">Fisheries: Does Catch Reflect Abundance?.</span> <span property="dc:creator"> <span property="foaf:givenName">Daniel</span> <span property="foaf:familyName">Pauly</span>, </span><span property="dc:creator"> <span property="foaf:givenName">Ray</span> <span property="foaf:familyName">Hilborn</span>, </span><span property="dc:creator"> <span property="foaf:givenName">Trevor A.</span> <span property="foaf:familyName">Branch</span>, </span>  (<span property="dc:date">2013</span>)  <span rel="http://purl.org/dc/terms/isPartOf" 
                            resource="[http://purl.org/dc/terms/journal]">
                        <span property="http://purl.org/dc/terms/title"
                                content=" Nature ">
                        </span>
                          <span property="bibo:shortTitle"> Nature </span>
               </span>  <span property="bibo:volume">494</span>    <a property="bibo:doi" href="http://dx.doi.org/10.1038/494303a">10.1038/494303a</a> </li>
<li> <span property="dc:title">Net Gains.</span> <span property="dc:creator"> <span property="foaf:givenName">unknown</span> <span property="foaf:familyName">unknown</span>, </span>  (<span property="dc:date">2013</span>)  <span rel="http://purl.org/dc/terms/isPartOf" 
                            resource="[http://purl.org/dc/terms/journal]">
                        <span property="http://purl.org/dc/terms/title"
                                content=" Nature ">
                        </span>
                          <span property="bibo:shortTitle"> Nature </span>
               </span>  <span property="bibo:volume">494</span>    <a property="bibo:doi" href="http://dx.doi.org/10.1038/494282a">10.1038/494282a</a> </li>
 </ul>
</div>

However, I have since decided that such markup is largely overkill. The DOI uniquely identifies the publication already, and allows us to programmatically retrieve the rest of the data (title, authors, journal, etc) from semantically identified XML by querying against services such as CrossRef. This is the essential concept of linked data, by which both source and referer are enriched.

Moreover, DOIs follows a specific construction that lets us reliably identify them in plain text using regular expressions, making any futher semantics to declare that we are citing the article mostly irrelevant. This is convient for identifying all citations appearing in the notebook without any markup. The CITO example above has the advantage of providing a link and associating the DOI with the reason for the citation, by virtue of being inside the same html anchor element.

Replacing the reference list?

If we are not going to semantically mark up the reference list, we could consider abolishing the reference list all together. After all, as a tool for the digital reader the concept is rather vestigal – I hate losing my place by scrolling to the end of an article just to see to what reference number 7 refers. With the method shown thus far, the reader can open the link to access this information, but that still interrupts the flow of reading. The digitally native solution is a mouse-over or tooltip effect that displays this information, as many professional publishers already use in their HTML versions.

Once again, this is straight forward to support using the knitcitations package, at least for sites that include the popular bootstrap javascript libraries, such as this notebook. I have added an option to the in-text citation functions to provide such tooltips in a span element, such that calling the command

<span class='showtooltip' title='Shotton D (2010). "Cito, The Citation Typing Ontology." _Journal of
Biomedical Semantics_, *1*. ISSN 2041-1480, <URL:
http://dx.doi.org/10.1186/2041-1480-1-S1-S6>.'><a href='http://dx.doi.org/10.1186/2041-1480-1-S1-S6' property='http://purl.org/spar/cito/usesMethodIn' >Shotton (2010)</a></span>

This behavior can be toggled on by calling

cite_options(tooltip=TRUE)

after loading the knitcitations library. EDIT: Note that this requires the javascript trigger on the class showtooltip, which can be done by adding this to your header:

    <script type="text/javascript">
      $(document).ready(function (){
        $(".showtooltip").tooltip();
      });
    </script>

Citing without DOIs

Not all the literature we may wish to cite includes DOIs, such as arXiv preprints, Wikipedia pages, or other academic blogs. Even when a DOI is present it is not always trivial to locate. With version 0.4-0, knitcitations can produce citations given any URL using the Greycite API (Lord, 2012). For instance, this citation is created with the command citep("http://knowledgeblog.org/greycite", cito="usesMethodIn").

Notes

2013-02-21T00:00:00-08:00

Applied Math Club meeting

(See reading material from Tuesday.

Start with stable age distribtution (indep of stock-recrutiment)
Consider the Entropy of that probability distribution, as function of mortality, by max age
Now consider the same for fecundity: the fraction/density of the population von Bertalanffy growth (fecundity)
effective mortality rate - mortality rate in an unfished population, distributed with same entropy as the fished population,
Kullback-Leibler distance to measure fishing influence?
Mutual information and the design of surveys?
(information in catch data?)
Bonsall: Information content of global dynamics from individual patch dynamics?

Misc

NCEAS Summer Institute for data synthesis. Intriguing.

Reading

Is there information in catch data: a comment piece highlighting the debate with Pauly on the affermative and Hilborn & Branch on the negative, (Pauly et. al. 2013) and editoral commentary (unknown, 2013).
Looking over this approximate dynamic programming (ADP) paper in MEE, Schapaugh et. al. (2012). Possibly promising. I need to map out the space of ADP approaches and advantages/disadvantages a little more to get a better impression.
Schapaugh et. al. (2012): A new method for ADP? Or maybe not so new? see: , Towards a unified theory of state abstraction for MDPs.
Cute analysis on malaria as branching process in policy forum piece in Science (with Alex’s mentor) Chiyaka et. al. (2013).
And following up on today’s chats about information theory, a neat paper from Marc Frank (2012)
and making the rounds on the blogosphere bashing MAXENT, Renner & Warton (2013)

The Stability of Malaria Elimination. C. Chiyaka, A. J. Tatem, J. M. Cohen, P. W. Gething, G. Johnston, R. Gosling, R. Laxminarayan, S. I. Hay, D. L. Smith, (2013) Science 339 10.1126/science.1229509
Natural Selection. V. How to Read The Fundamental Equations of Evolutionary Change in Terms of Information Theory. S. A. Frank, (2012) Journal of Evolutionary Biology 25 10.1111/jeb.12010
Fisheries: Does Catch Reflect Abundance?. Daniel Pauly, Ray Hilborn, Trevor A. Branch, (2013) Nature 494 10.1038/494303a
Equivalence of Maxent And Poisson Point Process Models For Species Distribution Modeling in Ecology. Ian W. Renner, David I. Warton, (2013) Biometrics 10.1111/j.1541-0420.2012.01824.x
A Simple Method For Dealing With Large State Spaces. Adam W. Schapaugh, Andrew J. Tyre, Robert Freckleton, (2012) Methods in Ecology And Evolution 3 10.1111/j.2041-210X.2012.00242.x
. ()
Net Gains. unknown unknown, (2013) Nature 494 10.1038/494282a

NULL

Seminar Notes

2013-02-20T00:00:00-08:00

Leo Polansky Seminar

(all notes cover published results only)

Wavelet analysis does a nice trick of seeing frequency-domain spectra change over time, rather than being entirely in frequency space or entirely in time space.
higher rank elephants decrease their movement more in dry, resource-poor season than low rank. Guarding the good territory? I wonder if this is a stochastic effect - asset protection, where you take more risk when you have low assets than when you have higher assets, a feature that would dissappear in a deterministic environment.
Nice hidden markov model analysis from the mhsmm package.
movement time series – high impact public data or closely guarded secret? (top secret)

Michael Bonsall

(Seminar to the Mangel megagroup meeting)

uncertainty and variability in population dynamics. Mike uses the very words I’ve used to describe my interests: dynamic, nonlinear, and noisy.

Blasocyst Dynamics

Development of mammalian blastocyst much more messy/complicated than development in other taxa. heterogeneity in timing
Begins with ODE model with time lags (to avoid poisson waiting times). Then add a distribution over the time lag.
Invasion analysis of cell types differing in time. Widening distribution of time lags increases fitness. Interesting! But intuition/mechanism unclear
Now add process noise, observation error and priors on parameters to compare model to data. Variability can be explained entirely by differences in rates. (as estimated from a particle filter). Identifiability?

Bruchid Dynamics

Spatial dynamics with beatles and their parasitic wasps.
Beautiful microcosm setup that follows Nicholson-Bailey dynamics.
Within patch dynamics show regimes.
Consider stochastic birth-death-diffusion model
Stochastic focusing / demographic heterogeneity observed

Bipolar disorder dynamics

(Inductive example)
EM algorithm for missing values in time lag model

Other activities today

working on nominations
ropensci sustainability plan (see email threads),
also see the Conceptualizing an Institute for Sustainable Earth and Environmental Software (ISEES), http://isees.nceas.ucsb.edu/
friendly review phylolm package and preprint
summarizing reimbursement form
updated issues lists for notebook
Bonsall speaker dinner

Notes

2013-02-19T00:00:00-08:00

Fixed notebook compile error, and adjusted timeouts on pageviews / google-analytics API
Some minor fixes in knitcitations (thanks to another user bug report). NEWS
Accepted TE review request; better finish up ENMO review soon.
Generating data tables for published figures for Dryad (from prosecutor’s fallacy). Data in Dryad

Applied Math club reading

with Marc and Mike Bonsall on Thurs, reading:

Strevens, C. M. J. & Bonsall, M. B. 2011 Density-dependent population dynamics and dispersal in heterogeneous metapopulations. The Journal of animal ecology 80, 282–93. (doi:10.1111/j.1365-2656.2010.01768.x)
Cover, T. M. & Thomas, J. A. 2006 Elements of Information Theory. Second. Hoboken, NJ, USA: John Wiley & Sons, Inc.
Liepe, J., Filippi, S., Komorowski, M. & Stumpf, M. P. H. 2013 Maximizing the Information Content of Experiments in Systems Biology. PLoS Computational Biology 9, e1002888. (doi:10.1371/journal.pcbi.1002888)

Reading

Silk, H., Demirel, G., Homer, M. & Gross, T. 2013 Exploring network dynamics with a mathematical triple jump. (Arxiv)
Lade, S. J., Tavoni, A., Levin, S. A. & No, W. P. 2013 Regime shifts in a social-ecological system Centre for Climate Change Economics and Policy. preprint
Torney, C. J., Levin, S. a. & Couzin, I. D. 2013 Decision Accuracy and the Role of Spatial Interaction in Opinion Dynamics. Journal of Statistical Physics (doi:10.1007/s10955-013-0700-5)
Wang, S. & Campbell, B. C. 2013 Mr. Bayes Goes to Washington. Science 339, 758–759. (doi:10.1126/science.1232290)
Kiernan, V. 2003 Diffusion of News about Research. Science Communication 25, 3–13. (doi:10.1177/1075547003255297) Interesting one of few papers discussing the role coverage in popular media has on citations

Notes

2013-02-14T00:00:00-08:00

Research

consolidate and revise EWS text(?)
replies to Luke and Brian on model choice

Notebook meta

fix google-analytics plugin (still needs to get toggle working?)
tests of notebook-parsing.Rmd. New errors on ; somehow in straight XML parsing, ignoring for the moment.
Opened a few new issues around improving site semantics. (#59, #60)

ropensci

Resubmit rfishbase – silently load to local, not global namespace. Updated data cache (and ASCII-ify and compress).
Also replaced data loading with system.file("data", "fishbase.rda", package="fishbase"). ASCII-only .rda is very annoying. Should work around this with alternative data file?
Resubmit rfigshare – punt on 3.0.0 compatible vignette for the moment
knitcitations: Fix the stupid disambiguation for inline citations (took way too long).

reading

Brandt, G. & Merico, A. 2013 Tipping points and user-resource system collapse in a simple model of evolutionary dynamics. Ecological Complexity (doi:10.1016/j.ecocom.2012.12.003)
Cuddington, K., Fortin, M. & Gerber, L. 2013 Process-based models are required to manage ecological systems in a changing world. Ecosphere 4, 1–12. (doi:10.1890/ES12-00178.1)
Reiner, R. C. et al. 2013 A systematic review of mathematical models of mosquito-borne pathogen transmission: 1970-2010. Journal of The Royal Society Interface 10, 20120921–20120921. (doi:10.1098/rsif.2012.0921)

Notes

2013-02-13T00:00:00-08:00

ropensci

rfishbase: 0.1-1 released to CRAN: bugfix updating server address. (Broke updateCache()). See NEWS for details.
rfigshare: 0.2-4 released to CRAN: added method to remove attached files (issue #42), see NEWS for details

Misc

My reply to John on Prosecutor’s Fallacy piece.
Thinking about a better way to provide convenient, stable access to code and data. Should provide csv versions of data.table outputs used to create each of the figures. Could by Dryad archived.

Reading

Deyle, E. R. & Sugihara, G. 2011 Generalized theorems for nonlinear state space reconstruction. PloS one 6, e18295. (doi:10.1371/journal.pone.0018295)
Brock, W., Xepapadeas, A. & Yannacopoulos, A. 2013 Robust Control of a Spatially Distributed Commercial Fishery. (preprint)
Costello, C., Ovando, D., Hilborn, R., Gaines, S. D., Deschenes, O. & Lester, S. E. 2012 Status and solutions for the world’s unassessed fisheries. Science (New York, N.Y.) 338, 517–20. (doi:10.1126/science.1223389)

Notes

2013-02-12T00:00:00-08:00

Writing

Working on nonparametric-bayes.

Reading

Interesting PNAS comment from Doney & Sailley, (2013) (“When an ecological regime shift is really just stochastic noise”) on the Di Lorenzo & Ohman, (2013) piece in the same issue, showing that time integrated zooplankton data correlates closely with environmental drivers: in this case, the pacific decadal oscillation (PDO) index, which would be missed by the traditional approach of direct correlations. This refutes the previous explanation of such shifts observed in the krill populations as being driven by nonlinear ecological interactions, rather than merely linearly tracking the environment ((Hare & Mantua, 2000)). While there is some mention of how this may complicate traditional assumptions of warning signals, they do not remark on the obvious converse that the integral data may be more informative of detecting such shifts (whether or not they are tracking some abiotic dynamics).

In a related vein in the same issue is Liu et. al. (2013), discussing their algorithm for inferring complex dynamics from limited observations. Seems to do a poor job of putting the method into context of the suite of related approaches to addressing this problem (hidden Markov models, POMDP, etc), let alone directly comparing to them, which makes for a somewhat frustrating read. Still, could be a gem in the method I haven’t had a chance to dig into.

ropensci

Example of caching for authentication
Finalize scoring, replies to challenge participants

References

From The Cover: A Double-Integration Hypothesis to Explain Ocean Ecosystem Response to Climate Forcing. E. Di Lorenzo, M. D. Ohman, (2013) Proceedings of The National Academy of Sciences 110 10.1073/pnas.1218022110
When an Ecological Regime Shift is Really Just Stochastic Noise. S. C. Doney, S. F. Sailley, (2013) Proceedings of The National Academy of Sciences 110 10.1073/pnas.1222736110
Observability of Complex Systems. Y.-Y. Liu, J.-J. Slotine, A.-L. Barabasi, (2013) Proceedings of The National Academy of Sciences 110 10.1073/pnas.1215508110
Empirical Evidence For North Pacific Regime Shifts in 1977 And 1989. Steven R Hare, Nathan J Mantua, (2000) Progress in Oceanography 47 10.1016/S0079-6611(00)00033-1

Ews Review Paper Chat

2013-02-11T00:00:00-08:00

Skype with Alan and Noam on EWS

Other

Some updates to knitcitations addressing special cases.
Re-organized methods section to nonparametric-bayes manuscript.

Updates Across Various Projects

2013-02-04T00:00:00-08:00

At home after birth of Edward since Jan 19th, leaving little time for research. Assembled notes from a few side projects I’ve moved forward in the wee hours. Covers the past several days.

XML and RDFa parsing

I have been using the lab notebook to explore technology for scientific communication, including making use of semantic and linked data concepts. To illustrate some of the potential these tools can offer I’ve started working on a short post with examples parsing and exploring the XML and RDFa data in the notebook. Still forthcoming. The any23 API provides a great tool for this (particularly RDFa to RDF XML, where we can then take full advantage of xpath tools on the semantics).

Other notebook updates

MathML: Can have Pandoc render source in MathML and MathJax will still perform the HTML rendering in all browsers. Unfortunately, certain commands seem not to parse in this mode that do parse in Pandoc mathjax mode (e.g. those with operatorname in 2012-11-01 post).

Updated timeout setting on twitter plugin. 0cb7a11

PDG Control Follow-up

Skype with Paul for follow-up on third working group meeting. Key steps forward on policy costs:

All comparisons should be with reference to total value/profit of the strategy, rather than induced cost. Then compare against induced values directly. (Issue #35)
Quadratic costs should appear in supplement only (Issue #45)
Real-world context framing worked out. (Issue #39)

These and further details are listed in issues tracker for the project.

Updates to knitcitations

v0.3-3

Added a new display option, bibliography('markdown')
fixed BUG in the printing of the text-based name in bibliography for bibstyle-based calls

v0.3-2

Semantic citations now enabled. bibliography can print with method ‘rdfa’ to provide RDFa enhanced HTML bibliography data, and citations can take cito arguments as an option. (Closes issues #16 and #17)
bibliography now takes more options to match the bibstyle and print.bibentry functions.
citet and citep take an optional argument for formating the citation, defaulting to authoryear format. Future versions may move this into an option.
newbib() function initializes and clears existing cache files.
README.md updated with semantic examples and more background text (generated by inst/examples/README.Rmd).
Some basic unit tests built in. Could use more, but meanwhile closes #8.
Test cases work with knit2html button in RStudio, closes #14

v0.3-1

Switched to handling citation data using environment rather than an external bibfile. This does not refer to the handling of user-provided bibtex files as input data, which are supported as before, but changes only the backend management of that data. The external bibtex file method can be switched back on by setting options("bibtex_data" = TRUE).
There is no need to call print(bibliography, "html"), one can simply call bibliography() or bibliography("html"). Sort is still an option, other options that were for internal use (debug, remove_duplicates) are removed as they are no longer necessary.
Several new functions have been added to handle the new methods.

v0.2-0

Fix vignette error that prevents knitr 1.0 submission
Fixed errors on installing package and lack of entries in bibliography
Semantic print option added but still in early testing

Reading

O’Dwyer, J. P. & Green, J. L. 2010 Field theory for biogeography: a spatially explicit model for predicting patterns of biodiversity. Ecology letters 13, 87–95. (doi:10.1111/j.1461-0248.2009.01404.x)
Stewart, E. C. A., Almes, G. T., Mccaulay, D. S. & Bradley, C. 2009 Cyberinfrastructure Software Sustainability and Reusability : Report from an NSF-funded workshop Report from an NSF-funded workshop held 27 and 28 March 2009.
Ingram, T. & Mahler, D. L. 2013 SURFACE: detecting convergent evolution from comparative data by fitting Ornstein-Uhlenbeck models with stepwise AIC. Methods in Ecology and Evolution , 1–25. (doi:10.1111/2041-210X.12034)
Shelton, A. & Munch, S. 2012 Maternal age, fecundity, egg quality, and recruitment: linking stock structure to recruitment using an age-structured Ricker model. Canadian Journal of … 1641, 1631–1641. (doi:10.1139/F2012-082)
Regan, T. J., Chadès, I. & Possingham, H. P. 2011 Optimally managing under imperfect detection: a method for plant invasions. Journal of Applied Ecology 48, 76–85. (doi:10.1111/j.1365-2664.2010.01915.x)
Gewin, V. 2013 Turning point: Carl Boettiger. Nature 493, 711–711. (doi:10.1038/nj7434-711a)
Nicol, S. & Chadès, I. 2012 Which states matter? An application of an intelligent discretization method to solve a continuous POMDP in conservation biology. PloS one 7, e28993. (doi:10.1371/journal.pone.0028993)
McDonald-Madden, E., Chadès, I., McCarthy, M. a, Linkie, M. & Possingham, H. P. 2011 Allocating conservation resources between areas where persistence of a species is uncertain. Ecological applications : a publication of the Ecological Society of America 21, 844–58.
Krell 2011 The U.S. Department of Energy Computational Science Graduate Fellowship 1991-‐2011: Recipient Outcomes and Programmatic Impacts.
Dougherty, D. T., Hilborn, R., Punt, A. E. & Stewart, I. J. 2013 Modeling co-occurring species : a simulation study on the effects of spatial scale for setting management targets. 8, 1–8.
Chadès, I., Martin, T. G., Nicol, S., Burgman, M. a, Possingham, H. P. & Buckley, Y. M. 2011 General rules for managing and surveying networks of pests, diseases, and endangered species. Proceedings of the National Academy of Sciences of the United States of America 108, 8323–8. (doi:10.1073/pnas.1016846108)
Chad, I., Carwardine, J., Martin, T. G. & Nicol, S. In press. MOMDPs : A Solution for Modelling Adaptive Management Problems. , 267–273.
Kapaun, U. & Quaas, M. 2012 Does the optimal size of a fish stock increase with environmental uncertainties?
Mattmann, C. A. 2013 Computing: A vision for data science. Nature 493, 473–5. (doi:10.1038/493473a)
Seekell, D. a., Cline, T. J. & Winchcombe, R. J. 2013 Can Management Reduce Harvest Inequality in Recreational Fisheries? North American Journal of Fisheries Management 33, 148–152. (doi:10.1080/02755947.2012.746246)

Data Uris For Image Archives

2013-01-24T00:00:00-08:00

Figures have been one of the standing challenges of the open notebook. Displaying figures online requires that they are first uploaded to a server somewhere. Recently I have used automated uploads to external servers such as figshare to host all images generated in the course of my research, and simply point to those graphs which I wish to include in a notebook page using an image link. Unfortunately, this means that the images themselves are not being permanently archived when I deposit my notebook entries into their annual archives on figshare. (Because figshare provides objects with unique identifier DOIs and benefits from CLOCKSS archival preservation, I use it to ensure a permanent backup of the notebook contents is available). I’ve recently discovered Data URIs, which provide a way to embed the image data directly into HTML (or markdown) entries. I am trying to decide if they offer a better way to address this challenge, so working through my thinking here.

History of Figure management in the notebook

When I first started an open online notebook on the MediaWiki platform OpenWetWare, I simply uploaded all images manually to the wiki. Soon I discovered that it was much easier to upload them programmatically. To accomplish this, I used Flickr as the image hosting service. Flickr gave me a few convenient tools like batch uploads, pre-sized images and slide-shows for embedding. More conveniently, Flickr allowed me to automate the uploading of images through the API. Using the API in my R scripts I could automate uploading of the figures as they were created when scripts were run. This integration became easier with my adoption of knitr, which has the option to automatically upload images built in (using imgur.com). I soon extended this to use Flickr or Wordpress.

Challenges of linked images

While this simplified the workflow considerably, it does not lend itself to archiving content well. Since migrating to Jekyll, it is relatively straight forward to archive the notebook entry by entry in markdown format, without all the external web content for the appearance. Unfortunately, with images linked in from Flickr and other external hosting services are not included in this way. Even archiving the entire HTML source for the site doesn’t help, since the figures are on a remote repository. When I migrated my OWW and Wordpress notebooks to Jekyll, the image links remained pointing to OWW or Flickr or wherever the images were originally uploaded. Should Flickr vanish (as Delicious effectively did), the images would be lost. The copies of the notebook entries I deposit on figshare only have the links to these entries, not the entries themselves. Even if the images were copied over, the entry would still point to the original URL, not the copy. To address this, I started archiving the images locally. This removed the external dependency and kept everything together so that it could be archived in a way that preserved the link structure, but has its own drawbacks.

As I do far more runs of the scripts then I post to the notebook (though most of these can be found in the Github links), the image archive is much larger than what images are actually included in the notebook. Another issue is that the notebook source files are all conveniently managed with git and stored on Github, so storing the images locally involves explicitly excluding the image files which are both much larger and as binaries not ideal for version management.

Data URIs

The simple solution is to avoid linking the images externally, but instead embed the image data directly into the notebook entry. Data URIs do exactly that, encoding all the image as part of the link url following a declaration of MIME type, such as

<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg=="

Image data URIs are usually introduced to reduce the load and waiting time on the server, which has to query each external image separately (CSS sprites are a more effective way to accomplish that goal; which is what I use for the icons on this site). They are also convenient for generating stand-alone HTML files, in which the externally linked data sources, such as images, CSS, etc, are encoded as 64bit data strings.

Generating URIs

Given an image, we can generate the data URI using the knitr function image_uri("image_name.png"). Knitr also supports data URIs as directly as an image output option, in just the same way that one would specify a custom upload function, using the command

opts_knit$set(upload.fun = image_uri)

Pandoc can also generate HTML with data URIs for images in place of the remote image links using the --self-contained option, where it will download all the external images and encode them. This may be the best strategy for creating archival versions of entries for figshare, etc. Unfortunately the conversion is not available when running markdown to markdown, but can be effected using markdown to HTML, and then back. The only downside of this approach is that the yaml header formatting is lost, and the output markdown used is Pandoc’s flavor.

Difficulties of Data URIs

Not displayed in Github Markdown

My chief disappointment is realizing that Github’s markdown renderer will not display data URIs, so images won’t show up on the Github copy of knitr markdown files. It would have been most convenient to have knitr always embed the images for my scripts and avoid external linking for Github files. This would be particularly helpful for scripts run on remote servers where the compute nodes do not have external web connections and so images have to be uploaded from the head node after running the script. Oh well, hopefully Github will consider addressing this.

I don’t see a good reason why Github markdown doesn’t display images, since markdown displays any other valid HTML. Perhaps it is security related since almost anything can be put in a data URI. It may have something to do with how there server is handling images to begin with, since the images that are displayed come not from the original URL but from a separate link on akamai.net. I did send in a suggestion that they address this and was told they would look into it. Meanwhile I will continue uploading these images externally, and converting to data URIs when generating the files for the notebook version.

Large File Sizes

Because all of the image encoding is embedded directly within the link, and these data URIs are huge. This creates several issues.

The markdown a deal less readable before it is rendered. A decent editor can just fold these away (e.g. select block and zf in vim).
The data URIs increase the filesize of the HTML dramatically. Compression is about 1/3rd more than the original image sizes. This can slow load times and makes the repository larger. As most images in the notebook are generated as basic pngs, the file sizes aren’t huge (100 Kb range), but much larger than the raw HTML.
Not all browsers support Data URIs, so images may not render in older versions of IE
The markdown parser takes much longer to generate the output. This appears to actually time out Jekyll building in my current notebook.

Resolution

I have a branch of the notebook, image-uri, in which (almost) all images are converted to data URIs. Unfortunately, the images do not display on the Github copy and the branch takes rather much longer to compile, making this an impractical standard solution for the notebook.

As I discussed above, I have recently experimented with hosting all images I generate on the local server, removing the dependency and archival concerns of Figshare. Unfortunately, it makes for a very large and potentially more fragile site to have to carry around the complete image log, 99% of which does not appear in the notebook but only on Github script runs. Keeping only the images used in posts makes more sense, but requires an extra step which is otherwise redundant, since the web being what it is can just point to the external copy. To address the primary concern is making sure that the archival copies on figshare include the images, I think I will stick with compiling an archival version with Data URIs while otherwise continuing to host the images on Flickr.

Archival copies can be made using Pandoc by generating the site using the --self-contained option for the pandoc parser in _config.yaml, though this is rather slow. Then the archive would be the HTML files in the _site directory, rather than the raw markdown files. While alternatively I could use a script to just render the markdown, overall it may make more sense to archive the stand-alone HTML, since it includes more metadata and the advantages of the XML formatting.

Tipping Points Comment In Nature

2013-01-12T00:00:00-08:00

On Wednesday my comment piece with Alan, Tipping points: from patterns to predictions appeared in Nature (next to a nice comment by Heather Piwowar on NSF’s move to recognizing products of research more broadly than publications alone).

Heather has written an excellent post on the process of writing, which rings true with my own experience. Working with an editor with clear expertise in science writing and communication who was otherwise unfamiliar with the details of this field was an immensely rewarding experience, and her contributions really ought to be recognized more clearly. She stuck with me as I struggled to explain concepts I wasn’t even sure were well understood in my subfield, and then helped me rephrase towards a larger audience. I learned how specific examples and even potentially extraneous details could help anchor and clarify a description. Of course, some compromise was inevitable. I later had some explaining to do when the fact-checker questioned some of my citations. Yes, I know Sebastian doesn’t actually talk about salemanders in Schreiber and Randolf 2008, and Stephen Lade and Thilo Gross don’t talk about cod collapse; but their models do, I promise!

I doubt any scholar takes on a topic of close personal interest in the space of two pages and feels completely happy with the resulting message. While the piece will no doubt be received as a pushback against the bandwagon, our goal was more road-map to interesting open questions than critique of the early steps. In the piece, I aimed to raise two important issues or gaps in the existing literature, and then to propose two potential ways forward for each issue. The goal is not to close the door on early warning signals, but to further unroll the map and draw more attention to the uncharted lands.

Warning signals without controls

The first gap we highlight is the need to understand how early warning signals might be applied in the real world, rather than in experimental designs that rely on replicates and controls to identify what is and what isn’t a warning signal. We propose two potential directions, one primarily empirical, one primarily theoretical:

Establish baselines. The challenges to doing this well are manifold, but we won’t know if or how to address them without more active research.
Statistical Models In the absence of controls, appropriate null models might be able to simulate comparison groups we do not have. I have begun to explore this in my own work, see Boettiger and Hastings (2012).

Inability to leverage context-specific information

In our comment, this comes off as condeming the generality of warning signals. Early warning signals such as variance (Carpenter and Brock 2006) or spatial autocorrelation (Dakos et al 2011) may incerase before a catastrophic transition, but they can also decrease (Schreiber and Rudolf 2008, Bel et al 2012).

Use all the data. While a lack of detailed mechanistic understanding of complex systems is more often the rule then the exception, we mustn’t give into the huberis that we can ignore context-specific information we know. When facing problems as challenging as detecting sudden shifts are no time to go battle with one hand tied behind our backs. By working with domain experts in areas where transitions are best understood, we may learn how best to do this.
Follow the data A more widely appreciated issue for any signal is the concern of an adequately long and well sampled data set. While the globe ushers in an era of “big data”, long time series aren’t easy to come by. Perhaps this is a case in which searching under the street light for the keys can pay off. Early warning signal research has already shown us once how to find weak patterns if we know what to look for. A promising line of of work would identify (potentially system-specific) signals in the kind of data that is most likely to be available, such as high-resolution spatial imagery. Ignoring such patterns because they do not generalize to other systems will only slow us down.

#icanhazpdf?

Unfortunately the piece is not available open access (though do write to me for a reprint if you are interested). I can distribute the original text in 6 months. As an advocate for open science, my 0/9 record of publishing in open access journals might raise eyebrows, if also unequivocally demonstrating my commitment to pragmatism ;-). All my other papers are available through as green open access/preprints on arXiv, and I hope this one can join them when the terms permit.

References

Lade, S. J. & Gross, T. Early Warning Signals for Critical Transitions: A Generalized Modeling Approach. PLoS Computational Biology 8, e1002360 (2012). doi:10.1371/journal.pcbi.1002360
Schreiber, S. J. & Rudolf, V. H. W. Crossing habitat boundaries: coupling dynamics of ecosystems through complex life cycles. Ecology Letters 11, 576–87 (2008). doi:10.1111/j.1461-0248.2008.01171.x
Dakos, V., Kéfi, S., Rietkerk, M., Nes, E. H. Van & Scheffer, M. Slowing Down in Spatially Patterned Ecosystems at the Brink of Collapse. The American Naturalist (2011).doi:10.1086/659945
Bel, G., Hagberg, A. & Meron, E. Gradual regime shifts in spatially extended ecosystems. Theoretical Ecology 5, 591–604 (2012). doi:10.1007/s12080-011-0149-6

Learning Jekyll

2012-12-30T00:00:00-08:00

Some quick thoughts on learning Jekyll

After recieving a few queries on how to get started with Jekyll I thought I’d jot down my own opinions here. Of course there is already rather good documentation for the software and plenty of blogs providing tutorials. While no doubt more than adequate, I have more opinions on how to get started actually learning what’s going on than I could easily fit into a tweet, and also a couple favorite links to references so I thought I’d jot down my reply in long form here.

Brief Background

Jekyll is a fantastic and popular static blogging platform, written in Ruby by a Github founder, Tom Preston-Werner, which powers this site (at least currently). Similar static website generators exist for Python (Hyde) Haskell (yst), and probably others, but Jekyll is quite widespread.

With the (unofficial) slogan of the bloging platform for hackers, Jekyll is aimed at folks who don’t mind a little coding. If you’re looking for a polished, user-friendly product where a huge community of developers has written most imaginable extensions, check out Wordpress.org. If you want something where you can easily get into the guts and customize everything, Jekyll may be the best game in town. Compared to a dynamic site, a static site is much more lightweight, which translates into faster, more stable, and less expensive. What’s not to like?

Where to start?

A couple pre-configured Jekyll setups have emerged to reduce the learning curve, most notably Octopress and Jekyll Bootstrap. In my opinion, in an attempt to make things easier through tricks such as automated Ruby makefiles (Rakefiles) I think these projects have actually made Jekyll appear much more complicated than it really is, as evidenced by the number of talented programmers running Octopress with the identical configuration. They may get you started faster, but their use of advanced techniques to provide more functionality out-of-the-box makes for a tougher learning curve.

See the Jekyll-Bootstrap’s introduction to Jekyll for a nice explanation of how Jekyll is laid out. Understand the basic YAML headers, and the role of the _layout, _includes and _posts directories, and you’ve mastered the basics. Or read on below for my take on getting started.

Running Jekyll

For a basic site hosted on Github, one never actually has to run Jekyll locally – just push the site to a gh-pages branch of a Github repository, or to the master branch of a repository named username.github.com for a personal website, and Jekyll will be run by the server. Jekyll is frequently run locally instead, allowing the user to copy the static website it generates to any (static) web server (see Hosting options, below). Simply install Jekyll following the directions on its homepage README.[^1]

My rough guide to learn-by-doing

I believe the best way to start with Jekyll is start with a really bare-bones site. You add features as you need them, making maintenance easy.

Start with an empty repository. Create a markdown file, say, index.md. Write whatever you want in standard markdown, just stick a YAML header on the top. Stick what? From your standardized testing days, think something like markdown:html :: yaml:xml. Three - on their own line to start, three - on another line to close define the block of YAML data, whose mere existence is enough to get Jekyll to pay attention to that file. Here we specify any metadata we want.

    ---
    layout: default
    ---

    # My site title

    some text

And voilà, we have a functional Jekyll site. The basic thing to remember is that everything else is gravy, to be mastered as you have a need for it. The advanced features can be super helpful once you know them, but in the beginning they may just get in the way of understanding what’s actually going on.

Getting deeper: Jekyll site file structure

Directories that are functional for Jekyll but do not appear in the generated website start with an underscore _.

Root directory. You don’t need any of the standard directories for a barebones site. As we saw above, any markdown file with YAML header matter in the root directory will be parsed. Don’t like markdown? We could write these files in html (with .html extension) instead. Jekyll will parse them as long as they have some YAML header matter, allowing us to still use things like page templates and other tricks from Liquid code described below.
_posts Your posts live here. They should be titled in YYYY-MM-DD-post-title.md format, which will allow Jekyll to automatically figure out the post date and title, while also keeping the posts neatly organized in this directory. (The title and the publication date, optionally including time can alternatively be given in the YAML header matter if you don’t like this convention. See below).
_layout For a completely barebones site, you don’t need this directory. Here you can provide files defining the basic HTML layout, or template, for each of the pages in your website. If your familiar with HTML, you can go ahead and set up a page as you like, and then add the single line of Liquid code in the body to pull in the HTML generated by parsing the markdown files. The mimimal layout file might look something like this (see an HTML tutorial for details):

<!doctype html>
<html lang=en>
<head>
<meta charset=utf-8>
<title> {{ page.title }} </title>
</head>
<body>

{{ page.content }}

</body>
</html>

where the double braces and words page.title and page.content are Liquid code that will tell Jekyll to insert the title and content of the page in the appropriate spot. To use a layout, save this file in _layout with the desired layout name, e.g. plainlayout, and then it can be applied to any page (markdown or html file in the root directory) or post by adding layout: plainlayout to the YAML header of that file. This provides a super convenient way to create page templates for a static site.

_includes Often one might have several layouts corresponding to different kinds of pages on your site. To avoid copy-pasting the parts of the layout that stay the same (perhaps all have the same header matter pointing to your CSS files and analytics tracker), you can just place these snippets inside the _includes directory with their own filenames. For instance, we could move the header text to a file called header in that directory with the contents:

<!doctype html>
<html lang=en>
<head>
<meta charset=utf-8>
<title> {{ page.title }} </title>
</head>

To automatically reuse a snippet of HTML code in any file on your site (a layout file or other), we need to learn one more Liquid command. This one goes between brace-percent symbols, and uses the command include, followed by the filename, like so:

{% include header %}
<body>
{{ page.content }}
</body>
</html>

The above example provides a shorter layout file, which pulls in the header snippet. If you have only one layout, this is of course pointless, but the more you add to and change your site, the more useful the ability to reuse a single HTML chunk with just a line of code becomes.

Configuration

YAML and the _config.yml file. Recall that YAML is the very human-readable data structure, whose syntax is explained quite throughly on it’s wikipedia page. One glance at an example reveals almost all you need to know about the syntax, such as:


---
layout: post
category: ecology
tags: 
  - howto
  - jekyll
modified: 2013-01-07
---

Lists can be done in a variety of ways, comma-s

One of the Jekyll beginner’s mysteries is knowing just what words/metadata/variables are already available in Jekyll and how to add those that are not. A list appears on the Jekyll Wiki. Note that the syntax uses the ruby structure, so page.title and page.date are both part of the page. We can create any custom variable by adding it to that page’s yaml header. For instance, modified is not defined anywhere in Jekyll’s default Template Data, but none-the-less we can access this modification date in this page’s layout or the page itself using {{ page.modified }} liquid tag.

Metadata belonging to the site as a whole, rather than a particular page, is specified in the _config.yml file, along with a few options telling Jekyll how to parse the site. Again, the Jekyll wiki has a great overview. As the extension implies, this entire file is in YAML. If we want to create new global variables, such as our twitter account, we can just add them to the YAML like so:

author:
  twitter: cboettig
  github: cboettig

(note the indentation for hierarchy) and now my twitter username is available through the liquid command site.author.twitter. Again, it’s just another trick to insert some text, just like _layout and _includes. Only use can prove how helpful this can be.

Markdown parser hell. A lot of Jekyll adopters are already familar with Github and it’s flavor of markdown with such features as fenced codeblocks, thus it comes as real suprise to run into advice explaining how code blocks need to be wrapped in the never-heard-of “Liquid” syntax {% highlight %}. Ignore this nonsense. How your markdown is parsed depends on what flavor of markdown you use, and there is a special level of hell for anyone inventing a new flavor (Atwood 2012). Fortunately Github’s Tom Preston-Werner finally gave up the perfect irony that Github’s flavor was not one of the built-in Jekyll parsers (though being a hacker system, this could be added manually), and you can now get that behavior by setting your chosen parser to be redcarpet with the convenient extensions [^2]

markdown: redcarpet
redcarpet:
  extensions: ["no_intra_emphasis", "fenced_code_blocks", "autolink", "tables", "with_toc_data"]

More advanced tricks

Liquid Your next stop should really be to read Liquid for Designers. Liquid is the glue that specifies all the rules needed to assemble your site, and is quite simple. Understand how Liquid works and you’ve mastered Jekyll. Jekyll provides some additional Liquid functions common for blogs.

Your next stop is to get some really nice customizable CSS scaffolding to control the layout for your site. While there’s a huge number of free CSS themes available, this isn’t 2002, so head on over to Twitter-Bootstrap (a project originally created by some of the programmers at Twitter, now evolved to it’s own community). The CSS-based grid layout and some very nice JQuery Javascripts tools can make your site as pretty and as interactive as you can imagine. Like Octopress, prebuilt CSS can lead to all sites looking the same. Check out themes for Bootstrap, and the excellent set of icons by FontAwesome.

Your final step is where the hacking really begins. Now and again you imagine something you just can’t do with these tools. Your only recourse is to ~~ask on Stackoverflow for someone to do it for you~~ learn a bit of Ruby. Anything you can write in Ruby you can add to Jekyll as a “plugin”, by defining a Liquid extension for a Ruby function. Ruby is clearly a favorite language of web-developers thanks to its dynamic-site implementation Ruby-on-Rails and a vibrant Github community, so there’s a wealth of useful tools, including an implementation to most APIs you might care to interact with (Github, Twitter, various Google APIs, are among the ones I’m using).

Hosting options

Because Jekyll generates a static site, it can be hosted almost anywhere for nothing or next to nothing in costs. The primary question to consider is your choice of domain name. If you wish to host your site from university domain name, just copy the contents of _site into your public_html or similarly named directory on the web server.

While having a university affiliated domain can appear more official and help with discoverability, buying your own domain name as your permanent Internet home may be a more reliable long-term solution. Once you’ve purchased a domain name from a provider, a static site can be hosted at your own domain name for next to nothing on Amazon S3, or for literally nothing through Github.

[1]: Details on installing and running ruby gems for Jekyll will vary by operating system, so I won’t bother with more details here. Some further background for linux based platforms at least appears in my site’s info file.

Notes

2012-12-27T00:00:00-08:00

Nonparametric Bayes

Continuing sensitivity analysis

The commit log to sensitivity.md and earlier to myer-exploration.md (now depricated) capture the summary figures for replicate runs of the observation data, with commits corresponding to various parameter configurations, etc. Here is a nice collection of replicates from sensitivity.md

Harvest during observations Tweaked calculation of observation data given the harvest regime under which observations were taken (simulation was not implementing all harvests).
Also adjusted norm used in GP (the parametric use log-normal densities in calculating the transition function on the untransformed data. The GP, also on untransformed data, uses normal noise as per its model).
Relaxed priors, shows no impact on GP performance.

Non-stationary dynamics

Ran the Allen et al. Allee model under conditions for non-stationary stable states (e.g. period 2, period 4) etc, with and without substantial process noise (to confirm the cyclical pattern). Shows good performance of GP against terrible perfomance of alternative parametric approaches. Closes issue #16

plot of chunk unnamed-chunk-2

plot of chunk unnamed-chunk-4

allee.md

Commit log links to some additional examples:

another allee example altering MLE initial condition to attempt better likelihood estimates. GP still outperforms. 04:40 pm 2012/12/27
in oscillatory regime with non-negligible noise. again GP performs nearly optimally while MLE methods suffer greatly 04:27 pm 2012/12/27
Example of Ricker-Allee in oscillation regime, shows non-trivial dynamics, GP does very well (closes #22) 04:17 pm 2012/12/27
work with non-named arguments in MLE, consistent ordering for plot legends 04:05 pm 2012/12/27

Other projects

Upgraded gems, had to update notebook’s liquid code metadata to handle dates in string format on pages vs being date objects in posts. de4d4cc
Setting up multiple_uncertainty as a separate repository (branched from pdg-control).

Notes

2012-12-23T00:00:00-08:00

added generation of observational data under varying harvest conditions (issue #19)
added MLE fit based on the data-generating model (issue #20)
GP plot with and without nugget variance (issue #17)
Run longer simulations under policy such that sustainable profits clearly beat out collapsing the fishery (part of issue #22)

Progress Summary

2012-12-21T00:00:00-08:00

Progress mid-October through mid-December

80% Time: nonparametric-bayes

The bulk of my time has been spent becoming familiar with the literature on Gaussian Processes and their numerical implementation. I have written my own Gaussian process code from scratch to convince myself of my understanding of the methodology, and explored some of the related numerical issues addressing computational speed and stability.

I then developed an algorithm for applying the Gaussian process inference of the state dynamics to stochastic dynamic programming methods for determining an optimal harvesting policy. I have compared the performance of the Gaussian-process inferred model to the optimal solution (given the exact underlying model) and to solutions based on estimated parametric models (both matching and not matching the underlying structure). This analysis shows the benefit of the nonparametric approach in better accounting for structural uncertainty in the underlying dynamics.

Steve and I have just started to look into the potential for state shifts and warning signals there-of in the ENSO / Pacific decadal oscillation. I obtained 2 appropriate datasets and have run a very preliminary analysis.

20% Time: Independent projects

pdg-control Working group paper. Updated analysis for Figures 3 and 4. Updated draft. Conference call for feedback from working group participants. (Current draft)
Wrote comment piece on Early Warning Signals for Nature.
Working on EWS review paper for Theoretical Ecology with Noam Ross. (Current draft)
Working on analysis optimal policies under multiple uncertainties (with Jim Sanchirico, Mike Springborn). #multiple-uncertainty tag in notebook; currently analyses are part of pdg-control repository.
Finished and submitted NSF post-doc application

Other academic activities

Wrote requested reviews for Conservation Letters, Ecology Letters, Proceedings of the Royal Society B. Declined to review for Systematic Biology.
Presented my exit seminar in the UC Davis Center for Population Biology Colloquia series.
Interviewed for the Nature column Turning Point.
Invited and attended the PLoS alt-metrics workshop and hackathon
Ran two sessions on version management, dynamic/reproducible papers, and building academic websites for the Mangel group.
Presented scaling from individual to population level models through the van Kampen expansion in Steve’s Applied Math Club
Participated regularly in applied math club meetings.
Established communication with FishBase team, arranged a coalition of developers interested in API access.
ongoing development in rOpenSci project

Goals for January - March

80% Time

Writeup of the Nonparametric Bayesian approach to management paper.

Initial goal for manuscript is proof of concept piece with the aim of demonstrating the value of a nonparametric approach to ecological decision-making. Alternatively a more technical piece could focus on the details of using Gaussian processes models for stochastic dynamic programming.

Outline of manuscript: Dec 31st
Decide on manuscript figures to include
Complete draft
Appendices
Solocit friendly revisions
Submit (by March 31?)

Next steps and Additional projects

The multi-species context is probably the next goal. Depending on timing, Steve may take the lead on that in these three months; I aim to be working in this in March. The step after will be the active/reinforcement learning/approximate SDP approaches. I aim to start into the background reading for that material but real progress on that front will happen after March.
Steve and I will likely continue to do something with the ENSO / PDO analysis as well. (Currently waiting on Steve’s pending queries to domain experts for further discussion)
I will also need to determine my summer conference schedule during this interval, and submit abstracts, etc. I have committed to the symposium to which I was invited at SIAM in San Deigo, July 8 - 12, and plan on attending ESA.

20% Time goals

Finish and submit EWS review paper: end of January
Finish and submit pdg-control policy costs paper (see current issues)
Finish the methods section for the Labrids paper
Finish my pending review for Environmental Modeling and Assessment.
Skype with pdg-control Working group, basic follow-up tasks.
Some next steps on multiple uncertainty.

Additional academic goals

Present in mega-group meeting: Feburary 20th
Join the phylotastic hackathon remotely(?)

Results Comparing Gp To Parametric

2012-12-20T00:00:00-08:00

Comparison of the Gaussian process inference to the true optimum and a parametric estimate.

Comparison across 100 simulations under the policies inferred from each approach show the nearly optimal performance of the GP and the tragic crashes introduced by the parametric management.

Sensitivity analysis

Working through an exploratory sensitivity analysis to see GP performance over different parameters.

Example from 32 replicates

Distribution of yield over replicates shows the parametric model performing rather poorly, while most of the GP replicates perform nearly optimally.

smaller noise:

larger growth-rate parameters:

from the commit log

smaller r, shows GP outperform parametric estimate 09:21 am 2012/12/21
small noise. works well. 11:01 pm 2012/12/20
GP performs poorly with weak noise prior (nugget) when not conditioned on 0,0 10:45 pm 2012/12/20
less successful with higher noise level 09:10 pm 2012/12/20
Ah here we go! with a decent allee effect model, the results are much clearer! 08:52 pm 2012/12/20
strange Ricker Allee example – even true model doesn’t harvest successfully 08:37 pm 2012/12/20
delta 1.5, r 1.3 K 4.5 08:17 pm 2012/12/20
higher K value results in better persistence in general, not always outperforming parametric model though. 07:57 pm 2012/12/20
harvest, and then escaped population recruits; comparable 07:09 pm 2012/12/20
shorter OptTime and lower initial state do not much change the fraction that fail to persist 06:31 pm 2012/12/20
Sensitivity analysis over replicates shows mixed results 05:35 pm 2012/12/20
fixed bug (had Ef and V in gp_transition_matrix) 05:18 pm 2012/12/20
still crashing 05:01 pm 2012/12/20
more iterations 04:34 pm 2012/12/20
corrected iterations 04:28 pm 2012/12/20
cpu=8 for running on one.ucdavis 02:21 pm 2012/12/20
unique seeds for parallel computing 02:11 pm 2012/12/20
30 replicates with full graphs 02:03 pm 2012/12/20
avoid external source call, print plots 12:49 pm 2012/12/20
trying methods for looping over replicates efficiently 12:38 pm 2012/12/20

next goals/issues

Exploring Gp Model Space

2012-12-19T00:00:00-08:00

Trying to think about a more systematic way to go about varying the parameters: the underlying parametric model has 3 parameters for the stock-recruitment curve’s deterministic skeleton, plus growth noise. (My first exploratory phase has been just to try different things. See my various tweaks in the history log Clearly time to be more systematic about both running and visualizing the various cases.)

Should I just choose a handful of parameter combinations to test? (Trying to think of a way to do this that is easy to summarize – at least I can summarize expected profit under each set). Presumably, for each set of these parameters, I’d want a few (many?) stochastic realizations of the calibration/training data.

Would it be worth digging up some real-world data-sets and base the selection of underlying model parameters on them?

Then there’s a variety of nuisance parameters: grid size, discount rate, price of fish (non-dimensionalization eliminates that one I guess), cost to fishing (and whether the cost is on effort or harvest, whether linear or quadratic, etc), harvest grid size / possible constraints on maximum or minimum allowable levels for the control; length of the calibration period (and related dynamics if we use any of the variable fishing effort models you showed me today).

Additionally there’s the MCMC-related nuisance parameters – parameters for the priors, possibly hyperpriors, and the MCMC convergence analysis (selecting burn-in period – currently 2000 steps out of 16000, etc) . Also the distributional shapes for the priors, and perhaps more meaningfully, the GP covariance function (using Gaussian for simplicity, but might want to look at Matern, and the various linear + Gaussian covariances).

New and progressing issues

from the commit log today

prior/posterior plots added to the BH-Ricker example. Comparable but non-optimal performance by GP. 06:07 pm 2012/12/19
oscillating ricker still does not give decent solutions 05:45 pm 2012/12/19
shows posteriors and priors of hyperparameters 05:38 pm 2012/12/19
fixed minor typo in calc of posterior distribution plots #18 05:15 pm 2012/12/19
plotting for posteriors and priors added #18 04:13 pm 2012/12/19
replicate 02:22 pm 2012/12/19
time to start writing 09:16 am 2012/12/19
Ah, good. With a bit more data, a very nice example of how GP can avoid problems of a structurally inaccurate parametric approach (Myers vs Ricker). 09:15 am 2012/12/19
more data going up to positive equilibrium, better but still does not avoid crash 08:29 am 2012/12/19
An interesting example – seems to work despite being calibrated on a crashing dataset. 08:08 am 2012/12/19
With smaller K, GP cannot quite determine a proper policy 07:37 am 2012/12/19
try different dynamic parameters on Myers (lower K) 05:59 am 2012/12/19

Misc: ropensci

resolved some outstanding issues on rfigshare (rfigshare/issues?state=closed)

Random_ews_example

2012-12-17T00:00:00-08:00

ENSO EWS

Let’s just see what happens with the MEI data for PDO:

dat <- read.table("http://www.carlboettiger.info/assets/data/mei.csv", header=TRUE)

For the moment let’s ignore annual structure and just collapse this into timeseries sampled bimonthly.

require(reshape2)
dt <- melt(dat, id="YEAR")
X <- dt$value
Z <- X[!is.na(X)]
Z <- data.frame(1:(length(Z)-1), Z[1:(length(Z)-1)])

png("mei.png")
require(earlywarnings)
a <- generic_ews(Z, detrending="gaussian")

How about the data from MacDonald (2005)

original data link

dat <- read.table("http://www.carlboettiger.info/assets/data/pdo-macdonald2005.csv", header=TRUE)

png("macdonald2005.png")
require(earlywarnings)
a <- generic_ews(dat, detrending="gaussian")
dev.off()

Nonparametric Bayes Comparisons

2012-12-15T00:00:00-08:00

Notes on structure of examples

Externalized code so that different example scripts call identical commands for fitting and plotting to avoid duplication of code. Example scripts tend to be pretty text poor at the moment, would probably benefit by better descriptions in the markdown. May still involve redundant and potentially out of date descriptions though.

Currently, externalized code for GP comparison experiments is in gaussian-process-control.R, and is called by may79-example.Rmd, myer-example.Rmd, and reed-example.Rmd.

Also fixed knitcitations to at least include citations in markdown files.

Performance of GP compared to Ricker on an underlying Allee model

Using stationary data only

Inferred GP

plot of policy functions

fish stock dynamics

harvest dynamics

(see myer-example.md for code and more graphs & details)

Using more data

Inferred GP

plot of policy functions

plot of sim-fish

plot of sim-harvest

(from earlier myer-example.md)

Comparing GP to estimated Ricker on BH dynamics:

GP performs well, but Ricker performs adequately as well: reed-example.md (shows an example with stationary data).

plot of chunk gp-plot

plot of chunk policy_plot

plot of chunk sim-fish

plot of chunk sim-harvest

Other advances

Approaches shown above address a variety of issues as well:

Compares to a Ricker estimated by MLE (#8)
Considers fewer non-stationary observations, but include 0,0 as observation (#13)
Includes scrap value (#10), though this does not guarentee no fishing under non-persistence estimation

A few next steps:

measurement error should be introduced in the simulations (#11)
Evaluate GP performance under larger noise conditions. (#14)

See issues log for further issues to explore and more details on closed issues. See commit log for full details, this entry summarizes progress over the latter half of the week.

Nonparametric Bayes Notes

2012-12-11T00:00:00-08:00

Code gp_transition_matrix for generic multi-dimensional case

Understanding Gaussian Process performance

If the estimated recruitment dynamics correspond to population dynamics that are non-persistent (might call this non self-sustaining, but in a rather stricter sense than when Reed (1979) introduced that term), and if no reward is offered at the terminal time point for a standing stock (zero scrap value), the GP dictates the rather counter-intuitive practice of simply removing the stock entirely.

Exploring this by comparing evolution of the probability density for the population size under the transition function. Consider this example from a May1979 model (full run in may1979-example.md): The Gaussian process infers a rather pessemistic evolution of the probabilty density (grey distribution becomes black distribution when unharvested, 20 years (OptTime)):

GP transition function

plot of chunk gp-F-sim

Whereas the actual transition function moves the stock to a tight window around the high carrying capacity:

true transition function

plot of chunk par-F-sim

Often this results in a policy function that harvests all the fish, since they won’t persist. Exploring approaches to avoid such solutions, such as adding a reward for leaving some standing stock at the boundary time (issue #10).

Multi-species examples (issue #7)

multidimensional.md

Fragility of parametric rigidity examples

infer under BH and simulate under allee
Infer ricker, simulate under BH
Other examples?

MCMC

Examples of controlling priors, resulting posteriors. See yesterday’s notes for details

additional R software support

Have been focusing recently on the MCMC implementation for treed Gaussian Processes, provided in the tgp package.

Lots of various implementations of Gaussian Proccesses in R in geospatial stats packages (e.g. Kriging implementations) including some the offer fully heirachical Bayesian approaches with a variety of twists:

psgp Projected Spatial Gaussian Process (psgp) methods, Implements projected sparse Gaussian process kriging for the intamap package
gstat
geoR
spBayes spBayes fits univariate and multivariate spatial models with Markov chain Monte Carlo (MCMC).
ramps Bayesian geostatistical modeling of Gaussian processes using a reparameterized and marginalized posterior sampling (RAMPS) algorithm designed to lower autocorrelation in MCMC samples. Package performance is tuned for large spatial datasets.

From the commit log…

another example (includes some data in the higher range) #7 06:49 pm 2012/12/11
a simple multi-demensional example (no policy function yet) #7 06:43 pm 2012/12/11
updated notes on mcmc approach #6 03:51 pm 2012/12/11
transition matrix method for GP takes Ef, Cf explicitly now. 03:50 pm 2012/12/11
GP that results in scorched earth fishing strategy 02:19 pm 2012/12/11

Prior Distributions For Tgp Mcmc

2012-12-10T00:00:00-08:00

I am trying to understand the interface for tgp method, but much of it is not well documented (yes, despite reading through the package R manual, two nice vignettes, and a the PhD Thesis describing it.)

Just to get some consistent notation down: consider the case of a Gaussian process with a Gaussian/radial basis function kernel, parameterized as:

And observations from , for which we seek to approximate as a (multivariate) Gaussian process, . The covariance matrix is given by our kernel , conditioned on our observations; the mean is given by a constant or linear model, likewise conditioned on our observations. If the multivariate normal of observed and predicted points is,

the conditional probability is

Our hyperparameters are , , and , for which we need priors. We’ll use the inverse gamma,

The distribution arises as the marginal posterior distribution for the unknown variance of a normal distribution if an uninformative prior is used; and as a useful conjugate prior if a less uninformative prior is preferred. However, it is common among Bayesians to consider an alternative parametrization of the normal distribution in terms of the precision, defined as the reciprocal of the variance, which allows the gamma distribution to be used directly as a conjugate prior.

is X, is Z
is the parameter s2 in the tgp package, coming from inverse gamma prior with parameters and , which are set by s2.p = c(a_0, g_0). Optionally, and can be taken from an exponential prior , in which the value of is given by s2.lam = c(lambda_a, lambda_g). Setting s2.lam = "fixed" “turns off” the use of a hyperprior.
I believe is d, which Grammarcy calls the range parameter (other terms include the “lengthscale”). I am not sure what either d or s2 correspond to if the covariance function corr is something other than the radial basis function. I’m also not clear how to set the correlation function to a strictly Gaussian covariance – corr is set to the exp family, which leaves the power as a parameter p_0:

(from equation 4 in the Vignette (1)). The parameter p_0 is fixed at 2, and not treated as a hyperparameter.

The length-scale comes from a hyperparameter that is the sum (mixture) of two Gamma distributions, with prior parameter c(a1,g1,a2,g2) where g1 and g2 are scale (1/rate) parameters. As before, these parameters c(a1, g1, a2, g2) can be drawn from an exponential distribution hyperprior whose values are given by d.lam, or turned off.

is nug. It seems to have same shape prior as , specified by c(a1,g1,a2,g2), and optional hyperprior as well. Additionally it can be fixed to a constant by setting nug.p = 0 and specifying the staring value as gd[1].
The mean is not fixed at zero but is instead a linear model, we need hyperparameters for it as well. The key parameter there is the slope . The functional form for it’s prior is specified as bprior, which takes values bflat, b0, bmzt, or bmznot. A sensible choice is as a Gaussian b0, as shown in Equation 1 of the vignette

\[ \begin{align} Z | \beta, \sigma^2, K &\sim N_n(\mathbf{F} \beta, \sigma^2 \mathbf{K}) \\ \beta | \sigma^2, \tau^2, \mathbf{W}, \beta_0 &\sim N_m(\beta_0, \sigma^2 \tau^2 \mathbf{W} ) \\ \beta_0 &\sim N_m(\mathbf{\mu}, \mathbf{B}) \end{align} \]

Example comparison

Say, given some observed data pairs and predictions desired on a grid ,

X = seq(-5, 5, length=50)
x = c(-4, -3, -1,  0,  2)
y = c(-2,  0,  1,  2, -1)

Manually I could just do:

d = .5; epsilon = .1; sigma = 1; #fixed hyperparamaters
SE <- function(Xi,Xj, d) sigma * exp(-0.5 * (Xi - Xj) ^ 2 / d ^ 2)
cov <- function(X, Y) outer(X, Y, SE, d) 
cov_xx_inv <- solve(cov(x, x) + epsilon * diag(1, length(x)))
Ef <- cov(X, x) %*% cov_xx_inv %*% y
Cf <- cov(X, X) - cov(X, x)  %*% cov_xx_inv %*% cov(x, X)

And Ef and Cf are the estimated (krieging) mean and covariance.

Priors

Note: it appears that the mixed gammas for d and nug actually treat the second argument, g1 and g2, as the rate instead of the scale, so I invert the desired scale:

s2.p <- c(50,50)
tau2.p <- c(20,1)
d.p = c(10, 1/0.01, 10, 1/0.01)
nug.p = c(10, 1/0.01, 10, 1/0.01)

Define as curves and plot the priors:

s2_prior <- function(x) dinvgamma(x, s2.p[1], s2.p[2])
tau2_prior <- function(x) dinvgamma(x, tau2.p[1], tau2.p[2])
d_prior <- function(x) dgamma(x, d.p[1], scale = d.p[2]) + dgamma(x, d.p[3], scale = d.p[4])
nug_prior <- function(x) dgamma(x, nug.p[1], scale = nug.p[2]) + dgamma(x, nug.p[3], scale = nug.p[4])
beta0_prior <- function(x, tau) dnorm(x, 0, tau)

xx <- seq(.0001, 5, length.out=100)
priors <- data.frame(x = xx, s2 = s2_prior(xx), tau2 = tau2_prior(xx), beta0 = beta0_prior(xx, 1), nug = nug_prior(xx), d = d_prior(xx))
priors <- melt(priors, id="x")
ggplot(priors) + geom_line(aes(x, value)) + facet_wrap(~variable, scale="free")

plot of chunk unnamed-chunk-5

gp <- bgp(X=x, XX=X, Z=y, verb=0,
          meanfn="constant", bprior="b0", BTE=c(2000,6000,2), m0r1=FALSE, 
          corr="exp", trace=TRUE, beta = 0,
          s2.p = s2.p, d.p = d.p, nug.p = nug.p,
          s2.lam = "fixed", d.lam = "fixed", nug.lam = "fixed", 
          tau2.lam = "fixed", tau2.p = tau2.p)

Extract the posterior Gaussian process mean and the standard deviations over the predicted grid from the fit:

V <- gp$ZZ.ks2
tgp_dat <- data.frame(x   = gp$XX[[1]], 
                  y   = gp$ZZ.km, 
                 ymin = gp$ZZ.km - 1.96 * sqrt(gp$ZZ.ks2), 
                 ymax = gp$ZZ.km + 1.96 * sqrt(gp$ZZ.ks2))

and likewise for our manual data:

manual_dat <- data.frame(x = X, 
                         y = Ef, 
                         ymin = (Ef - 1.96 * sqrt(diag(Cf))), 
                         ymax = (Ef + 1.96 * sqrt(diag(Cf))))

Compare the GP posteriors:

ggplot(tgp_dat) +
    geom_ribbon(aes(x, y, ymin = ymin, ymax = ymax), fill="red", alpha = .1) + # Var
    geom_line(aes(x, y), col="red") + # mean
    geom_point(data = data.frame(x = x, y = y), aes(x, y)) + # raw data
    geom_ribbon(dat = manual_dat, aes(x, y, ymin = ymin, ymax = ymax), fill = "blue", alpha = .1) + # Var
    geom_line(dat = manual_dat, aes(x, y), col = "blue")  + #MEAN    
    theme_bw() + theme(plot.background = element_rect(fill = "transparent",colour = NA))

plot of chunk gp-plot

Posteriors of hyperparameters

require(reshape2)
hyperparameters <- c("index", "s2", "tau2", "beta0", "nug", "d", "ldetK")
posteriors <- melt(gp$trace$XX[[1]][,hyperparameters], id="index")
ggplot(posteriors) + geom_histogram(aes(value)) + facet_wrap(~variable, scales="free")

plot of chunk unnamed-chunk-9

Github Issues Tracker: The Perfect Research Todo List

2012-12-06T00:00:00-08:00

Github issues tracker has increasingly become my research to-do list. Far beyond bugs and features of the code associated with the project, the issues sign-post different directions for investigation and the progress I’ve made in each. Tags serve to group issues related to a common sub-project (as in my pdg-control) repo or priotize tasks (as in my nonparametric-bayes repo.

Issues not only have title and tags, but support a comment thread for progress and discussion of the issue. Thanks to github-flavored-markdown, issues can reference each other simply by number, and can be updated or closed automatically by mentioning the issue number in a commit.

Issues can also be grouped into shared deadlines, or milestones; a feature I haven’t fully exploited (but see our ews-review paper). In any collaborative project the issues can be assigned to different people, (though currently this requires they have a Github account).

A consequence of this workflow is a conveniently numbered, color-coded and cross-linked collection of steps involved in a given research project. This tends to be a higher-level overview than the individual commit log, particularly as I often use commits to track multiple runs with different parameters, or move code across to different supercomputers that do the actual runs.

Example from closed issues on pdg-control

I am still figuring out the right level or “resolution” on which to create and track issues.¹ On one extreme, almost every commit could be seen as resolving an issue. The ideal use for me is probably nearer the other extreme, where individual issues are rather big-picture, and may be referenced by many commits. Perhaps the right way to think about it is that the questions addressed by resolving an issue are on the level of what is interesting to others, while changes in individual commits are more for me. Hopefully I get better at finding this relevant level.

Another nice feature of issues is that they can be closed when a particular line of investigation hits a dead-end, or stalls, or when the problem is resolved. Unlike the resulting paper from an investigation which essentially summarizes the issues that were closed successfully, the issues tracker also reveals the dead-ends, as well as those issues that were not closed a time of publication (but perhaps left to “further research”). Hopefully I will have some decent examples of this in the repositories accompanying my next papers.

Unlike issues, commits do not have a native tag structure (so-called “tags” mark important events in the commit history rather than grouping common commits). So at least this would group commits by the tag of the associated issue.↩

Notes

2012-12-05T00:00:00-08:00

pdg-control Call

Send Figure 3 and 4 with stock dynamics to Paul.

Detailed to do items coming out of talk filed as Github Issues for pdg-control (policycosts)

Comment piece

Round 1 revisions sent to Lucy and Alan.

Workflow Workshop

At Simone’s invitation, I’m running a little session on workflow in git/Github, Rstudio, and knitr for the Mangel lab group. A quick brainstorm for an outline:

A first tutorial

Create a new repository on Github
launch an RStudio session, select create new project
Add the git@ url to the project, select a local directory, hit “create”
In Rstudio, create new R Markdown file.
Editing the file. Hit “Knit HTML” button.
add the files to git: git tab, check boxes.
commit the .Rmd and md files, and push files to Github.
Viewing files on github.
Navigating Github: Project directory, viewing .md files, raw mode, viewing history of a file, getting permanent links to specific versions.
10: Github extras: using issues. (Referencing issues in commits)
Recovering a deleted file.

Git More Advanced

Create a gh-pages branch and add the html generated from the Rstudio/knitr. (Covers branching & webpage hosting on github)
Create a github homepage
Merging changes, resolving conflicts.
collaborating via github.
why sha hashes are awesome
private repositories
Repo Readmes

Tgp Mcmc For Gaussian Processes

2012-12-04T00:00:00-08:00

Exploring implementation through tgp package MCMC routine and background reading (manual, vignettes, thesis).

Example call:

gp <- bgp(X=X, XX=XX, Z=Z, meanfn="constant", bprior="b0", BTE=c(1000,6000,2), m0r1=TRUE, verb=4, corr="exp", trace=TRUE, s2.p=c(5,10), tau2.p=c(5,10), s2.lam="fixed", tau2.lam="fixed")

Verbose return:

n=39, d=1, nn=101
BTE=(1000,6000,2), R=1, linburn=0
RNG state RK using rk_seed
preds: data krige
T[alpha,beta,nmin,smin,bmax]=[0,0,10,1,1]
mean function: constant
beta prior: b0 hierarchical
s2[a0,g0]=[5,10]
s2 prior fixed
tau2[a0,g0]=[5,10]
tau2 prior fixed
corr prior: isotropic power
nug[a,b][0,1]=[1,1],[1,1]
nug prior fixed
gamlin=[0,0.2,0.7]
d[a,b][0,1]=[1,20],[10,10]
d prior fixed

The parameters in this run are constant across the trace points (because hyperparameters are fixed).

X is the observed X values (or matrix of appropriate dimension)
XX the desired predicted grid. (will also predict on X)
Z is observed Y values (or matrix of appropriate dimension)
m0r1 means scale data to mean zero and unit range
bprior is the prior distribution, defaulting to bfalt. b0 is a Gaussian. This prior is for the Gaussian process and is not to be confused with the hyperpriors describing the distribution of various parameters.
BTE Burn-in B steps, Terminate after T steps, sample Every E steps.
verb Very verbose output
Other options (for adaptive/query learning, etc, igored for now. Focus on priors).
The priors for the hyperparameters are given by s2.p, tau.p etc (both are Inverse Gammas). Their parameters can vary hiearchically in general, but we hold them fixed, s2.lam = 'fixed', etc.)
s2 is in Gramacy and in Rassmussen and Williams, is the scale of the noise. Comes from an inverse gamma prior. Set by inverse-gamma parameters a0 and g0.
tau2 Also comes from an inverse gamma prior. It is also set by inverse-gamma parameters a0 and g0.

We keep the heirachical priors fixed in the tau.lm, s2.lm, etc. nug fixed by default. Note this results in the the priors from which these are sampled are fixed: e.g. gp$trace$hier$s2.a0, gp$trace$hier$s2.g0
nug.p This is the “nugget,” the measurement error. Comes from a mixture of gamma prior parameter (initial values) for the nugget parameter c(a1,g1,a2,g2) where g1 and g2 are scale (1/rate) parameters. Default reduces to simple exponential prior. Specifying nug.p = 0 fixes the nugget parameter to the “starting” value in gd[1], i.e., it is excluded from the MCMC
index is the sampling point (from BTE, we see we sample starting at step 1000 and ending at step 6000, recording every 2 steps, so there are index points).
lambda possibly a mixing parameter in the importance sampler temperature??
beta0 No mention in the manual
d The parameter for priors using the hierachical exponential distribution for the parameters a1, a2, g1, g2.
b ??
Where are the hyperparameters for the correlation function (e.g. the length-scale for the Gaussian??)

Traces

pars <- melt(gp$trace$XX[[1]], id = "index")
ggplot(pars, aes(index, value)) + geom_line() + facet_wrap(~variable, scales="free_y")

plot of chunk unnamed-chunk-5

It is possible to calculate summary statistics and check out the distribution of these fellows.

mean(gp$trace$XX[[1]][,"nug"])

[1] 0.009755

ggplot(pars, aes(value)) + geom_histogram() + facet_wrap(~variable, scales="free")

plot of chunk unnamed-chunk-6

Extracting the estimated process

(The MAP, maximum a posteriori expected mean and variance).

Extract the posterior Gaussian process mean and the standard deviations over the predicted grid from the fit:

V <- gp$ZZ.ks2
dat <- data.frame(x   = gp$XX[[1]], 
                 y    = gp$ZZ.km, 
                 ymin = gp$ZZ.km - 1.96 * sqrt(gp$ZZ.ks2), 
                 ymax = gp$ZZ.km + 1.96 * sqrt(gp$ZZ.ks2))

Plot the posterior Gaussian Process:

  p1 <- ggplot(dat) +
    geom_ribbon(aes(x=x,y=y, ymin=ymin, ymax=ymax), fill="grey80") + # Var
    geom_line(aes(x=x,y=y), size=1) + #MEAN
    geom_point(data=gp$obs,aes(x=x,y=y)) + 
    labs(title=paste("llik =", prettyNum(gp$llik)))
  if(!is.null(true))
    p1 <- p1 + geom_line(data = true, aes(x, y), col = "red", lty = 2) 
p1 + theme_notebook

plot of chunk gp-plot

Find the actual optimum policy from the true parametric model

x_grid <- dat$x
h_grid <- x_grid
matrices_true <- f_transition_matrix(f, p, x_grid, h_grid)
opt_true <- find_dp_optim(matrices_true, x_grid, h_grid, 20, 0, profit, delta=.01)

Estimate a policy from the Gaussian Process:

rownorm <- function(M) t(apply(M, 1, function(x) x/sum(x)))
matrices_gp <- lapply(h_grid, function(h){
    S <- dat$y - h
    F_ <- t(sapply(1:length(S), function(i){
      if(S[i]>0) {
        out <- dlnorm(x_grid/S[i], 0, V[i])
      } else {
        out <- numeric(length(x_grid))
        out[1] <- 1
        out
      }
    }))
    F_ <- rownorm(F_)
  })
opt_gp <- find_dp_optim(matrices_gp, x_grid, h_grid, 20, 0, profit, delta=.01)

and plot

policies <- melt(data.frame(stock=x_grid, GP = x_grid[opt_gp$D[,1]], Exact = x_grid[opt_true$D[,1]]), id="stock")
policy_plot <- ggplot(policies, aes(stock, stock - value, color=variable)) +
  geom_point() + xlab("stock size") + ylab("escapement") 
policy_plot + theme_notebook

plot of chunk policy_plot

We can see what happens when we attempt to manage a stock using this:

z_g <- function() rlnorm(1,0, sigma_g)
set.seed(1)
sim_gp <- ForwardSimulate(f, p, x_grid, h_grid, K, opt_gp$D, z_g, profit=profit)
set.seed(1)
sim_true <- ForwardSimulate(f, p, x_grid, h_grid, K, opt_true$D, z_g, profit=profit)

df <- data.frame(time = sim_gp$time, stock_gp = sim_gp$fishstock, stock_true = sim_true$fishstock, harvest_gp = sim_gp$harvest, havest_true = sim_true$harvest)
df <- melt(df, id="time")
simplot <- ggplot(df) + geom_line(aes(time,value, color=variable))
simplot + theme_notebook

plot of chunk simplot

Figures 3 And 4 For Policycosts

2012-12-03T00:00:00-08:00

Straight-forward tasks can sometimes take forever. In advance of our upcoming conference call for the Pretty Darn Good Control (pdg-control) NIMBioS working group, I had a few remaining issues to touch up in Figures 3 and 4.

Figure 3 shows harvest and stock dynamics under the different functional forms fo the cost. The goal is to provide a very intuitive, visual sense of how the policies differ under these different functional forms, such as a fixed “transaction fee” (fixed) or a cost that is proportional to the size of the change in policy (L1), or the square of that change (L2), etc. I had already cobbled together the code for this, but simply had to run over the same set of replicates and pick a window length appropriate to see the differences. The trick was make sure we were comparing functional forms that had the same overall net impact on the Net Present Value of the stock. In this way, the differences observed are due only to differences in the form of the cost, and not the total magnitude of the cost. Figure 2 already provided a way to do this scaling, I simply needed to back out the scaling programmatically.

Finding these “apples-to-apples” levels is something of a computational brute force excercise, computing the costs along a grid of coefficients for each method. Otherwise the idea is pretty straight forward. One trick was being able to use the policies calculated in this step to generate the Figure 3 simulations, rather than recalculating the policy once the right scaling value c2 is known.

Figure 4 provides a more quantitative way to look at the comparison. I had already created a version of this graph that showed how variance and autocorrelation in the policy and state variable dynamics changed as the total magnitude of the cost increased. Unfortunately this suffered from the apples-to-oranges problem for comparing between functional forms. The x-axis needed to be calculated in terms of the reduction of the NPV, rather then the coefficients (c2, which mean different things in the different forms), for a proper apples-to-apples comparison.

Okay, straight forward exercise. 47 commits and 19 hours later, I have what I need. And it is 5am. Oh well, this post is still going down as “Monday”.

Figure 3

Stock (state variable) dynamics

Harvest (control variable) dynamics

Figure 4

Stock (state variable) dynamics

Harvest (control variable) dynamics

full knitr file here

These are calculated with c0 = 30. There’s corresponding figures when fishing effort itself is cost-free, c0 = 0. See costfree.md. Looks like the c2 range needs to be extended for this run.

Figure 2

The apples-to-apples illustration is also generated in this script.

Misc notes

Caching is pretty helpful in fiddling with these slow computations. Sometimes it helps to copy the cache of the intensive part of a run over from the supercomputer to the local machine, and load the whole cache into an interactive R session. Loading the whole cache can be done as such:

e <- environment()
files <- gsub(".rdb", "", system('ls *.rdb', intern=TRUE))
sapply(files, lazyLoad, env=e)

providied the target chache directory doesn’t have caches of different commits.

Note On Notebook Figures

2012-11-30T00:00:00-08:00

plots for the notebook

I have started uploading images to my own server, rather than flickr. Programmatically, this is just a matter of switching the knitr upload function:

opts_knit$set(upload.fun = socialR::notebook.url)

from my custom flickr.url function (a wrapper to Rflickr) to a custom function notebook.url using a system call to cp (to my local jekyll notebook) and rsync (so the figure displays before my next jekyll push).

The new script names image by date and the first 10 characters of the sha hash, as well as the knitr chunk name. This way multiple runs tweaking an individual file are not recorded as separate files if no commits are made to the repository, but every commit is guaranteed to link to a unique set of figures (unless the chunk is loaded from cache and does not generate a new image). All figures also appear sorted by date on the webserver in the manner. I could probably put some lightweightphotobrowser on the server, if browsing images serves any purpose.

The real reason for the change is to maintain the images in my labnotebook directory, such that the directory is a complete archive that can exist independent of web services such as flickr. The images aren’t included in the git management for obvious reasons. My current archiving on figshare has simple involved zipping up the markdown versions of all my posts by year (in _posts). If I continue in this way, I should need to archive the figures/ directory along with it. Perhaps I should archive the entire Jekyll directory instead.¹

An alternative strategy would be to upload each figure to figshare directly in the same way I was doing with flickr. While this has the advantage of clockss archiving, mostly I’m not sure that I should flood my figshare account in this way. I really don’t need a different doi for every attempt to create a figure (just as I’ve decided not to archive each notebook entry as a separate figshare object.)

Plot appearance (transparency) in the notebook

It’s not easy to generate figures that display nicely in both dark and light themes (well, I should just tell ggplot to use a solarized set of theme colors, but I’m too lazy to write that). Meanwhile, just setting the transparent background to the plot, and transparent grid lines against the grey boxes seems to do pretty well:

Example transparent plot

This is accomplished with the codeblock below. Note that we need to tell knitr chunks to set the background of the png device to transparent, as well as telling ggplot what we want to be transparent:

opts_chunk$set(dev.args=list(bg="transparent"))
theme_notebook <- 
  theme_grey() + 
  theme(
    plot.background = element_rect(fill = "transparent", colour = NA),
    panel.grid.major = element_line(colour = "transparent")),
    panel.grid.minor = element_line(colour = "transparent"))
theme_set(theme_notebook)

(It appears this makes the gridlines invisible, since they appear over the grey panel.background. For now, best to leave them white, which shows up against the grey background in any event).

One challenge this introduces is private posts (such as the reviews I write in peer review, which I sign but cannot publicly disclose). Currently these are ignored by github (.gitignore) and by Jekyll (exclude: in _config.yml). I suppose I could likewise simply tell tarball to ignore _posts/private.↩

early results in nonparametric-bayes approaches to optimal policy

2012-11-28T00:00:00-08:00

Resolving Numerical and Computational issues

Sequential approach? (issue #4)

Thinking through whether or not it is worth optimizing the computational performance of the sequential approach.

Can it be demonstrated that this actually has higher numerical stability? Alternatively, can it be demonstrated that this is (or is not) equivalent to the calculation performed by the direct inversion? What is the difference in computational complexity between these approaches? e.g. the inverse ends up effectively O(n^3), but what is the recursion?

If we assume measurement noise, is stability an issue? e.g. introducing such small noise or jitter apparently a standard method to address stability; see pg 12 and citations there-in (Matheron, 1963; Cressie, 1991, Neal, 1997).

How much accuracy is lost with reasonable grids? Compared to adding jitter?

Is there any evidence of use for this approach in the substantial GP literature of computational experts?

With large data sets, the determinant does get close to zero, but solutions appear stable with direct (DGESV) inversion. See issue #4 for a discussion.

seeing if I get singular matrices in the inversion 09:55 pm 2012/11/28
print determinant value 10:00 pm 2012/11/28

Choleski, (issue #2)

basic example comparing methods 08:10 pm 2012/11/28
choleski approach using backsolve 05:49 pm 2012/11/28

Some explaining to do here.

Hyperparameter optimization (issue #3)

Added necessary hyperparameter for variance in the covariance kernel , in kernel . Alternatively should be scaling data s.t. variance is unity. Manually tuning this does allow for better fits. Need to try optimization still. Have added a method to condition on the function going through (0,0) with negligible error, currently in a somewhat direct/inelegant code (and potential for numerical instability).

Reed example using tau not set to 1 and conditioning on going through (0,0) 05:34 pm 2012/11/28
allow for tau as hyperparameter 02:55 pm 2012/11/28

Now getting reasonable performance of the Gaussian process relative to the optimal policy (also see earlier examples below). Comforting to see that the Gaussian Process results in slightly larger uncertainty and manages slightly more conservatively (higher standing stock, lower harvests). Very conservative outside range of data – resulting in a policy function deviating strongly from Reed’s constant escapement.

plot of chunk unnamed-chunk-2

plot of chunk policy_plot

plot of chunk simplot

Functionalize package

Existing methods now all implemented in package function gp_fit, and method for SDP via GP in gp_transition_matrices.

as before but with kernlab fit (underestimates lengthscale) 02:16 pm 2012/11/28
Identical to previous reed-example commit but with kernlab engine (not fit) 02:14 pm 2012/11/28
Example of direct method using subroutines from R/ 02:11 pm 2012/11/28

Different GP fitting routines can all be called with gp_fit and plotted with plot.gpfit. transition matrices for GP can be estimated with gp_transition_matrices.

See example use in reed-example.R 01:48 pm 2012/11/28

Policy Costs Work

2012-11-27T00:00:00-08:00

Scripts to estimate apples to apples comparisons for different penalty functions and plot instances of the resulting policy. Example run with fixed linear costs on effort.

Double the actual variance leads to terminal overfishing 11:16 am 2012/11/28
use fixed seed. 09:45 am 2012/11/28
Compare GP to Reed optimum for BH (successful run this time) 09:37 am 2012/11/28
basic GP example in BH system, compared to Reed Optimum 09:17 am 2012/11/28

Content of current working example, run on server zero. See commit 942b2faa91

seed <- 123  # Random seed (replicable results)
delta <- 0.05  # economic discounting rate
OptTime <- 100  # stopping time
gridsize <- 50  # grid size for fish stock and harvest rate (discretized population)
sigma_g <- 0.2  # Noise in population growth
reward <- 0  # bonus for satisfying the boundary condition
z_g <- function() rlnorm(1, 0, sigma_g)  # mean 1
z_m <- function() 1  # No measurement noise,
z_i <- function() 1  # No implemenation noise
f <- BevHolt  # Select the state equation
pars <- c(1.5, 0.05)  # parameters for the state equation
K <- (pars[1] - 1)/pars[2]  # Carrying capacity (for reference
xT <- 0  # boundary conditions
x0 <- K
profit <- profit_harvest(price = 10, c0 = 30, c1 = 0)
x_grid <- seq(0.01, 1.2 * K, length = gridsize)
h_grid <- seq(0.01, 0.8 * K, length = gridsize)

SDP_Mat <- determine_SDP_matrix(f, pars, x_grid, h_grid, sigma_g)
opt <- find_dp_optim(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, delta, reward = reward)

L1 <- function(c2) function(h, h_prev) c2 * abs(h - h_prev)
free_increase <- function(c2) function(h, h_prev) c2 * abs(min(h - h_prev, 0))  # increasing harvest is free
free_decrease <- function(c2) function(h, h_prev) c2 * max(h - h_prev, 0)  # decreasing harvest is free
fixed <- function(c2) function(h, h_prev) c2 * as.numeric(!(h == h_prev))
L2 <- function(c2) function(h, h_prev) c2 * (h - h_prev)^2
none <- function(h, h_prev) 0
penaltyfns <- list(L2 = L2, L1 = L1, free_decrease = free_decrease, fixed = fixed, 
    free_increase = free_increase)
c2 <- seq(0, 10, length.out = 30)

Apples to Apples levels

This can take a while, so we use explicit parallelization,

require(snowfall)
sfInit(cpu = 8, parallel = T)

R Version:  R version 2.15.2 (2012-10-26)

sfLibrary(pdgControl)

Library pdgControl loaded.

sfExportAll()

Loop over penalty functions and magnitudes

policies <- sfSapply(penaltyfns, function(penalty) {
    policies <- sapply(c2, function(c2) {
        policycost <- optim_policy(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, 
            delta, reward, penalty = penalty(c2))
        i <- which(x_grid > K)[1]
        max(policycost$penalty_free_V[i, ])
    })
})

Note that optim_policy has been updated to return the equilibrium value of profits from fish harvests before the adjustment costs have been paid, penalty_free_V. This containst the values for all possible states, we simply evaluate it at the carrying capacity (which is our initial condition.) The index in x_grid that corresponds to the carrying capacity (initial condition) i indicates this.

Quadratic costs on fishing effort have to be done separately,

quad <- sapply(c2, function(c2) {
    effort_penalty = function(x, h) 0.1 * c2 * h/x
    policycost <- optim_policy(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, 
        delta, reward, penalty = fixed(0), effort_penalty)
    i <- which(x_grid > K)[1]
    max(policycost$penalty_free_V[i, ])  # chooses the most sensible harvest in t=1
})
dat <- cbind(policies, quad)

Tidy up the data and plot the net present value (before the penalty has been paid) relative to that achieved when managed without a penalty.

npv0 <- dat[1, 3]
npv0

free_decrease 
        709.7

dat <- data.frame(c2 = c2, dat)
dat <- melt(dat, id = "c2")
ggplot(dat, aes(c2, value, col = variable)) + geom_point() + geom_line()

plot of chunk npv-plot

Find the value of c2 that brings each penalty closest to 75% of the cost-free adjustment value:

ggplot(dat, aes(c2, (npv0 - value)/npv0, col = variable)) + geom_point() + geom_line()

plot of chunk apples_plot

closest <- function(x, v) {
    which.min(abs(v - x))
}
dt <- data.table(dat)
index <- dt[, closest(0.25, (npv0 - value)/value), by = variable]
apples <- c2[index$V1]
names(apples) = index$variable
apples

           L2            L1 free_decrease         fixed free_increase 
        1.034         1.724         0.000         8.276         0.000 
         quad 
        1.034

Results

Solve the policy cost for the specified penalty function

L2_policy <- optim_policy(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, delta, 
    reward, penalty = L2(apples["L2"]))

fixed_policy <- optim_policy(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, delta, 
    reward, penalty = fixed(apples["fixed"]))

L1_policy <- optim_policy(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, delta, 
    reward, penalty = L1(apples["L1"]))

free_increase_policy <- optim_policy(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, 
    delta, reward, penalty = free_increase(apples["free_increase"]))

free_decrease_policy <- optim_policy(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, 
    delta, reward, penalty = free_decrease(apples["free_decrease"]))

We also compare to the case in which costs of harvesting increase quadratically with effort; a common approach to create smoother policies.

quad_profit <- profit_harvest(price = 10, c0 = 30, c1 = apples["quad"])
quad_costs <- optim_policy(SDP_Mat, x_grid, h_grid, OptTime, xT, quad_profit, 
    delta, reward, penalty = none)

sims <- list(L1 = simulate_optim(f, pars, x_grid, h_grid, x0, L1_policy$D, z_g, 
    z_m, z_i, opt$D, profit = profit, penalty = L1(apples["L1"]), seed = seed), 
    L2 = simulate_optim(f, pars, x_grid, h_grid, x0, L2_policy$D, z_g, z_m, 
        z_i, opt$D, profit = profit, penalty = L2(apples["L2"]), seed = seed), 
    fixed = simulate_optim(f, pars, x_grid, h_grid, x0, fixed_policy$D, z_g, 
        z_m, z_i, opt$D, profit = profit, penalty = fixed(apples["fixed"]), 
        seed = seed), increase = simulate_optim(f, pars, x_grid, h_grid, x0, 
        free_increase_policy$D, z_g, z_m, z_i, opt$D, profit = profit, penalty = free_increase(apples["increase"]), 
        seed = seed), decrease = simulate_optim(f, pars, x_grid, h_grid, x0, 
        free_decrease_policy$D, z_g, z_m, z_i, opt$D, profit = profit, penalty = free_decrease(apples["decrease"]), 
        seed = seed), quad = simulate_optim(f, pars, x_grid, h_grid, x0, free_decrease_policy$D, 
        z_g, z_m, z_i, opt$D, profit = quad_profit, penalty = none, seed = seed))

# Make data tidy (melt), fast (data.tables), and nicely labeled.
dat <- melt(sims, id = names(sims[[1]]))
dt <- data.table(dat)
setnames(dt, "L1", "penalty_fn")  # names are nice

Plots

p0 <- ggplot(dt) + geom_line(aes(time, alternate), col = "grey20", lwd = 1) + 
    geom_line(aes(time, fishstock, col = penalty_fn, lty = penalty_fn)) + labs(x = "time", 
    y = "stock size", title = "Stock Dynamics")
p0

plot of chunk p0

p1 <- ggplot(dt) + geom_line(aes(time, alternate), col = "grey20", lwd = 1) + 
    geom_line(aes(time, fishstock), col = rgb(0, 0, 1, 0.8)) + facet_wrap(~penalty_fn) + 
    labs(x = "time", y = "stock size", title = "Stock Dynamics")
p1

plot of chunk p1

p2 <- ggplot(dt) + geom_line(aes(time, harvest_alt), col = "grey20", lwd = 1) + 
    geom_line(aes(time, harvest), col = rgb(0, 0, 1, 0.8)) + facet_wrap(~penalty_fn) + 
    labs(x = "time", y = "havest intensity (fish taken)", title = "Harvest Policy Dynamics")
p2

plot of chunk p2

Citing Lab Notebook Entries

2012-11-23T00:00:00-08:00

C. Titus Brown has an excellent post discussing his exploration into the merits and technicalities cross-posting his blog posts to figshare. The rfigshare package written by Ted Hart and myself can do just this, once we puzzled out some of the same challenges (Notes to Titus: though the documentation doesn’t mention it, you can get a programmatic list of available categories from http://api.figshare.com/v1/categories. Figshare can also take code as a fileset or dataset, and may soon add a type for it. The coolest thing about figshare is perhaps the way they add types in response to how they see users using the service).

The real discussion, though, is not about how, but why? Currently figshare doesn’t render the html or markdown/restructured-text source (other than as plain text), so it’s not a great place to read the posts. This may change in the near future as well, but the primary motivation for doing this seems to be on the ability to get an honest-to-goodness DOI for your entry. So why a DOI?

The answer most often put forward is that “a DOI facilitates citing a paper in the formal literature.” I think that’s not entirely accurate. For one, the formal literature has no trouble citing things that do not have a DOI. Perhaps the answer is meant to mean track citations to my content, e.g. for statistical/impact purposes. This may be closer to the mark, but it still needs work. What counts depends on whose doing the counting. Anyone browsing the citation counts of a work through different mediums has surely encountered Thompson-Reuters has very specific criteria to be included, so I don’t believe they are tracking citations to anything with a DOI. Nor, I think is Scopus. Perhaps the target is Google then?

If you want Google Scholar to count the number of citations to your blog post, I have good news for you. You do not need a DOI and cross-posting to figshare. You just need to follow the metadata requirements as outlined by Google Scholar itself. This will help Google Scholar identify your blog post as a academic object, and add it to your profile. Anything indexed by Google scholar that cites your blog post will count as a reference. The basics are really simple. Have metadata such as:

<meta name="resource_type" content="Lab Notebook"/>
<meta name="citation_journal_title" content="Lab Notebook"/>
<meta name="citation_publication_date" content="2012-11-23 00:00:00 -0800"/>
<meta name="citation_date" content="2012-11-23 00:00:00 -0800"/>
<meta name="citation_author" content="Carl Boettiger"/>
<meta name="citation_title" content="Citing Lab Notebook Entries"/>

Just replace Lab Notebook etc. with the correct values for your site. The example above will do so automatically if you use Jekyll using information specified in the YAML header. It seems the entry also needs a section called either “References” or “Bibliography” followed by a list of references. Not sure what format Google Scholar will do best at disambiguating. (Note that what I refer to as “Google Scholar” metadata looks like it was originally the convention adopted by HireWire Press.)

Phil Lord and colleagues have written the fantastic API called greycite [Lord 2012], which can be used to generate citation data from any website that has semantic markup (including HTML5 semantics, also Dublin Core or OpenGraph ontologies and Google Scholar metadata). The API can return citations formatted in bibtex or JSON. Try it out at greycite.knowledgeblog.org

Now there is another reason to use DOIs and figshare, which is, I think, entirely independent – archival preservation (decoupled journal, anyone?). DOIs are potentially more permanent than URLs (though I’ll leave that debate to the experts). Figshare content is backed up by CLOCKSS. Forever is a long time, but this certainly sounds like a better archiving strategy. Figshare should be providing the necessary Google Scholar metadata on each object now. My current practice has been to archive my notebook in annual chunks on figshare (posting entries individually feels fragmented and cluttered to me), and rely on the more native web solution of HTML metadata to allow my entries to “citable”. Perhaps this is not ideal, but for the short term, the content is discoverable and citable via Google Scholar, which points to my address. If my site vanishes from the web, one might hope that an academic search for my lab notebook might recover the content from the archived version.

Edit: thoughts, examples and disclaimer

I stumbled across Google’s indexing of some of my lab notebook entries somewhat by accident. For instance, one of my entries on optimal control in which I had listed some references started turning up in Google Scholar Being able to engage in a scholarly exchange through blogs that can cite and be cited by the formal literature certainly sounds like an important step forward in generating new publishing models. Of course it is also obvious that the procedure I have outlined could be trivially exploited for some rather blatant gaming of Google Scholar’s citation statistics. This post certainly is not an endorsement of gaming such statistics. If nothing else, perhaps this once again underscores the weakness of reliance on metrics of quantity when trying to infer quality. Will we ever let go of that fallacy?

References

Titus Brown (2012) Posting blog entries to figshare, Living in an Ivory Basement. http://ivory.idyll.org/blog/posting-blog-entries-to-figshare.html
Phillip Lord (2012) Greycite. Knowledge Blog. http://knowledgeblog.org/greycite
Carl Boettiger (2011) Optimal Control examples continued: Bellman, Dynamic Programming, Lab Notebook http://www.carlboettiger.info/2011/10/28/optimal-control-examples-continued-bellman-dynamic-programming.html

Notes

2012-11-22T00:00:00-08:00

multiple-uncertainty

Working out the multiple uncertainty computational performance and noise forms. Fixing a few things in the way log-normal noise was calculated. Uniform noise in particular can still give rather non-smooth policy, needs a bit of digging. Uniform noise doesn’t show the non-monotonicity of the log-normal noise in measurement uncertainty though.

Uniform

uniform noise policy functions

lognormal

log-normal noise policy functions

log

A few small tweaks to probability calculations

Handle the case of mu = grid zero, not just exactly zero, to avoid introducing NAs
Calculate dlnorm as (x_grid/mu, 0, sigma), rather than as (x_grid, mu, sigma).
Transpose of measurement error in M %*% F. Because now we want to treat x as given, integrate over range of y? (no, probably not?)
standardize noise level between old and g 12:52 pm 2012/11/22

With transposed M time F. (include image) 01:04 pm 2012/11/22
And now without transpose, M %*% F 01:07 pm 2012/11/22
Larger (log-normal in all variables) noise, weirder results. 01:39 pm 2012/11/22
Changes improving performance of F calculation (rate-limiting step)

vectorized calculation of mu.
matrix-based calc of deterministic part of growth rate 02:13 pm 2012/11/22

Another efficiency change. Not identical but appears to be a decent approximation: Snap mu to the x_grid, and look up the pdfn value rather than calculating it each time in F. 03:00 pm 2012/11/22

Now pretty efficient. Matrix multiplication is dominant time sink, followed by the applys. snap_to_grid is probably the slowest functional contribution.

Notes

2012-11-21T00:00:00-08:00

writing comment piece. writing writing writing.

Also today (misc):

Check in with Peter re: Labrids paper
Fix issue #32 rfigshare
Follow up with Nina about improved FishBase API
pay ESA and SSE dues
Santa Cruz SOE email lists
uniform growth noise version on corrected multiple-uncertainty run
transparent plots in new ggplot theme? Set in dev.args=list(bg="transparent")
Consider SOE AMS lecturer pool? Description, Application
CV, Letter of application with course list.
Optional: Teaching evaluations. Sample course material.

multiple uncertainty

Whoops, not preventing negative harvest values can lead to some weird results. 06:16 pm 2012/11/21
numerical instabilities in uniform noise case? 04:01 pm 2012/11/21
notes on policycost manuscript 01:34 pm 2012/11/21

Sequential Updating

2012-11-19T00:00:00-08:00

Fixed the sequential updating algorithm (avoids numerical instabilities associated with matrix inversion). Currently defined as the function recursion:

for the covariance:

C_seq <- function(X, X_prime, i){
    if(i <= 1)
          cov(X, X_prime) - mmult(cov(X,x[i]), cov(x[i], X_prime)) / as.numeric( cov(x[i], x[i]) + sigma_n^2)
            else
                  C_seq(X, X_prime,   i-1) - mmult(C_seq(X,x[i],   i-1), C_seq(x[i], X_prime,   i-1)) / as.numeric( C_seq(x[i], x[i],   i-1)  + sigma_n^2  )
}

for the mean:

mu_seq <- function(X, i){
    if(i <= 1)
          cov(x[i], X) * (y[i]-mu[i]) / as.numeric( cov(x[i], x[i]) + sigma_n^2)
            else
                  mu_seq(X, i-1) + C_seq(x[i], X,  i-1) * (y[i]-mu[i]) / as.numeric( C_seq(x[i], x[i], i-1)  + sigma_n^2 )
}

Note that this needed a custom matrix multiplication to handle scalar by vector cases:

mmult <- function(x,y){
  if(length(x) == 1){
    x <- as.numeric(x) 
    x * y
  } else if(length(y) == 1){ 
    y <- as.numeric(y)
    x * y
  }  else 
  x %*% y
}

And probably needs to be written in C at least to have any hope of scaling. See implementation in sequential-method

Match is perfect (jittered to avoid overplotting the lazy way):

Sequential vs direct calculation of the mean

Now should rewrite to use a grid for the covariance matrices of each data point. Memory intensive but should be much faster than function recursion.

(Should also confirm variance calculation in comparison, and test on larger grid sizes)

Multiple uncertainty corrected

2012-11-16T00:00:00-08:00

Fixed the multiple uncertainty calculation implementation. Code had transposed , also one of the matrices. Should still confirm final implementation. In the below example each case has log-normal growth noise present. The non-monotonic section of the measurement uncertainty

See notes on code changes inline.

Additional notes from walking through algoritm step by step.

index in the true state space
index in the observed state space
index in the true action
index in the policy/decision space
the Expected value of policy and observed state
the profit of a harvest of given a true stock of
is the probability of a true state given an observed stock . (How does this relate to the probability the observed stock is given the true stock is ? Should be the transpose? Is symmetric?)
is the probability of implementing a harvest , given a quota set to .
the Expected profit from a policy and observed state . In Einstein summation,

Misc

Notes

2012-11-15T00:00:00-08:00

Stuck on many things

Have not correctly defined the recursion in the sequential updating approach. Can I really calculate in place of ? e.g. instead of calculating:

Is there some scalar such that

e.g. a term like:

and a similar recursion for the mean?

Cannot get posterior covariance matrix from kernlab’s gausspr. Can this be reconstructed from alpha? (question posted to cross-validated)
Likelihood optimization does not handle the noise term. Examples in Rasmussen and Wiliams suggest this should work. (See commit log below)

nonparametric-bayes log

re-run example with updated plots, but otherwise as before. 07:40 pm 2012/11/15
Question on how to get the covariance matrix back from the kernlab gausspr method. 07:39 pm 2012/11/15
Attempted to fix the sequential method example.

Still incorrect. See sequential-method.Rmd 07:37 pm 2012/11/15 - Attempting to write the sequential method algorithm.

Not correct at the moment, need to understand the recursion properly. 07:36 pm 2012/11/15 - Stick with direct method.

Current version of sequential method is not correct. 07:34 pm 2012/11/15 - Updated hyperparameter-optimization example

Shows that we can estimate l appropriately but fails to get sigma_n; instead estimating the upper boundary. 07:32 pm 2012/11/15

Better Tagging Practices

2012-11-09T00:00:00-08:00

Better tagging practices

Standardize use of tags: Match tags in notebook to project repository name. Tag papers in Mendeley by these tags as well.

I have also started tracking the medium of discovery for each paper I add to my Mendeley collection through a little set of Mendeley tags. I am curious to see through which mediums I obtain the most content, and what content. As with any ontology, I will struggle to select a vocabulary that is precise enought to be useful without being too verbose (b) to bother to continue to use this. We’ll see how I do.

tag	description
source:reader	Google Reader (RSS feed)
source:twitter	Mentioned on twitter stream (or G+ stream)
source:search	Searched for this article directly or by some keyword
source:bibliography	In the reference list of something I’m reading
source:conference	Discussed at a conference
source:blog	Mentioned in a blog I read
source:human	Directly recommended to me (in person or email, from reviewer, adviser, etc)
source:journal-club	Something we read in a journal club
source:wikipedia	Cited in a wikipedia article I read

Notes

2012-11-08T00:00:00-08:00

Complete IACUC training for fish room. File forms: http://ehs.ucsc.edu/programs/research-safety/animal-research/index.html
Online training – possibly see https://www.citiprogram.org/Default.asp?
Shiny: http://www.rstudio.com/shiny/
Comment piece
Prep exit seminar
Applied math club: Networks. Reading http://dx.doi.org/10.1155/2011/284909

Discussion of when network structure has made an important difference on our understanding of a problem. When have the common summary statistics of a network (degree-distribution) allowed us to compte network dynamics without fully resolving the network?

nonparametric-bayes

Comparing methods for Gaussian process fits

mlegp package appears more carefully constructed, but notation is terrible. Also unclear how to change covariance kernel. Example shows fitting with zero noise only, though somehow that should be adjusted (in the nugget parameters?). Has a vignette but it has not been particularly enlightening for me. Seems kernlab::gausspr may be the only intelligent package function for GP I’ve seen.
What explains the descrepancy in the Cholesky approach?
What explains the descrepancy in the sequential approach?

Sequential updating algorithm avoids inverting the matrix, which can become unstable as state space becomes large. Instead of dealing with the vector of observations at states simultaneously, we update the GP mean and covariance at each observed point sequentially. Because we can compute this over the desired prediction grid simultaneously still, is a vector of the same length as the prediction grid (), and the is a matrix of dimension by . Let be the covariance function of , and be the noise model of the GP, then

Work out the multi-dimensional Gaussian process algorithm

Multiple uncertainty

Verify orientation of each matrix?
higher resolution grid example (same problem).

Notes

2012-11-06T00:00:00-08:00

Monday

altmetrics post
post for multiple uncertainty, email to Steve, Marc
Marc meeting
ropensci proposals
Phylotastic

Notebook configuration

trying out http://greycite.knowledgeblog.org/. Add to knitcitations.
Add Greycite recommended opengraph and google scholar metadata metadata.html
Import metadata from _config.yml. Any custom variable defined in config can be accessed at site.varname. See updated _config.yml

Reading

Schnitzler, J., Graham, C. H., Dormann, C. F., Schiffers, K., & Peter Linder, H. (2012). Climatic niche evolution and species diversification in the Cape flora, South Africa. (S. Higgins, Ed.)Journal of Biogeography, n/a–n/a. doi:10.1111/jbi.12028
Clark, T., Dale, R., Waard, A. D., Herman, I., Hovy, E. H., Shotton, D., Birukou, A., et al. (2012). Force11 White Paper : Improving The Future of Research Communications and e-Scholarship, 11.
Shotton, D. (2009). Semantic publishing: the coming revolution in scientific journal publishing. Learned Publishing, 22(2), 85–94. doi:10.1087/2009202
Shotton, D. (2011). The Five Stars of Online Journal Articles–an article evaluation framework. Nature Preceedings. doi:10101/npre.2011.6542.1

Tuesday

FishBase paper is out
Dissertation on FigShare
Updated geospatial life history data list

Reading

Allesina, S. (2012). Modeling peer review: an agent-based approach. Ideas in Ecology and Evolution, 5(2), 27–35. doi:10.4033/iee.2012.5b.8.f
Costello, C., Ovando, D., Hilborn, R., Gaines, S. D., Deschenes, O., & Lester, S. E. (2012). Status and solutions for the world’s unassessed fisheries. Science (New York, N.Y.), 338(6106), 517–20. doi:10.1126/science.1223389
Palumbo, M. J., Johnson, S. A., Mundim, F. M., Lau, A., Wolf, A. C., Arunachalam, S., Gonzalez, O., et al. (2012). Harnessing Smartphones for Ecological Education, Research, and Outreach. Bulletin of the Ecological Society of America, 93(4), 390–393. doi:10.1890/0012-9623-93.4.390
Sibert, J., Senina, I., Lehodey, P., & Hampton, J. (2012). Shifting from marine reserves to maritime zoning for conservation of Pacific bigeye tuna (Thunnus obesus). Proceedings of the National Academy of Sciences of the United States of America, 109(44), 1–5. doi:10.1073/pnas.1209468109

Altmetrics Conference

2012-11-03T00:00:00-07:00

From Thursday Nov 1 through Saturday Nov 3 I attended the PLoS Altmetrics workshop as a member of the ROpensci project. An exciting meeting driven mostly by the opportunity to interact with a small and incredibly creative and innovative group of science publishers, funders, start-ups, and other researchers. Details of the event are largely recorded on the altmetrics 2012 workshop website. We also have a few slides addressing the ropensci slant.

The workshop followed up with a hackathon which produced prototypes of several ideas. I worked in the altviz team. Also completed the first release of rfigshare to CRAN.

Another side goal of the conference for me was to discuss some of the ideas that had arisen in the BeyondThePDF workshop over a year ago in regards to semantic markup, an idea I’ve recently explored in this notebook. To this end it was useful talking with Martin Fenner, Paul Groth, and John Kunze about semantic issues, implicit/inferred semantics vs explicit markup, etc. Also some progress in incorporating these ideas into knitcitations. For more in this space, see the Force11 project (successor/outcoem of the BeyondThePDF).

Multiple Uncertainty Corrections

2012-11-01T00:00:00-07:00

Consider a stock at time that recruits stochastically following harvest from the escaped population (), , where is a multiplicative stochastic shock to the growth. We imagine the decision maker sets a policy determining the harvest quota each year, which is implemented as the real harvest with some error , . Before setting a quota, the decision maker assesses the stock , so is a noisy measurement of the true stock . The decision maker seeks to maximize the expected present-value of profits that are a function of both the actual stock and the realized harvest over some long term period subject to some discount rate ,

Discrete formulation

We consider this problem in a discrete state space where and , and all take values on grids of , , , and points respectively.

Let be the by matrix of all possible profits in a time step, as given by , where rows represent possible stock sizes and columns possible harvests.
Let be the row-normalized by matrix whose elements are , the probability of observing a stock size from a true size .
Let be the row-normalized by matrix whose elements are , the probability of implementing a harvest given a quota Edit: I had been created in the code as by , so had to be transposed.
Let be the tensor of row-normalized by transition matrices given by . By summing over the uncertainty in measurement and implementation, is in the space of observed state into the true stock at the next timestep , given quota . We transfer into the future measured state through another application of .
The profits expected given a measurement of the stock for a quota are . Hence the matrix of expected profits for each possible combination of and is given by the matrix product
Let be the by matrix at time giving the value of choosing quota having observed stock and having selected the optimal , that is:

Let be the vector with the maximum value of the each row (i.e. for each state ) of the matrix , and
let be the vector whose elements give the column number (e.g. the determine the choice ) of the quota that corresponds to that value.

Thus . Starting from the final timepoint, we know the ending value as a function of the ending state . The probability of ending up in each possible state for , given a choice of quota and the current state is given by the transition matrix . The Bellman recursion becomes

R code algorithm:

    Ep <- M %*% P %*% I
    V <- Ep
    for(t in 1:Tmax){                                 
      D[,(Tmax-t+1)] <- apply(V, 1, which.max)        
      v_t <- apply(V, 1, max)                        
      V <- sapply(1:n_h, function(j){               
        Ep[,j] + (1-delta) * M %*% F[[j]] %*% v_t  
      })
    }
    D

Where

    P <-  outer(x_grid, h_grid, profit) 
    M <- rownorm( outer(x_grid, x_grid, pdfn, sigma_m) )
    I <- rownorm( outer(h_grid, h_grid, pdfn, sigma_i) )

F is a bit more complicated:

    F <- lapply(1:n_h, function(q){  
      t(sapply(1:n_x, function(y){
        out <- numeric(n_x)
        mu <- sum(sapply(1:n_x, function(x)
          f(x_grid[x],h_grid,p) %*% 
            I[q,] *  # Implementation error
            M[x,y])) # Measurement error
        if(mu==0)
          out[1] <- 1
        else 
          out <- dlnorm(x_grid/mu, 0, sigma_g)
        out/sum(out)
      }))
    })

Something seems to be a mistake, since the optimal policy under uncertainty does not make sense:

policy function under each source of uncertainty

example script

Updated Edit

See notes on code changes inline

Notes

2012-10-31T00:00:00-07:00

I enjoyed an excellent meeting with Annette Thomas, CEO of MacMillian Publishing and Nature Publishing group at UC Davis. It was impressive to here that Annette wanted to meet with UC Davis faculty and graduate students interested in open access and open science to discuss our opinions, desires and frustrations in the scientific publishing process. It was particularly valuable to discuss some of the failed experiments in open science at Nature. Following this event I sent the letter below to Annette recording my reflections on those experiments and what I think they might have revealed. The 2 hour discussion with four graduate students, two faculty members and members of UC Davis and Cal Digital Libraries hit on many other topics as well, and these reflections capture just a few of the issues discussed.

Yesterday we discussed the failure of the 2004 experiment in open peer review and the failure of the pre-print server, Nature Precedings. In my humble opinion, both of these were doomed by insufficient incentives to participate.

A more successful model to open up peer review has been that practiced by EMBO, as discussed in a Nature Comment in 2010 (doi: 10.1038/468029a). As we are already doing the work of peer review, their is an immediate incentive to get credit for that work through this mechanism, and more journals should consider it. (Biology Direct has a slightly more extreme take on this policy. Both have been far more successful than either the Nature experiment, comments sections of PLoS, or F1000 approaches to post-publication peer review).

Precedings was a bold and powerful idea that addressed many real gaps in the current publishing infrastructure – a preprint server for fields not represented in arXiv, support for posters and presentations, dois (okay, EZID handles; crossref DOIs might have been better), and a more modern interface than arXiv provided. I was sad to see this close. As you both surely know, preprints run strongly counter to the grain of mainstream biology and their concept of precedence (unlike physics, economics and other fields where the opposite is true). As recent genomic examples have illustrated, the value of faster communication and a broader pre-publication review may slowly shift that culture, but journals such as Nature could play a key role in easing the transition, as your 2004 experiment showed.

What is needed here is a button to submit work to the preprint server integrated into the normal submission form to the journal. This will dispel the persistent myth that the journal is not okay with it, and will meet with much larger uptake than simply providing the server separately would do. I would bet your own data from the 2004 experiment shows this pattern – more authors choose to participate in that, as a fraction of the total submissions, than have independently uploaded work to a preprint server. As you both observed, the failure of the 2004 experiment was in the breadth and depth of comments received, but as a study in willingness to use a preprint, it was an important positive discovery about promoting preprint culture that would only strengthen the final journal copies.

Notes

2012-10-29T00:00:00-07:00

Misc

Mostly writing application today…
Review for EcoLet
looking over pomdp/momdp
Notebook archives now on figshare (plain-txt versions). Can now also cite entries by doi, e.g.: Lab Notebook (2010) doi:10.6084/m9.figshare.96916, and Lab Notebook, (2011) doi:10.6084/m9.figshare.96919. Would publishing individual entries and individual figshare objects be preferable or just annoying?
Meeting tomorrow with Annette Thomas, CEO NPG/MacMillian. review interview with David Worlock

Munch data project

Data extraction for geospatial life history project (file

R script for converting place names into latitude/longitude:

library(RCurl)
library(RJSONIO)
construct.geocode.url <- function(address, return.call = "json", sensor = "false") {
    root <- "http://maps.google.com/maps/api/geocode/"
    u <- paste(root, return.call, "?address=", address, "&sensor=", sensor, sep = "")
    return(URLencode(u))
}
gGeoCode <- function(address,verbose=FALSE) {
  if(verbose) cat(address,"\n")
  u <- construct.geocode.url(address)
  doc <- getURL(u)
  x <- fromJSON(doc,simplify = FALSE)
  if(x$status=="OK") {
    lat <- x$results[[1]]$geometry$location$lat
    lng <- x$results[[1]]$geometry$location$lng
    return(c(lat, lng))
  } else {
  return(c(NA,NA))
  }
}

XML compliance in HTML5

See labnotebook/issues/21

HTML5 allows tags to be unclosed, e.g. <meta charset="utf-8"> instead of <meta charset="utf-8" /> but this is not valid XML.

See: * Polyglot/xhtml5 author guide * Benefits of XML * W3C XHTML media type * Optional tags in html5

Note that tags with no content must be closed with /> (like the meta example) while tags with content must get a normal closing (<li> item </li> etc). Despite this, the HTML5 spec is kinda logical since closing is syntax redundant even though not closing is not valid XML, see SO question for discussion.

example

Basic XML parsing is now possible, e.g. in R:

require(XML)
require(RCurl)
tt <- getURLContent("http://www.carlboettiger.info/2012/10/25/stochastic-dynamic-programming-with-gaussian-process-approx.html")
doc <- xmlParse(tt)
getNodeSet(doc, "//x:meta", namespace="x")

Validation requires meta tags only appear in <head> and not <body> (but still do use property and not name for RDFa properties). Validation also requries all images have alt-text, which means I should remember to use anchor text om pandoc images (e.g. also name my graph knitr chunks intelligently). Otherwise the site validates as HTML5.

Ignoring the declared content-type will nearly validate as XML. Had to make the attribute onClick from the Google tracker javascript lowercase onclick, like all other attributes. Now th only challenge is that HTML5 needs lang= declared in <html> tag, (while is tolerant of xml:lang), but XML doesn’t want to see lang in the <html> tag. Perhaps can go somewhere else and be valid HTML5? (Using a meta tag for this is now obsolete).

MathML

Mathml namespace is loaded by default in HTML5 (a native namespace). Currently using the javascript MathJax which has very nice display and interactive toggles (including getting the latex or mathml source). It looks like MathJax can take either latex input or mathml input, and Pandoc can render the latex source (much much nicer to type) into mathml. Just need to get pandoc to convert the source to mathml and then have the mathml rendered pretty by MathJax css, e.g. http://www.mathjax.org/demos/mathml-samples/. This may also load faster than javascript-based rendering of MathJax(?)

ALM goals

Headed to the PLOS ALM conference with ropensci next week. Should be a good chance to catch up on projects:

rfigshare to CRAN
semantic knitcitations
Mark Hahnel: figshare archiving of notebook – archive the markdown? archive code? archive git repos?
Martin Fenner: Scholarly HTML standards, notebook standards (can we get John Kunze involved?)

Stochastic Dynamic Programming With Gaussian Process Approx

2012-10-25T00:00:00-07:00

Here we work out an SDP solution of the Reed (1979) using a Gaussian process approximation. In this example the Gaussian process hyper-parameters haven’t been tuned, so in principle the solution would perform even better, but this is just for illustrative purposes.

We simulate 40 points under a stochastic growth (lognormal multiplicative noise) Beverton-Holt model and infer the Gaussian process. We start the simulation from a low stock value, giving reasonable but not perfect coverage of the state-space. The prior still has a strong influence outside these regions, so the GP doesn’t quite match the Beverton-Holt curve, as seen in the first figure.

We solve for the optimal policy function by discretizing the system and using the Gaussian process to fill out the stochastic transition matrix, which we pass to my little stochastic dynamic programming library to determine the policy function. We compare this policy function to the solution for the true F (Beverton-Holt) in the second figure, and note somewhat substantial differences (in particular a clear deviation from the constant escapement rule that emerges, which in itself is quite interesting, all though clearly non optimal in this case.

The GP doesn’t learn during the actual test period when the management is being implemented, though of course it could easily do so (after accounting for the application of the control). A single replicate simulation compares the optimal management under the true model to the GP management, which does less well (obviously), but not by a dramatic amount.

Code provides explicit statements of the models and should generate a reproducible example.

Next steps should optimize the GP and compare against the more realistic test case where the parameterized model is estimated from the simulation data (possibly under different functional forms, etc).

Coded example below, from reed-example.md

load my libraries and plotting lib.

require(nonparametricbayes)
require(pdgControl)
require(ggplot2)

(push images to flickr)

opts_knit$set(upload.fun = socialR::flickr.url)

Beverton-Holt function

Simulate some training data under a stochastic growth function with standard parameterization,

f <- BevHolt
p <- c(1.5, 0.05)
K <- (p[1] - 1)/p[2]

Parameter definitions

x_grid = seq(0, 1.5 * K, length = 101)
T <- 40
sigma_g <- 0.1
x <- numeric(T)
x[1] <- 1

Noise function, profit function

z_g <- function() rlnorm(1, 0, sigma_g)  #1+(2*runif(1, 0,  1)-1)*sigma_g #
profit <- profit_harvest(1, 0, 0)

Simulation

for (t in 1:(T - 1)) x[t + 1] = z_g() * f(x[t], h = 0, p = p)

Predict the function over the target grid

obs <- data.frame(x = x[1:(T - 1)], y = x[2:T])
X <- x_grid
library(nonparametricbayes)
gp <- gp_fit(obs, X, c(sigma_n = 0.5, l = 1.5))

Gaussian Process inference from this model. True model shown in red.

df <- data.frame(x = X, y = gp$Ef, ymin = (gp$Ef - 2 * sqrt(abs(diag(gp$Cf)))), 
    ymax = (gp$Ef + 2 * sqrt(abs(diag(gp$Cf)))))
true <- data.frame(x = X, y = sapply(X, f, 0, p))
require(ggplot2)
ggplot(df) + geom_ribbon(aes(x, y, ymin = ymin, ymax = ymax), fill = "gray80") + 
    geom_line(aes(x, y)) + geom_point(data = obs, aes(x, y)) + geom_line(data = true, 
    aes(x, y), col = "red", lty = 2)

Stochastic Dynamic programming solution based on the posterior Gaussian process:

rownorm <- function(M) t(apply(M, 1, function(x) x/sum(x)))
h_grid <- x_grid

Define a transition matrix from the Gaussian process, giving the probability of going from state to . We already have the Gaussian process mean and variance predicted for each point on our grid, so this is simply:

V <- sqrt(diag(gp$Cf))
matrices_gp <- lapply(h_grid, function(h) {
    F <- sapply(x_grid, function(x) dnorm(x, gp$Ef - h, V))
    F <- rownorm(F)
})

True

matrices_true <- lapply(h_grid, function(h) {
    mu <- sapply(x_grid, f, h, p)
    F_true <- sapply(x_grid, function(x) dnorm(x, mu, sigma_g))
    F_true <- rownorm(F_true)
})

Calculate the policy function using the true F:

opt_true <- find_dp_optim(matrices_true, x_grid, h_grid, 20, 0, profit, 
    delta = 0.01)

Calculate using the inferred GP

opt_gp <- find_dp_optim(matrices_gp, x_grid, h_grid, 20, 0, profit, 
    delta = 0.01)

require(reshape2)

policies <- melt(data.frame(stock = x_grid, GP = x_grid[opt_gp$D[, 
    1]], Exact = x_grid[opt_true$D[, 1]]), id = "stock")
q1 <- ggplot(policies, aes(stock, stock - value, color = variable)) + 
    geom_point() + xlab("stock size") + ylab("escapement")
q1

We can see what happens when we attempt to manage a stock using this:

z_g <- function() rlnorm(1, 0, sigma_g)

set.seed(1)
sim_gp <- ForwardSimulate(f, p, x_grid, h_grid, K, opt_gp$D, z_g, 
    profit = profit)
set.seed(1)
sim_true <- ForwardSimulate(f, p, x_grid, h_grid, K, opt_true$D, 
    z_g, profit = profit)

df <- data.frame(time = sim_gp$time, stock_gp = sim_gp$fishstock, 
    stock_true = sim_true$fishstock, harvest_gp = sim_gp$harvest, havest_true = sim_true$harvest)
df <- melt(df, id = "time")
ggplot(df) + geom_line(aes(time, value, color = variable))

Total Profits

sum(sim_gp$profit)

## [1] 25.43

sum(sim_true$profit)

## [1] 29.95

Multiple Uncertainty Value Function Iteration

2012-10-24T00:00:00-07:00

Multiple Uncertainty algorithm

Define each of the transition matrices:

    D <- matrix(NA, nrow=length(x_grid), ncol=Tmax)  
    P <- outer(x_grid, h_grid, profit)
    F <- outer(x_grid, f(x_grid, 0, p), pdfn, sigma_g)
    M <- outer(x_grid, x_grid, pdfn, sigma_m)
    I <- outer(h_grid, h_grid, pdfn, sigma_i)

Probably row-normalize each:

rownorm <- function(M) t(apply(M, 1, function(x) x/sum(x))

(Note the transpose is needed since the silly function of a vector turns the column into a row. Not an issue of margin 2)

If we have no uncertainty in measurement or implementation, then the algorithm is:

    V <- P
    for(t in 1:Tmax){
      D[,(Tmax-t+1)] <- apply(V, 1, which.max)
      V <- P + delta * F %*% V %*% D[, (Tmax-t+1)]  
    }
    D

With uncertainty

Ep <- M %*% P %*% I
U <- t(M) %*% F %*% M 
V <- Ep
  for(t in 1:Tmax){
    D[,(Tmax-t+1)] <- apply(V, 1, which.max)
    V <- Ep + delta * U %*% V %*% I %*% D[, (Tmax-t+1)]  
  }

Description

** rambling thinking while working out the above **

The October 1st entry derives the rule for updating the state (measured stock),

but this isn’t sufficient to complete the value iteration. Recall from the Bellman recursion,

In the final timestep with no scrap value, we assert that the vector of values for each possible state, . The previous year is then the first year that can have profits, given by . Of course is unknown, but is given by the recursion in (1).

Our strategy in discrete space is to work backwards from , keeping track of the that maximizes the value-to-go for each possible state. So at , we put each possible state and each possible havest into and select the that maximizes the profit. In the case of a deterministic harvest and a lack of measurement error this first iteration is trivial: harvest what you see (unless is bounded or there is a cost on effort), yielding the value for each state is . When profits are proportional to harvest, up to a constant this is simply .

For a generic profit function that may have a cost on effort, we must store the map of which is optimal for each . In discrete space, call this the decision vector where the indices correspond to the states and the values correspond to the harvest (or harvest index) that is optimal for that state, and we write

If there is uncertainty in either implementation of the harvest or measurement of the stock, or both, then we must integrate over this in determining the optimal . The expected profit derived from choosing a harvest quota and having measured a stock at is given by

In matrix form, if is the matrix of profits where element and vector with elements give the probability of harvesting fish given the quota , then is a vector of profits for each . Representing distribution in the measurement similarly as , the integration for the expected profit is (giving the scalar value ).

We can store & lookup using a matrix whose columns give for each value , and similarly for given . These are our matrices (each row is a fixed observed , maps true into observed , elements ) (mapping quota (columns) to implemented harvest , elements ) from before. Then we can write as a matrix resulting from the product of matrices:

In the next iteration, , we realize that our choice of impacts the value we could get a time as well. In choosing the harvest quota in this interval we must also consider what value it introduces in the interval. We already have the matrix above giving the profits for each and , we just need to transition from the space of the current to the future through application of the transition matrix . , , , , so

Evolutionary Covariance

2012-10-24T00:00:00-07:00

Thinking through disparity measures while writing my thoughts in R…

Load some sample data

library(geiger)
data(geospiza)
dat <- treedata(geospiza$geospiza.tree, geospiza$geospiza.data$wingL)

To find the disparity relative to a particular model, we must compute the variance-covariance matrix for that model

X <- dat$data
V_bm <- vcv(dat$phy)
mean_X <- geiger:::phylogMean(V_bm, X)
bm_disparity <- t(X-mean_X) %*% V_bm %*% (X-mean_X)

I believe this needs to be scaled by the Brownian rate parameter of the trait, e.g.

bm <- fitContinuous(dat$phy, dat$data)
bm_disparity <- bm[[1]]$beta * bm_disparity
OU would be similar, but transform the tree

ou <- fitContinuous(dat$phy, dat$data, model="OU")
alpha <-  ou[[1]]$alpha
V_ou <- vcv(ouTree(dat$phy, alpha))
ou_var <- ou[[1]]$beta/(2*alpha)
X_mean <- phylogMean(V_ou, X)
ou_disparity <- t(X-X_mean) %*% V_ou %*% (X-X_mean) * ou_var

There’s a few things I’m unsure about off the top of my head in this. I should double-check that beta in geiger is the same as , and I should double check the ou variance-covariance formula (e.g. compare to the un-numbered equation after equation 5 in Butler & King). I also not sure I quite have the right X_mean in the OU case.

Semantic Markup Examples For The Lab Notebook

2012-10-23T00:00:00-07:00

All notebook entries are formatted with XHTML compliant (polyglot) HTML5 semantic structure. This means that any entry can be parsed with a generic XML parser to extract the entry content in <article>, the <header>, <footer>, <aside>, etc. The <head> section provides <title> and essential <meta> tags declaring the character encoding (which also sets the MIME type for HTML5).

Head

In _includes/header.html we introduce some basic academic archive metadata using the Dublin Core ontology. The html5 <header> seems like a good place for dc:title. Other metadata is added invisibly using content in a meta tag.

<head prefix="dc: http://purl.org/dc/terms/">
<!-- HTML5 metadata -->
<meta charset="utf-8" /> <!--same as <meta http-equiv='Content-Type' content='Type=text/html; charset=utf-8'> -->
<meta name="author" content="Carl Boettiger" />
<meta name="keywords" content="Ecology, Evolution, Open Science, Reproducible Research" />
<meta name="description" content="My open lab notebook: research in theoretical ecology and evolution" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>{{ page.title }}</title>
<!-- RDFa Metdata (in Dublin Core) -->
<meta property="dc:creator" content="Carl Boettiger" />
<meta property="dc:title" content="{{ page.title }}" />
<meta property="dc:type" content="Lab Notebook" />
<meta property="dc:format" content="text/html" />
<meta property="dc:language" content="en" />

More metadata is displayed in the sidebar (in an html5 aside element), including date, tags and category. I’ve added the following RDFa to describe these (see examples with the Jekyll liquid code in _includes/sidebar.html)

<aside prefix="dc: http://purl.org/dc/terms/">
Posted on
<time datetime="{{ page.date | date_to_xmlschema }}" 
      property="dc:created">{{ page.date | date_to_string }}</time>.

This puts the date in an HTML5 time element described as the dc.created time of the publication. Tags and categories use a little Liquid code so Jekyll can import them from the YAML header. The link to the tag page is described with the HTML rel attribute for tag, while the tag itself is given the RDFa property of a blog keyword from schema.org. Categories are given the more formal dc:subject property. (See _includes/pagetags.html for Jekyll/Liquid code for looping over multiple tags, etc).

  <a rel="tag" class="tag" href="/tags.html#{{ tag | cgi_escape }}">#<span 
  property="http://schema.org/BlogPosting/keywords">{{ tag }}</span></a>
  <a rel="tag" class="category" href="/categories.html#{{ category | cgi_escape }}"><span property="dc:subject">{{ category }}</span></a>

The next and previous buttons get the HTML5 rel values next and prev (see HTML5 semantics). Note that all the Dublin Core elements are properties of the page on which they occur.

The <footer> contains licenses and contact information through social media. I’ve used the FOAF vocabulary to indicate my membership to the various social networks provided in the links. The social networks are properties of a FOAF:person, (me), which is indicated in the RDFa resource:

<div class="span4" vocab="http://xmlns.com/foaf/0.1/" typeof="Person"
resource="http://www.carlboettiger.info#me">
  <a property="account" href="mailto:cboettig@gmail.com"><img
  src="/assets/img/icon-email.png" alt="email"> </a>
  <a property="account" href="https://twitter.com/cboettig"><img
  src="/assets/img/icon-twitter.png" alt="twitter"> </a>
  <a property="account" href="https://github.com/cboettig"><img
  src="http://www.carlboettiger.info/assets/img/icon-github.png"
  alt="github"> </a> <a property="account"
  href="http://www.mendeley.com/profiles/carl-boettiger"><img
  src="/assets/img/icon-mendeley.png" alt="mendeley"></a>
<a href="/atom.xml"><img src="/assets/img/icon-rss.png"
alt="feed"></a> </div>

It might be worth adding more FOAF terms describing the relationship of these networks (such as distinguishing my profile name and the base url) using meta elements to avoid adding clutter text. I use the Creative Commons namespace to identify my CC0 license. The license is a property of the page on which it is found, like the Dublin Core metadata.

<a property="http://creativecommons.org/ns#license"
href="http://creativecommons.org/publicdomain/zero/1.0/"><img
src="http://i.creativecommons.org/l/zero/1.0/88x31.png" alt="CC0"
style="float:right"></a>

When these elements are combined in a given entry, the RDFa generates this structure

Easily entering RDFa

All of this markup content is either generally static or filled in programatically by Jekyll from the YAML headers (e.g. title, categories, tags, date), leaving me free to put the RDFa into the template files just once and forget about it. In this way I can continue to write everything in Markdown, rather than the more cumbersome HTML riddled by the addition of RDFa attributes. Unfortunately there is no easy way to escape this in writing the Homepage markup, described below, but a single case isn’t so bad.

Another obvious candidate for linked data in the posts is the citation information. This markup can probably be automatically generated for each post using scripts (see Automated feeds in notebook entries), so I plan to add this to my knitcitations package when I get a chance (including CiTO support!).

Homepage

My homepage contains a brief description about me that is ripe for FOAF and schema.org. While this content is reasonably static, it does mean writing directly in HTML+RDFa, with the markup making the source almost unreadable. (It might be possible to force more of this data into YAML header and have it added through the invisible meta tags, but that seems rather convoluted). Capturing the “buisness card” information as well as the “social” information requires mixing ontologies, which can lead to some interesting complications. To make a long story short:

Implicit logic is possible with linked data: we can infer that schema:jobTitle can be a property of a foaf:Person because it is a property of a schema:Person and someone declares them as owl:equivalentProperty.
In practice it may be most robust to explicitly declare the relation by exploiting the fact that we can give multiple arguments to attributes like property and typeof.

<div id="me" prefix="foaf: http://xmlns.com/foaf/0.1/" prefix="schema:
http://schema.org/Person" typeof="foaf:Person schema:Person"
resource="http://www.carlboettiger.info#me"> 
I am  <a property="foaf:homepage schema:url"
href="http://www.carlboettiger.info"><span property="foaf:name
schema:name"><span property="foaf:givenName schema:givenName">Carl</span>
<span property="foaf:familyName schema:familyName">Boettiger</span></span></a>, <span
property="schema:jobTitle">a graduate student</span>

(This excerpt gives the general flavor, see the html source of index.html for a full example.)

The homepage generates this RDFa structured graph (from rdfa.info/play)

Notes

Some background reading on semantics

Nice collection of posts on scholarly html, but too little mention of the serious linked data stuff coming out of biscol, ievobio/nescent, etc. In particular, Martin’s history gives some nice perspective.
The war for schema. Which ontologies should we choose? (e.g. schema.org’s take
On the good & the bad of schema.org and the role of search giants

Mixing ontologies had me very confused.

How do we mix ontologies
How do we meet Google Rich Snippets requirements using FOAF?
How do we avoid redundant vocabulary terms?
We can avoid the redundancy using implicit reasoning (exploiting the logic of linked data), but we may prefer to be explict. This video does a nice job of explaining these concepts. If we do semantics right, we should be able to “dereference” an object away from a particular ontology.

Tools and Resources

Google rich snippets
rdfa extractor (into RDF, JSON, n3). Also has validator, but doesn’t point to the nu variant for html5.
RDFa Play. Wow, very nice.
sameAs.org – search onotologies for an existing term, identify identical terms in different ontologies, etc. Simply brilliant, what I’ve always wanted!

knitcitations update: adding formats an semantics.

Looking at citeproc / CSL github page.
Raw citeproc is an XML stylesheet, XSLT, which can render XML into HTML (and much more). Possible other options are using something like citeproc-js or citeproc-ruby, or writing citeproc-R.
Not quite clear how to take a CSL and a set of references and generate HTML; aforementioned packages may address this.
For the moment, probably easier to convert R bibitem to display with RDFa with scholarlyhtml recommended ontology than to enter raw XML reference and format via citeproc. Should also support adding in the cito tags. With full addresses in property, should be able to avoid any need for namespace/header modifications.

Gaussian process inference on sample functions

2012-10-23T00:00:00-07:00

This example uses the direct inversion of the covariance function. I have also been exploring the potentially faster Cholksy decomposition without success. Sometimes the match is very good, other times not, see compare-cholsky.md
This example also on github as reed-example.md.
Basic likelihood optimization needs to be checked. See commits estimating sigma_n and estimating l. Compare the example below to a different set of fixed hyperparameters here

Example

require(pdgControl)
require(ggplot2)

Simulate some training data under a stochastic growth function with standard parameterization,

f <- BevHolt
p <- c(1.5, 0.05)
K <- (p[1] - 1)/p[2]

Noise function

z_g <- function(sigma_g) rlnorm(1, 0, sigma_g)  #1+(2*runif(1, 0,  1)-1)*sigma_g #

Parameter definitions

x_grid = seq(0, 1.5 * K, length = 100)
T <- 40
sigma_g <- 0.1
x <- numeric(T)
x[1] <- 1

Simulation

for (t in 1:(T - 1)) x[t + 1] = z_g(sigma_g) * f(x[t], h = 0, p = p)

plot(x)

Predict the function over the target grid

obs <- data.frame(x = x[1:(T - 1)], y = x[2:T])
X <- x_grid
library(nonparametricbayes)
gp <- gp_fit(obs, X, c(sigma_n = 0.05, l = 10))

Gaussian Process inference from this model. True model shown in red.

df <- data.frame(x = X, y = gp$Ef, ymin = (gp$Ef - 2 * sqrt(abs(diag(gp$Cf)))), 
    ymax = (gp$Ef + 2 * sqrt(abs(diag(gp$Cf)))))
true <- data.frame(x = X, y = sapply(X, f, 0, p))
require(ggplot2)
ggplot(df) + geom_ribbon(aes(x, y, ymin = ymin, ymax = ymax), fill = "gray80") + 
    geom_line(aes(x, y)) + geom_point(data = obs, aes(x, y)) + geom_line(data = true, 
    aes(x, y), col = "red", lty = 2)

Another example using the May model

f <- May
p <- c(r = 0.75, k = 10, a = 1.2, H = 1, Q = 3)
K <- 8  # approx

Model dynamics look like this:

birth <- function(x) p["r"] * (1 - x/p["k"])
death <- function(x) p["a"] * x^(p["Q"] - 1)/(x^p["Q"] + p["H"])
df <- data.frame(x = x_grid, b = sapply(x_grid, birth), d = sapply(x_grid, 
    death))
ggplot(df) + geom_line(aes(x, b), col = "blue") + geom_line(aes(x, 
    d), col = "red")

Simulation

x[1] = 2.5
for (t in 1:(T - 1)) x[t + 1] = z_g(sigma_g) * f(x[t], h = 0, p = p)
plot(x)

Predict the function over the target grid

obs <- data.frame(x = x[1:(T - 1)], y = x[2:T])
X <- x_grid
library(nonparametricbayes)
gp <- gp_fit(obs, X, c(sigma_n = 0.05, l = 10))

Gaussian Process inference from this model

df <- data.frame(x = X, y = gp$Ef, ymin = (gp$Ef - 2 * sqrt(abs(diag(gp$Cf)))), 
    ymax = (gp$Ef + 2 * sqrt(abs(diag(gp$Cf)))))
true <- data.frame(x = X, y = sapply(X, f, 0, p))
ggplot(df) + geom_ribbon(aes(x, y, ymin = ymin, ymax = ymax), fill = "gray80") + 
    geom_line(aes(x, y)) + geom_point(data = obs, aes(x, y)) + geom_line(data = true, 
    aes(x, y), col = "red", lty = 2)

Comparing numerical methods

2012-10-22T00:00:00-07:00

require(MASS)
require(ggplot2)
require(kernlab)

Parameterization-specific

X <- seq(-5,5,len=50)
obs <- data.frame(x = c(-4, -3, -1,  0,  2),
                  y = c(-2,  0,  1,  2, -1))
l <- 1
sigma_n <- 0.8

Radial basis function/Gaussian kernel:

  SE <- function(Xi,Xj, l) exp(-0.5 * (Xi - Xj) ^ 2 / l ^ 2)
  cov <- function(X, Y) outer(X, Y, SE, l)

Cholksy method

  n <- length(obs$x)
  K <- cov(obs$x, obs$x)
  I <-  diag(1, n)

  L <- chol(K + sigma_n^2 * I)
  alpha <- solve(t(L), solve(L, obs$y))
  k_star <- cov(obs$x, X)
  Y <- t(k_star) %*% alpha
  v <- solve(L, k_star)
  Var <- cov(X,X) - t(v) %*% v
  loglik <- -.5 * t(obs$y) %*% alpha - sum(log(diag(L))) - n * log(2 * pi) / 2

Direct method

  cov_xx_inv <- solve(K + sigma_n^2*I)
  Ef <- cov(X, obs$x) %*% cov_xx_inv %*% obs$y
  Cf <- cov(X, X) - cov(X, obs$x)  %*% cov_xx_inv %*% cov(obs$x, X)

gp <- gausspr(obs$x, obs$y, kernel="rbfdot", kpar=list(sigma=1/(2*l^2)), fit=FALSE, scaled=FALSE, var=0.8)
y_p <- predict(gp, X)

Things that should be equivelent but aren’t quite:

ggplot(data.frame(x=X, Ef=Ef, Y=Y, y_p = y_p))+ geom_point(aes(x,Ef), col='red') + geom_point(aes(x,Y)) + geom_line(aes(x,y_p))

plot of chunk unnamed-chunk-8

cov_xx_inv %*% obs$y

        [,1]
[1,] -1.4059
[2,]  0.5010
[3,]  0.1288
[4,]  1.2275
[5,] -0.7119

alpha

[1] -1.21293  0.46074  0.07548  1.44144 -0.73949

alpha(gp)

        [,1]
[1,] -1.2475
[2,]  0.4011
[3,]  0.1661
[4,]  1.1010
[5,] -0.6394

Notes

2012-10-19T00:00:00-07:00

Semantic Notebook notes

Reading

Nice collection of posts on scholarly html, but too little mention of the serious linked data stuff coming out of biscol, ievobio/nescent, etc. In particular, Martin’s [history] (http://blogs.plos.org/mfenner/2011/03/19/a-very-brief-history-of-scholarly-html/) gives some nice perspective.
The war for schema. Which ontologies should we choose? (e.g. schema.org’s take

Puzzles

How do we mix ontologies
How do we meet Google Rich Snippets requirements using FOAF?
How do we avoid redundant vocabulary terms?

Basically we can do this using implicit reasoning (exploiting the logic of linked data), but we may prefer to be explict. This video does a nice job of explaining these concepts. Basically, if we do this right, we should be able to “dereference” an object away from a particular ontology.

On the good & the bad of schema.org and the role of search giants

Resources

rdfa extractor (into RDF, JSON, n3). Also has validator, but doesn’t point to the nu variant for html5.
RDFa Play. Wow, very nice.
sameAs.org – search onotologies for an existing term, identify identical terms in different ontologies, etc. Simply brilliant, what I’ve always wanted!
Still not clear on rdfa vs html5. Why in the world should dc:title be preferable to <title> ?

knitcitations update: adding formats an semantics.

Looking at citeproc / CSL github page.
Raw citeproc is an XML stylesheet, XSLT, which can render XML into HTML (and much more). Possible other options are using something like citeproc-js or citeproc-ruby, or writing citeproc-R.
Not quite clear how to take a CSL and a set of references and generate HTML; aforementioned packages may address this.
For the moment, probably easier to convert R bibitem to display with RDFa with scholarlyhtml recommended ontology than to enter raw XML reference and format via citeproc. Should also support adding in the cito tags. With full addresses in property, should be able to avoid any need for namespace/header modifications.

Notes

2012-10-17T00:00:00-07:00

Talking points

for phone interview with Lucy

Finding warning signals in the wild

Gold standard – experimental group, control group
Real world: We have no control group.

Would we know a warning signal when we saw it?

From statistics to models: We fit a generalized models of the processes that are and aren’t approaching a tipping point, and compare the models. The modeling approach is also more sensitive than the simple statistics.

In another recent paper we consider the alternative ground-truth approach – historical data. There’s a problem in using historical data though, which is a bit subtle. It has tricked juries, confounded medical doctors, and confused scientists. We call it the Prosecutor’s fallacy (Thompson & Schumann 87). We need the probability the “accused pattern” is “innocent” of heading for a critical transition, given the data. We calculate instead the probability of data, given that pattern is innocent, is small. By itself, that is not enough to “convict” our data of being a warning signal.

Critical transitions in networks vs early warning signals – more must be done to map the connections.
Be careful not to mistate generality – we know systems can collapse without transition, and warning signals can be seen without transitions

Basic regression in Gaussian processes

2012-10-17T00:00:00-07:00

Working out coded examples for basic Gaussian process regression using R. I’ve just read through the first few chapters of Rasmussen & Williams (2006), this implements the examples discussed in Chapter 2.1-2.5.

Required R libraries (for multivariate normal, also for plotting):

require(MASS)
require(reshape2)
require(ggplot2)

Set a seed for repeatable plots

set.seed(12345)

Define the points at which we want to compute the function values (x values of the prediction points or test points), and the scale parameter for the covariance function

x_predict <- seq(-5,5,len=50)
l <- 1

We will use the squared exponential (also called radial basis or Gaussian, though it is not this that gives Gaussian process it’s name; any covariance function would do) as the covariance function,

SE <- function(Xi,Xj, l) exp(-0.5 * (Xi - Xj) ^ 2 / l ^ 2)
cov <- function(X, Y) outer(X, Y, SE, l)

COV <- cov(x_predict, x_predict)

Generate a number of functions from the process

values <- mvrnorm(3, rep(0, length=length(x_predict)), COV)

Reshape the data into long (tidy) form, listing x value, y value, and sample number

dat <- data.frame(x=x_predict, t(values))
dat <- melt(dat, id="x")
head(dat)

       x variable   value
1 -5.000       X1 -0.6450
2 -4.796       X1 -0.9227
3 -4.592       X1 -1.1587
4 -4.388       X1 -1.3277
5 -4.184       X1 -1.4139
6 -3.980       X1 -1.4103

Plot the result

fig2a <- ggplot(dat,aes(x=x,y=value)) +
  geom_rect(xmin=-Inf, xmax=Inf, ymin=-2, ymax=2, fill="grey80") +
  geom_line(aes(group=variable)) +   theme_bw() +
  scale_y_continuous(lim=c(-3,3), name="output, f(x)") +
  xlab("input, x")
fig2a

plot of draws from the null distribution

Posterior distribution given the data

Imagine we have some data,

obs <- data.frame(x = c(-4, -3, -1,  0,  2),
                  y = c(-2,  0,  1,  2, -1))

In general we aren’t interested in drawing from the prior, but want to include information from the data as well. We want the joint distribution of the observed values and the prior is:

No observation noise

Assuming the data are known without error and conditioning on the data, and given , then the conditional probability of observing our data is easily solved by exploiting the nice properties of Gaussians,

(We use solve(M) which with no second argument will simply invert the matrix M, but should use the Cholsky decomposition instead for better numerical stability)

cov_xx_inv <- solve(cov(obs$x, obs$x))
Ef <- cov(x_predict, obs$x) %*% cov_xx_inv %*% obs$y
Cf <- cov(x_predict, x_predict) - cov(x_predict, obs$x)  %*% cov_xx_inv %*% cov(obs$x, x_predict)

Now lets take 3 random samples from the posterior distribution,

values <- mvrnorm(3, Ef, Cf)

and plot our solution (mean, 2 standard deviations, and the ranom samples.)

dat <- data.frame(x=x_predict, t(values))
dat <- melt(dat, id="x")

fig2b <- ggplot(dat,aes(x=x,y=value)) +
  geom_ribbon(data=NULL, 
              aes(x=x_predict, y=Ef, ymin=(Ef-2*sqrt(diag(Cf))), ymax=(Ef+2*sqrt(diag(Cf)))),
              fill="grey80") +
  geom_line(aes(color=variable)) + #REPLICATES
  geom_line(data=NULL,aes(x=x_predict,y=Ef), size=1) + #MEAN
  geom_point(data=obs,aes(x=x,y=y)) +  #OBSERVED DATA
  scale_y_continuous(lim=c(-3,3), name="output, f(x)") +
  xlab("input, x")
fig2b

plot of draws from the posterior, with no proces noise

Additive noise

In general the model may have process error, and rather than observe the deterministic map we only observe . Let us assume for the moment that are independent, normally distributed random variables with variance .

sigma.n <- 0.25
cov_xx_inv <- solve(cov(obs$x, obs$x) + sigma.n^2 * diag(1, length(obs$x)))
Ef <- cov(x_predict, obs$x) %*% cov_xx_inv %*% obs$y
Cf <- cov(x_predict, x_predict) - cov(x_predict, obs$x)  %*% cov_xx_inv %*% cov(obs$x, x_predict)

Now lets take 3 random samples from the posterior distribution,

values <- mvrnorm(3, Ef, Cf)

and plot

dat <- data.frame(x=x_predict, t(values))
dat <- melt(dat, id="x")

fig2c <- ggplot(dat,aes(x=x,y=value)) +
  geom_ribbon(data=NULL, 
              aes(x=x_predict, y=Ef, ymin=(Ef-2*sqrt(diag(Cf))), ymax=(Ef+2*sqrt(diag(Cf)))),
              fill="grey80") + # Var
  geom_line(aes(color=variable)) + #REPLICATES
  geom_line(data=NULL,aes(x=x_predict,y=Ef), size=1) + #MEAN
  geom_point(data=obs,aes(x=x,y=y)) +  #OBSERVED DATA
  scale_y_continuous(lim=c(-3,3), name="output, f(x)") +
  xlab("input, x")
fig2c

plot of chunk unnamed-chunk-16

Note that unlike the previous case, the posterior no longer collapses completely around the neighborhood of the test points.

We can also compute the likelihood (and marginal likelihood over the prior) of the data directly from the inferred multivariate normal distribution, which can allow us to tune the hyperparameters such as the characteristic length scale and the observation noise . The most obvious approach would be to do so by maximum likelihood, giving point estimates of the hyper-parameters, though presumably we could be Bayesian about these as well.

Gaussian processes

2012-10-16T00:00:00-07:00

Great methods tutorial/overview with Steve. Now getting up to speed on background reading in both Gaussian processes and embedding dimensions.
Will also want to look into basis approximations for SDP acceleration.

Gaussian Processes

New project, New tag: nonparametric-bayes, and associated github repostitory

Reading

Rasmussen & Williams (2006) 2nd ed, full text avialable online. read Ch 1 & 2.

Can be thought of as a Bayesian version of Support Vector Machines

Who knew? Working through basic regression scenarios.

Also reading:

Deisenroth, M. P., Rasmussen, C. E., & Peters, J. (2009). Gaussian process dynamic programming. Neurocomputing, 72(7-9), 1508–1524. doi:10.1016/j.neucom.2008.12.019
Kocijan, J., & Girard, A. (2005). Dynamic systems identification with Gaussian processes. … of Dynamical Systems. doi:10.1080/13873950500068567
Turner, R. (2010). State-space inference and learning with Gaussian processes. Proceedings of 13th …, 868–875.

R support:

gausspr, see also the package vignette (From the core team, probably a reasonable starting point).
gptk (crude choice, poor software development practices)
mlegp
manually (e.g. for learning rather than performance)

Embedding dimensions

[Taken’s Theorem](http://en.wikipedia.org/wiki/Takens’_theorem) But how about the stochastic case?
Convergent cross-mapping paper, Sugihara et al. 2012, recently in Science.
Stochastic Takens: perhaps see Stark et al 2003, doi:10.1007/s00332-003-0534-4

rOpenSci

Checked ritis, Scott submitted to CRAN
More requests for increased functionality on rfishbase
Contact with Roy about developing an API for FishBase. Contacted Deng
httr 0.2 is on CRAN! Updated rfigshare dependency, plan to release soon.

Notebook

Contacted Ben about documentation and use of EML.
Posted updated semantic notebook
Hmm, FOAF person types don’t seem to contain enough data to be valid Person objects according to Google Rich Snippets. Unfortunately this cannot be easily solved by mixing “Person” types. Note that Dublin Core and FOAF agree on what a person/agent is, but it seems data-vocabulary.org rdf does not?
Tools for discovery and navigating ontologies? http://www.ebi.ac.uk/ontology-lookup/

Semantic Lab Notebook

2012-10-14T00:00:00-07:00

As the lab notebook grows, to make the maximum use of content it would be particularly useful to maximize the ability for a computer to understand the content, allowing us to identify, manipulate, and connect data using scripts and software. This is the concept of linked data, or a semantic notebook. I have explored this this idea before in the context of a wordpress-based platform, but now that Jekyll has let me strip away some of the abstraction of Wordpress it seems a good time to revisit this idea.

Semantics in HTML5

Already the notebook is written in HTML5, which has considerable semantic structure compared to it’s predecessors. HTML5 introduces the structural elements

<nav>
<header>
<article>
<section>
<aside>
<footer>

that intuitively define the organization of an page, setting off the important content from the window-dressing. HTML5 also defines inline elements <time> (see the links in Caveats), and <mark>, and the existing tags for metadata, which let us specify <title>, and basic <meta> tags for metadata such as author, <meta name="author" content="Author Name">, encoding, description, and keywords. Links <link> and <a> use the rel attribute to describe the link target. Though one can write anything at all in this text, HTML5 defines a small vocabulary with recognized meaning, some old, some new, such as <a rel="tag", rel="license" as well as the older <link rel="stylesheet" href="style.css"> and <a rel="nofollow". Here is a great overview of the semantics in HTML5, with a bit more about the available . To expand our vocabulary beyond these elements, however, we will need more tools.

Linked Data

W3C confusingly provides two standards for formally defining semantic content using an external vocabulary or ontology. The first is microdata, introduced as a simpler alternative to the second, RDFa, an HTML-adaptation of the RDF XML standard (originally developed for the now-defunct XHTML 2.0).

Microdata

Microdata introduces new attributes into HTML tags like <div>, <span>, <h1>. The first of these is itemtype, which points to an external resource such as schema.org to define the vocabulary. To have this vocabulary apply to child elements, we just add the attribute itemscope. Then we can set the value of attribute itemprop in this or following elements to give it semantically defined meaning, such as

<div itemscope itemtype="http://data-vocabulary.org/Person">
  <h1 itemprop="name">Carl Boettiger</h1>
  <a itemprop="url" href="http://www.carlboettiger.info">http://carlboettiger.info</a>
  <img itemprop="photo" src="https://en.gravatar.com/userimage/12904315/7edea703b826fbbe07f2ae4d95b8416b.jpg?16"/>
</div>

where the terms such as name and url have precise meanings attached to “Person”, as specified at the “http://data-vocabulary.org/Address”. This ability to point to an external vocabulary is really the key concept of linked data. In the spirit of HTML5, microdata is much simpler than RDFa, but also more limited. Here is an excellent comparison, but for our purposes we will use RDFa as it is more common in academic use and will more seamlessly allow us to use academic ontologies. (While microdata has a clear mapping to RDF, it is not clear that any ontology that can be expressed in RDF can also be expressed in microdata).

RDFa

RDFa can be written in a very similar manner. The itemprop attribute is replaced by the property attribute, and vocab replaces itemtype and automatically implies itemscope to child nodes.

<div vocab="http://xmlns.com/foaf/0.1/" typeof="Person">
  <h1 property="name">Carl Boettiger</h1>
  <a property="homepage" href="http://www.carlboettiger.info">http://carlboettiger.info</a>
  <img property="depiction" src="https://en.gravatar.com/userimage/12904315/7edea703b826fbbe07f2ae4d95b8416b.jpg?16"/>
 </div>

We could have used only the property attribute along with the complete URI to the term definition instead of this structure. typeof indicates that the child elements are all of the same type (in this case, all belong to the same “Person”). If we were omitting vocab and writing out all URIs, we would have had <div typeof="http://xmlns.com/foaf/0.1/Person"> as well. In RDFa, we also have the resource attribute, which allows us to specify a URI for the element being described. This could be a relative or absolute URL. If the object is a link, using resource is not necessary, and the property is taken to describe the URL given in href or src rather than the anchor text.

Two older options lie at either extreme: the most basic is the older technique of microformatting, basically relying on standard class and rel attributes to convey semantic information. Simple and without new attributes, but probably too limited for our purposes so we won’t concern ourselves with it further. The other venerable approach is serve the page as XHTML, which renders in the browser in much the same way but can be parsed by a machine as XML, with all the power and extensibility that presents. Unlike earlier HTML standards, HTML5 is already valid XML (already “serialized”), so the same page can be served in either format (earlier specs were SGML, standardized general markup languages, of which XML is just a subset). To allow parsing of the HTML5 pages as either type (see polyglot), we need only add the language and namespace to the opening <html> tag,

<html lang="en" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">

Caveats

An essential caveat is that these are all new approaches which may not render well in legacy browsers, particularly Windows Internet Explorer. Some of these, like recognizing and styling the HTML5 semantic elements in IE, can be addressed in CSS, for which Twitter Bootstrap does a decent job. Another caveat The HTML5 spec has not been finalized, and some things are still in flux, as the removal and reinstatement of the <time> element illustrates.

Useful tools for checking implementation

Scholarly Semantic Markup for the Notebook

Now that we’ve familiarized ourselves with the options, it’s time to see what semantic content we can implement.

Content/Data	Example types	Links to Potential vocabularies
Page structure	`<header>`	HTML5
Post metadata	keywords, timestamps	Schema.org microdata or Dublin Core
Author metadata	Name, contact, networks,	FOAF, Dublin Core
	interests, publications)
Licenses	CC0	CreativeCommons
Citations	bib info, reason for citing	CiTO, BiRO,
		bibo, Dublin Core
Taxonomic data	species names	Darwin Core
Ecological data	measurements, units, etc	~~EML~~ OBOE

In another entry I will try and highlight where and what semantic content I have added (work in progress), with examples of each vocabulary. The first four types are relatively static content that can be easily woven into the Jekyll template files. Using Jekyll & Liquid to pull in template information from the _config.yaml should help avoid repetitive entry and make updating the linked data easier. The last three are entry-specific content, and will be more challenging. I hope to add semantic support to knitcitations, including the option for CiTO types, which should make entry of citation data quick and easy (SO question illustrating semantic citation).

The last two are much richer, specific vocabularies. For the moment, it might be best to use these to give more precise meaning to tags, which are already used as metadata on posts. This would allow posts to be still created in simple markdown without the burden of adding in lines of RDFa and cluttering the markup. Meanwhile full datasets provided in EML are likely to live as separate files, rather than as a random table in the middle of a notebook entry.

As always, feedback, corrections or suggestions are appreciated!

Notes

2012-10-11T00:00:00-07:00

EWS review

Composing a simple EWS review highlighting where signals are and are not expected and what this means for the system dynamics. With Alan and Noam. See manuscript repository, cboettig/ews-review.

pdg control

Does the matrix formulation fail to account for the treating profits as a function of realized harvest and true stock size rather than quota and measured size?

notebook

fixing some inline equations 04:34 pm 2012/10/09
Updated Readme 02:57 pm 2012/10/09

scicomp discussions on reproducibility

Asked about longevity of code (re: data managment plan)
Replied to best practices for linking code to publications

discussions/tweets

@hylopsar @mwpennell thanks - I always find it tricky to explain conditional probabilities clearly, so that’s nice to hear
Nice paper by +Matt Pennell, Saver &@lukejharmon on biases in inferring early bursts of evolution. http://t.co/ltRAQcCM12:51 2012/10/10
@figshare thoughts on supporting code and relevant licenses? http://t.co/042Zreni11:19 2012/10/10
my answer to: How to link code to publications http://t.co/9bsazYSj10:15 2012/10/10
Know any tips for increasing the archival longevity of scientific code? http://t.co/PPdzsxfK09:27 2012/10/10
Thanks! RT @noamross Out now in PRSB: Early warning signals and the prosecutor’s fallacy, by @cboettig & Alan Hastings http://t.co/dAq6tlmC09:23 2012/10/10

My Prosecutor's Fallacy paper and other recent warning signals literature

2012-10-10T00:00:00-07:00

My recent paper with Alan Hastings, “Early warning signals and the prosecutor’s fallacy”, is now out in PRSB. An open access preprint can be found on the (arXiv) and accompanying (source code) for the analysis can be found on Github.

Recent papers in early warning signals

This invistigation is in a very similar spirit to two other recent papers on the subject. Kéfi et al. addresses a different kind of false positive for early warning signals, arising in systems approaching other transitions or bifurcations than the intended culprit, the saddle-node bifurcation. Peretti and Munch address the flip side, in which detection fails when it should work due to commonly-observed levels of noise.

Model-based approaches are clearly on the rise (following my own 2012 paper in PRSI and Lade and Gross 2012), with Brock and Carpenter (2012) putting forward a very similar model, and Ives & Dakos (2012) bringing in temporally heterogenous ARMA models.

Unfortunately such model-based approaches are too new to make it into either of the recent review-like comparisons of existing approaches to a common set of simulated data (Dakos et al. 2012) or empirical data (Lindegren et al. 2012).

References

Boettiger, C., & Hastings, A. (2012). Early warning signals and the prosecutor’s fallacy. Proceedings of the Royal Society B: Biological Sciences, (October). doi:10.1098/rspb.2012.2085
Kéfi, S., Dakos, V., Scheffer, M., Van Nes, E. H., & Rietkerk, M. (2012). Early warning signals also precede non-catastrophic transitions. Oikos, (June), no–no. doi:10.1111/j.1600-0706.2012.20838.x
Perretti, C. T., & Munch, S. B. (2012). Regime shift indicators fail under noise levels commonly observed in ecological systems. Ecological Applications, 22(6), 1772–1779. doi:10.1890/11-0161.1
Boettiger, Carl, & Hastings, A. (2012). Quantifying limits to detection of early warning for critical transitions. Journal of The Royal Society Interface, 9(75), 2527 – 2539. doi:10.1098/rsif.2012.0125
Lade, S. J., & Gross, T. (2012). Early Warning Signals for Critical Transitions: A Generalized Modeling Approach. (M. Pascual, Ed.)PLoS Computational Biology, 8(2), e1002360. doi:10.1371/journal.pcbi.1002360* Brock, W. A., & Carpenter, S. R. (2012). Early Warnings of Regime Shift When the Ecosystem Structure Is Unknown. (R. V. Solé, Ed.)PLoS ONE, 7(9), e45586. doi:10.1371/journal.pone.0045586
Ives, A., & Dakos, V. (2012). Detecting dynamical changes in nonlinear time series using locally linear state-space models. Ecosphere, 3(June). Retrieved from http://www.esajournals.org/doi/abs/10.1890/ES11-00347.1
Dakos, V., Carpenter, S. R., Brock, W. a., Ellison, A. M., Guttal, V., Ives, A. R., Kéfi, S., et al. (2012). Methods for Detecting Early Warnings of Critical Transitions in Time Series Illustrated Using Simulated Ecological Data. (B. Yener, Ed.)PLoS ONE, 7(7), e41010. doi:10.1371/journal.pone.0041010
Lindegren, M., Dakos, V., Gröger, J. P., Gårdmark, A., Kornilovs, G., Otto, S. a., & Möllmann, C. (2012). Early Detection of Ecosystem Regime Shifts: A Multiple Method Evaluation for Management Application. (S. Thrush, Ed.)PLoS ONE, 7(7), e38410. doi:10.1371/journal.pone.0038410

Data Management Plan

2012-10-09T00:00:00-07:00

Writing out a data management plan for myself. Suggestions and feedback welcome.

Data Types and Structure

All source code, documentation, scripts and data for the analyses performed in the course of this research shall be maintained in a digital compendium using the R package structure as recommended in Gentleman and Lang (2007). The progress and results of this research shall be regularly chronicled in an electronic lab notebook, maintained openly at carlboettiger.info/lab-notebook.html. Figures and other results are maintained with the source-code required to reproduce them by writing these analyses using knitr dynamic documentation.

Data Acquisition, Integrity and Quality

The lab notebook is written and maintained in plain text (UTF-8) and rendered in HTML5. Likewise, the R package compendium will maintain all source code, scripts and documentation in plain-text (UTF-8) files. Plain-text files with standard encodings help retain compatibility independent of software. Both the notebook entries and the compendium will be maintained in unique git repositories.

Git repositories use unique SHA hashes to protect against corruption. Synchronized backups of the git repositories are maintained on local and remote servers (RAID 6) to protect against hardware failures, as well as on the public international software repository, Github github.com/cboettig. Version history preserves a time-line of changes and protects against user error.

Archival copies of notebook entries shall be published annually to figshare where they are assigned DOIs and preserved by the CLOCKSS geopolitically distributed 12 node global archive. Likewise an archival copy of the R compendium shall be published to figshare at the time of each peer-reviewed publication.

Any data produced by simulations used in these analyses will be archived accordingly in the compendium. Any data associated with a peer-reviewed publication will be deposited in plain text csv files in the Dryad digital repository for the biosciences, unless provided by a third party whose terms of use forbid this. No generation of empirical data is anticipated.

Rights Management & Dissemination

All products generated by this research will be licensed under permissive licenses supporting reuse, re-distribution, and derivatives for free for any purpose without request from a major online repository; Table 1.

Licenses reflect the Panton Principles for open data and the Science Code Manifesto. Distribution uses internationally recognized and freely accessible public repositories best suited for the dissemination of each product.
Product	License	Distribution
publications	cc-by	arXiv
software	FreeBSD	Github, figshare
data	cc-zero	Dryad
research notebook	cc-zero	Github, figshare

Peer-reviewed publications will target preprint-friendly publishers and an author’s preprint will be posted on the arXiv under a Creative Commons Attribution license to facilitate access and distribution.

References

Gentleman R and Temple Lang D (2007). “Statistical Analyses And Reproducible Research.” Journal of Computational And Graphical Statistics, 16. ISSN 1061-8600, doi: 10.1198/106186007X178663

Automated feeds in notebook entries

2012-10-05T00:00:00-07:00

Exploring the use of automated feeds in my notebook entries, which would make writing posts both faster and more complete. I had recently modified my Jekyll plugins to include commits on the day of the post. Consequently, I could add a bit of liquid code to any post, such as:

  {% octokit_commits pdg_control%}

(or even to the post template) and have the commit log for the specified repository automatically imported. While such liquid tags are very handy for pages that are otherwise static and generate their updated content on compile (e.g. my lab-notebook homepage, this approach makes less sense for day-to-day entries (as Noam points out to me). For such entries, the content is already being composed from scratch, but is designed to be fixed once written. (In fact, that was one of the challenges in updating the plugin to pull in the commit log from the day of the post instead of the most recent updates to the repository). Generating this content for every post every time the site is build would be unnecessarily intensive, since it does not change. Further, I like to keep the post entries in pure markdown as much as possible. This makes the actual content not dependent on Jekyll (or on outside databases), which means the content displays fine by browsing the markdown, or using pandoc to compile to pdf or some other format.

To get the same time-savings from automatic imports, I followed Noam’s suggestion and added a little wrapper around my ruby scripts that would import the text of a commit log using a few vi commands. vi, my editor of choice, makes calling arbitrary scripts really easy. In fact one can just write out the bash script command as a line on the editor, and hit !!sh, which replaces the command with its output. So for this purpose I just type the command out into the entry like so,

   commits.rb pdg_control today

I get a listing the commits for that day (defaults to the current day if no day is given). Thanks to the Chronic gem, the script recognizes a variety of natural language commands for the time, such as “3 days ago”. This pulls in all commits made at any point on the day given. For more background, see my vim SO question to pass visual blocks to a function.

Solarized Colors with Twitter-Bootstrap?

2012-10-02T00:00:00-07:00

Trying out rendering my notebook using Ethan Schoonover’s Solarized color scheme. The solarized palette has been selected according to Ethan’s exacting standards for contrast, symmetry, and readability.

My site CSS is built with Twitter-Bootstrap, which uses LESS to generate CSS files. Normal CSS is valid LESS code too, but doesn’t have the concept of a variable. So changing colors in CSS requires changing each and every occurance of some hex colorcode. LESS just lets you define the color as a variable, such as @red: #dc322f; or @background: #fdf6e3;, and then use @red or @background throughout the CSS. It helps you avoid repetition in lots of other ways too, but having variables was enough to get me going.

I mapped all the twitter-bootstrap color variables to the 16-color Solarized palette in my variables.less file. I wanted the ability to render the site with either the dark or light background schemes, which required customizing quite a few of the other Twitter Bootstrap LESS files – it seems Bootstrap was build with exclusively light backgrounds in mind. Solarized accent colors are chosen to provide identical contrast on either background, but the base gray colors are slightly different for light and dark. The Solarized palette indicates which colors are designed for body text, comments and secondary text, background highlights, and optionally, for emphasized content, as described in the usage and development documentation. Overall it’s nice to have a new look to the site, but it has raised a couple questions on which I’d be happy to hear more opinions.

Questions

Links in blue? yellow? Or a more subtle base color?
Light or dark background prefered?
Should code blocks be rendered in the inverse background scheme, or a matching one?
Do the navbar colors work? should they come from the inverse scheme?
Would a stronger contrast in colors be more readable?
other reactions?

Here are a few screenshots in the different themes, varying the codeblock colors as well:

Challenges

My black icons in the footer and lab-notebook homepage do not render well in the dark theme.
Button text doesn’t render well in the dark theme
My flickr-hosted figures render with a white background even if I generate transparent-background pngs, since flickr converts them to jpg.

Updates

I have added a little javascript to allow the user to toggle the theme between dark and light (see switch-css.js)

Multiple Uncertainty

2012-10-01T00:00:00-07:00

Let

y be measured stock size, subject to measurement error from true stock x
q be harvest quota, subject to implementation error from actual harvest, h

Stock next year is also subject to stochastic growth shock , (note that will also depend on the harvest, h, unless is taken as the escapement population, ).

In discrete space, let X be a vector representing the probability distribution of having stock at time , restricted to some finite grid of dimension . Let be the probability density function of the shock over states , given parameter and mean . We can write the distribution as the result of a matrix-vector product, where the elements of F are given by

and is also a function of the harvest level, .

When we consider measurement uncertainty, we allow the vector to represent the measured stock size, and the true stock size is a random variable distributed around it. In the discrete space we may represent this as , where is an by grid representing the role of measurement uncertainty, . The state equation now evolves in belief space,

If there is implementation uncertainty as well and is a function of the escapement population (those remaining after a harvest ), we can impliment this as an additional transition matrix , and the state equation becomes

Note that the implementation matrix is a function of the harvest level, so that we need a different matrix for each , whether or not we introduce measurement uncertainty (instead of a different matrix for each harvest as before.)

Results

Linked in the automatically generated commit log:

render table 10:43 pm 2013/05/18
farm run on writeup using the 50% NPV comparison level note that the asymmetric cases still not working, prossibly NAs induced by non-vector max/min functions… 10:41 pm 2013/05/18
Provide a table of c2 values
- also allow varying the comparison point. set to 50% npv 09:21 pm 2013/05/18
- writeup run using NPV apples-to-apples #35 06:23 pm 2013/05/17
- Working out the Apples-to-Apples case based on total costs, #35 05:34 pm 2013/05/17
- attempt to convert tex and comments to markdown 09:34 pm 2013/05/13
- Tex version with comments from Paul 09:33 pm 2013/05/13
- generalize par uncertainty f 07:57 pm 2013/05/07

Welcome to my Lab Notebook - Reloaded

2012-09-28T00:00:00-07:00

Welcome to my lab notebook, version 3.0. My original open lab notebooks began on the wiki platform OpenWetWare, moved to a personally hosted Wordpress platform, and now run on a Jekyll-powered platform (site-config), but the basic idea remains the same. For completeness, earlier entries from both platforms have been migrated here. Quoting from my original introduction to the Wordpress notebook:

Disclaimer: Not a Blog

Welcome to my open lab notebook. This is the active, permanent record of my scientific research, standing in place of the traditional paper bound lab notebook. The notebook is primarily a tool for me to do science, not communicate it. I write my entries with the hope that they are intelligible to my future self; and maybe my collaborators and experts in my field. Only the occasional entry will be written for a more general audience. […] In these pages you will find not only thoughts and ideas, but references to the literature I read, the codes or manuscripts I write, derivations I scribble and graphs I create and mistakes I make.

Why an open notebook? Is it working?

My original introduction to the notebook from November 2010 dodged this question by suggesting the exercise was merely an experiment to see if any of the purported benefits or supposed risks were well-founded. Nearly three years in, can I draw any conclusions from this open notebook experiment?

In that time, the notebook has seen six projects go from conception to publication, and a seventh founder on a null result (see #tribolium). Several more projects continue to unfold. I have often worked on several projects simultaneously, and some projects branch off while others merge, making it difficult to capture all the posts associated with a single paper into a single tag or category. Of course not all ideas make it into the paper, but they remain captured in the notebook. I often return to my earlier posts for my own reference, and frequently pass links to particular entries to collaborators or other colleagues. On occasion I have pointed reviewers of my papers to certain entries discussing why we did y instead of x, and so forth. Both close colleagues and researchers I’ve never met have emailed me to follow up on something they had read in my notebook. This evidence suggests that the practice of open notebook science can faciliate both the performance and dissemination of research while remaining compatible and even synergistic with academic publishing.

I am both proud and nervous to know of a half dozen other researchers who have credited me for inspiring them to adopt open or partially open lab notebooks online. I am particularly grateful for the examples, interactions, and ideas from established practitioners of open notebook science in other fields. My collaborators have been largely been somewhere between favorable and agnostic towards the idea, with the occasional request for delayed or off-line notes. More often gaps arise from my own lapses in writing (or at least being intelligible), though the automated records from Github in particular, as well as Flickr (image log), Mendeley (reading log), and Twitter and the like help make up for some of the gaps.

The Integrated Notebook becomes the Knitted Notebook

In creating my wordpress lab notebook, I put forward the idea of an “Integrated Lab Notebook”, a somewhat convoluted scheme in which I would describe my ideas and analyses in Wordpress posts, embed figures from Flickr, and link them to code on Github. Knitr simplified all that. I can now write code, analysis, figures, equations, citations, etc, into a single Rmarkdown format and track it’s evolution through git version control. The knitr markdown format goes smoothly on Github, the lab notebook, and even into generating pdf or word documents for publication, never seperating the code from the results. For details, see “writing reproducibly in the open with knitr.”

Navigating the Open Notebook

You can page through the notebook chronologically just like any paper notebook using the “Next” and “Previous” buttons on the sidebar. The notebook also leverages all of the standard features of a blog:

the ability to search,
browse the archives by date,
browse by tag or
browse by category.
follow the RSS feed
add and share comments in Disqus

I use categories as the electronic equivalent of separate paper notebooks, dividing out my ecological research projects, evolutionary research topics, my teaching notebook, and a few others. As such, each entry is (usually) made into exactly one category. I use tags for more flexible topics, usually refecting particular projects or methods, and entries can have zero or multiple tags.

It can be difficult to get the big picture of a project by merely flipping through entries. The chronological flow of a notebook is a poor fit to the very nonlinear nature of research. Reproducing particular results frequently requires additional information (also data and software) that are not part of the daily entries. Github repositories have been the perfect answer to these challenges.

(The real notebook is Github)

My Github repositories offer a kind of inverted version of the lab notebook, grouped by project (tag) rather than chronology. Each of my research projects is now is given it’s own public Github repository. I work primarily in R because it is widely used by ecologists and statisicians, and has a strong emphasis on reproducible research. The “R package” structure turns out to be brilliantly designed for research projects, which specifies particular files for essential metadata (title, description, authors, software dependencies, etc), data, documentation, and source code (see my workflow for details). Rather than have each analysis described in full in my notebook, they live as seperate knitr markdown files in the inst/examples directory of the R package, where their history can be browsed on Github, complete with their commit logs. Long or frequently used blocks of code are written into functions with proper documentation in the package source-code directory /R, keeping the analysis files cleaner and consistent.

The issues tracker connected to each Github repository provides a rich TO DO list for the project. Progress on any issue often takes the form of subsequent commits of a particular analysis file, and that commit log can automatically be appended to the issue.

When scripting analyses or writing papers, pretty much everything can be captured on Github. I have recently added a short script to Jekyll which will pull the relevant commit logs into that day’s post automatically. Other activities fit less neatly into this mold (reading, math, notes from seminars and conferences), so these things get traditional notebook entries. I’m exploring automated integration for other activities, such as pulling my current reading from Mendeley or my recent discussions from Twitter into the notebook as well. For now, feed for each of these appear at the top of my notebook homepage, with links to the associated sites.

Migrating Oww Posts To Jekyll

2012-09-27T00:00:00-07:00

This Open Lab Notebook started on a public MediaWiki-based site, OpenWetWare (OWW), which was created with the ambitious vision of having a single wiki for biological sciences (or at least, synthetic microbiology, since such the wiki grew out of a collection of such labs). Users could have their own pages that served as a personal website or lab group site, contribute pages on specific protocols, host course websites, and most interestingly, were encouraged to create open lab notebooks using specially designed templates for the purpose. From January through October of 2010, I kept all my notes in two separate OWW notebooks: one for entries on my Comparative Phylogenetics research, and a seperate one for entries in Stochastic Population Dynamics. Each notebook had a cover page and then entries by day, linked by the calendar box on the cover page. This was a neat idea an excellent introduction into the world of open lab notebooks.

In September of 2010 I began to leave the OpenWetWare platform to start my own lab notebook based on a privately hosted copy Wordpress. The benefits of being on a shared community platform were minimized by the relatively small number of active users mostly belonging to a different field. Downtime on the site and a lack of any easy way to access my content off-line or back up content also made me increasingly nervous, while the difficulty in adding additional tools or functionality (such as RSS feeds to new entries, personal tags, improved search) collectively spurred my decision to move to Wordpress. At the time I had knew of no easy way to migrate my OWW posts, so I allowed them to remain where they were while I continued further notes in my newly minted Wordpress installation.

In April of 2012 I began to experiment with the static-blogging engine, Jekyll. My adoption of markdown, facilitated by my discovery of pandoc and the advent of knitr, together with my complete submersion into a Github-based workflow led me to first consider Jekyll. Wordpress was slow and heavy, both for creating entries in the online editor (though I did start pushing markdown-based entries to Wordpress through it’s API and the associated R package). Though Jekyll has a slightly steeper learning curve to start, once you have the basics down complicated things aren’t any harder than easy ones, whereas on Wordpress the basics are trivial but complex customization becomes a full-time web-developer challenge. In Jekyll, it was rather easy to migrate all my old entries from the previous platforms onto the new one.

In the previous post I describe exporting my Wordpress-based entries using a python script from Thomas Frossman. To export my OpenWetWare entries I relied on the rather excellent MediaWiki API. While I began by grabbing the content of each post in MediaWiki syntax, I soon realized it was easier to grab the content already parsed as HTML (using a different API call) and then parse the HTML into markdown using Pandoc. (Pandoc’s developmental version does have experimental support for reading MediaWiki syntax now). I then added a few substitutions to standardize my tags and categories. Here’s my R call to the MediaWiki API (see wiki_get.R):

wiki_parse <- function(page, baseurl, format="json", ...){
  require(httr)
  action = "parse"
  addr <- paste(baseurl, "/api.php?format=", format, "&action=", action, "&page=", page, sep="")
  config <- c(add_headers("User-Agent" = "rwiki"), ...)
  out <- GET(addr, config=config)
  parsed_content(out)
}

I create a few helper functions for parsing the html into markdown, and also extracting and adding the metadata into Jekyll’s YAML headers (see export_oww.R)

require(httr)
require(gsubfn)

oww_to_md <- function(pages, user="Carl_Boettiger",  baseurl = "http://openwetware.org"){
  # Store filenames (with dates) and categories for each page, for reference later
  filenames <- get_filenames(pages)
  categories <- get_categories(pages)
  ## Use the API function to extract the content
  parsed <- export_oww(pages, user, baseurl)
  ## Add the baseurl back into all the links
  htmls <- add_baseurl(parsed)
  ## Create YAML header text
  headers <- add_header_txt(parsed, categories)
  ## Use pandoc to convert the html to markdown
  mds <- html_to_md(htmls, filenames) 
  ## Delete the OWW header info we don't want in the markdown
  mds <- clean_md(mds)
  ## Stick headers onto markdown files and name them according to filenames
  write_mds(mds, headers, filenames) 
}

get_filenames <- function(pages){
  lapply(pages, function(p){
    p <- gsub(" ", "_", p)
    filename <- gsub("/", "-", p)
    filename <- gsub("(.*)-(\\d+-\\d+-\\d+)", "\\2-\\1", filename)
  })
}

#' Take a list of page titles to a user's OWW notebooks and export them as Jekyll markdown entries.  
get_categories <- function(pages){
  lapply(pages, function(p){
    p <- gsub("_", " ", p)
    filename <- gsub("/", "-", p)
    category <- gsub("(.*)-(\\d+-\\d+-\\d+)", "\\1", filename)
  })
}


#' Take a list of page titles to a user's OWW notebooks and export them as Jekyll markdown entries.  
export_oww <- function(pages, user="Carl_Boettiger",  baseurl=  "http://openwetware.org"){
  lapply(pages, function(p){
    p <- gsub(" ", "_", p)
    page <- paste("User:", user, "/Notebook/", p, sep="")
    out <- wiki_parse(page, baseurl=baseurl)
  })
}

add_baseurl <- function(parsed,  baseurl = "http://openwetware.org"){
  lapply(parsed, function(out){
    html <- out$parse$text[[1]]
    html <- gsubfn("src=\"/images/", paste("src=\"", 
                   baseurl,"/images/", sep=""), html)
    html <- gsubfn("href=\"/wiki/", paste("src=\"", 
                   baseurl,"/wiki/", 
                    sep=""), html)
  })
}

add_header_txt <- function(parsed, mycategories){
  sapply(1:length(parsed), function(i){
    out <- parsed[[i]]
    category <- mycategories[[i]]
    if(length(out$parse$categories)>0){
      categories <- sapply(out$parse$categories, function(x) x$`*`)
      gsub("_", " ", categories)
      ## Standarize some of my tags, called "categories" on OWW
      tags <- paste("tags: ", "[", 
                          paste0(categories, collapse = ", "),
                          "]", sep="")  
    } else {
      tags <- ""
    }
    
    header <- c("---", 
                "layout: post",
                tags, 
                paste("categories: ", category),
                "---\n\n")
  })
}


html_to_md <- function(htmls, filenames){
  lapply(1:length(htmls), function(html){
    writeLines(htmls[[i]], paste(filenames[i], ".html", sep=""))
    md <- system(paste("pandoc ", filenames[i], ".html -w markdown", 
                       sep=""), intern=TRUE)
  })
}


clean_md <- function(mds){
  lapply(mds, function(md){
    ## Remove OWW header
    md <- gsub("!\\[image\\]\\(http://openwetware.org/images/f/f8/Owwnotebook_icon.png\\)", "", md)
    md <- gsub("!\\[image\\]\\(http://openwetware.org/images/9/94/Report.png\\)",  "", md)
    md <- gsub("Main project", "", md)
    md <- gsub("page\\\\", "", md)
    md <- gsub("!\\[image\\]\\(http://openwetware.org/images/c/c3/Resultset_previous.png\\)",  "", md)
    md <- gsub("Previous$",  "", md)
    md <- gsub("^entry Next$", "", md)
    md <- gsub("!\\[image\\]\\(http://openwetware.org/images/5/5c/Resultset_next.png\\)", "", md)
    md <- gsub("^entry$", "", md)
    md <- gsub("^Stochastic Population Dynamics", "", md)
    md <- gsub("^Comparative Phylogenetics$", "", md)
    md <- gsub("^Teaching$", "", md)
  })
}



write_md <- function(mds, headers, filenames){
  lapply(1:length(mds), function(i){
    writeLines(c(headers[[i]], mds[[i]]), paste(filenames[i], ".markdown", sep="")) 
  })
}

If or when I change platform again, it should be even easier. With all my posts in markdown, I can easily move them into an ebook or pdf using Pandoc (currently requires stripping the YAML headers first, though it would be great if Pandoc eventually supported metadata headers in YAML). Github renders the markdown of each post automatically, so the content of most posts can be read directly from the Github repository, cboettig/labnotebook.

Misc tricks

The unix rename utility comes in handy:

rename 's/Comparative_Phylogenetics/Comparative-Phylogenetics-Notebook-(from-OWW)/' *Comparative_Phylogenetics.markdown
rename 's/Stochastic_Population_Dynamics/Stochastic-Population-Dynamics-Notebook-(from-OWW)/' *Stochastic_Population_Dynamics.markdown

And a vim mapping to flip through 100s of source files quickly:

:map <F2> :N<CR>
:map <F3> :n<CR>

Migrating From Wordpress To Jekyll

2012-09-19T00:00:00-07:00

Thanks to a recent bugfix in the exitwp scripts I was able to export all my Wordpress entries. The script pulls the entries from the Wordpress database and formats them in markdown, along with extracting all metadata such as timestamp, tags, categories, publication status, and Wordpress id number, which are all embedded as YAML header information.

Why migrate the old entries?

When I moved to the Jekyll notebook, I left the Wordpress site standing with all its existing content and simply remapped the home pages. While it was nice to have the Wordpress system as a fall back while I got used to Jekyll, it meant that I couldn’t benefit from the faster and lighter nature of Jekyll while much of my traffic occurred on the Wordpress pages. I was running a dedicated virtual private server for the Wordpress hosting, and still the site wasn’t very responsive. To get an idea, try “flipping through the pages” of the new Jekyll notebook using the “next” and “previous” post buttons. The new pages load immediately, making it easy to flip through looking for some word or image I can’t find with tags or searches. Flipping through pages is the time-honored way of interacting with paper notebooks, so I’m glad that the new system is responsive enough to imitate this well.

Converting shortcodes from Wordpress Plugins

Migrating files using exitwp is a relatively seamless affair. Unfortunately, my Wordpress posts make rather heavy use of plugins, which work through shortcodes that still appear in the converted files, such as [flickr], [latex], [code lang="r"], and [cite], for image embedding, equation embedding, syntax highlighting, and citations by DOI, respectively. While a suite of Jekyll plugins exist for each of these tasks, I’d like my regular posts to be pure markdown (other than the YAML header), making it easy to migrate or reuse the content in the future in different platforms. Here is the R script I used for converting my shortcodes, as described below.

Syntax Highlighting short-codes

For instance, several markdown extension languages including, Github-flavored markdown, already allow for syntax-highlighting with fenced code blocks. I wrote a short script to replace all (most) of the shortcodes in use on my Wordpress site. (Over the years, I’d actually used a variety of different syntax highlighting plugins, making it necessary to match a variety of different shortcodes). Other than matching a lot of patterns, this was pretty straight forward.

flickr images

My workflow pushes all the images and graphs I create to flickr when the script runs. (I could also automatically push the images to Wordpress, or simply to my own site, but started using flickr pretty early on for this. The SOAP API is excellent, as is the Rflickr R package) Images were embedded into my Wordpress site using the shortcode and image ID number. To generate stand-alone markdown, I wanted tp replace all the image codes with the flickr URLs to the images, which is a trivial task with the API and Rflickr.

LaTeX/Mathjax Equations

Again I had used a variety of shortcodes for latex plugins before discovering MathJax, so I had to handle a variety of syntaxes here. Previously I’ve tried to write equations in a manner compatible with the redcarpet markdown parser that powers Github-Flavored Markdown (see the corresponding Jekyll plugin), e.g. surrounding equations with <div> blocks to avoid the parser mangling them. Instead, I’ve found it easier now to switch to pandoc for my markdown parser (and corresponding Jekyll Plugin). Pandoc uses the basic TeX $ and $$ for inline and display equations, rendering it into the MathJax (and modern LaTeX standard) \( and \[ on conversion. Being pandoc-compatible is quite nice, making it easy to create an e-book, epub, or latex-based pdf of my notebook. I hate having to use a markdown syntax that isn’t compatible with most other extended markdown parsers, but with so many different, inconsistent parsers this cannot be easily avoided. Regardless, replacing the various shortcodes to the Pandoc syntax is just a simple regexpr step.

Citations

Many of my posts relied on kcite to generate bibliographies for the post using DOIs. A short call to my knitcitations package allowed me to generate the parenthetical citations and bibliographies instead. In principle this step could be run at the time I compile the site with Jekyll. Pages would be valid .Rmd, needing knitr as “markdown” interpreter. This is an intriguing way to go, but probably too complicated for the moment. Running additional R code would be nice, but could slow the time needed to build the site, even with knitr’s caching. Meanwhile, converted pages simply have the citations already processed and the bibliography added in as HTML at the end of the post.

URL Redirects

My Jekyll site uses the SEO recommended structure of year/month/day/page-title for URLs, while my Wordpress site used simple random id numbers. To ensure that links to the old Wordpress pages resolve to the newly migrated pages, I added a little Jekyll plugin, redirects.rb. One of the elegant things about Jekyll is the ability to create such plugins that do this just the way you would image doing so – creating auxiliary pages at each of the redirect URLs with a simple redirect command. To install, place redirects.rb in _plugins, redirects.html in _layouts, and in _config.yml add the line redirects: yes (see site source).

Comments

I am using Disqus as the comment engine for entries. Since URLs change in the above step and I do not provide a unique disqus code to each page, I am attempting to migrate comments using the Disqus “migrate threads” tool, which simply takes a csv file listing old and new urls in consecutive columns. Will have to wait and see if this works.

Private entries

Several entries on the Wordpress site were marked private, and only visible after administrative login. Most of these entries are actually solicited journal reviews of various manuscripts which I cannot make public. exitwp marks these posts as published: false in the metadata, so it is easy to remove them from the Jekyll source site. They can be managed in a separate Github-excluded directory (so that they cannot be seen on the website source pages on Github either) that can be .htaccess password protected.

Categories and Tags

Categories and tags are handled automatically in the conversion. One of the greatest things about having all the content on a common system is the ability to share a common tag-pool.

RSS

If you follow the notebook through RSS feeds, all the migrated posts will be showing up as new unread entries, as the atom feed dates entries by when they appear, not the stated post date. My apologies!

Rough edges

A couple posts have rare shortcodes which are probably easier to correct by hand. My apologies for any rough edges resulting from the conversion (or the exchange of markdown parsers). Hopefully I’ll get these ironed out in the near future, but feel free to leave a comment on the page or the issues tracker on any such errors.

Analytic Solution To Multiple Uncertainty

2012-09-14T00:00:00-07:00

No idea where I went wrong, but the final graphs are not what I expected Update: Integrated out some uncertainty in the expectation step, need to preserve full set of transtions

We consider adding additional sources of noise through observation error and implementation error to the orginal model of Reed (1979), which considers only growth error. This follows in the line of exploration taken by Clark and Kirkwood (1987), Roughgarden (1996) and Sethi (2006).

In adding these two sources of error, our state variable becomes our measured (or believed) stock , rather than the true stock size , and our control variable becomes the quota set, rather than the realized harvest .

Analytical integrals

The central calculation to accomodating additional sources of uncertainty is the determination of the transition matrix; the probability of going from state at time to state in the next interval, for each possible value of the control variable (quota). We discritize the statespace onto a fine grid of values and to seek a stochastic dynamic programming solution.

xmin <- 0
xmax <- 150
grid_n <- 100

We seek a harvest policy which maximizes the discounted profit from the fishery using a stochastic dynamic programming approach over a discrete grid of stock sizes from 0 to 150 on a grid of 100 points, and over an identical discrete grid of possible harvest values.

x_grid <- seq(xmin, xmax, length = grid_n)
h_grid <- x_grid

With purely growth noise, a row of the matrix is given by the probability density of the growth noise with mean , that is, the probability of tranistion from to at harvest is given by the function for some probability density function . In the case of uncertainty in stock and harvest , we must integrate over the probability distributions for possible harvests and possible stock sizes,

In general this convolution can be non-trivial to compute, but in the case of uniformly distributed harvest error of around some quota , and uniform measurement error of around the true stock size , the integrals can be written as,

Note we enforce the simple non-negative boundary on stock and harvest.

So now we are left to merely integrate the over these finite intervals. We can do this for arbitrary through multidimensional numerical integration (In fact we could have done this in the first place, but it is too slow for the convolutions of arbitrary probability densities)

int_f

function(f, x, q, sigma_m, sigma_i, pars){
  K <- pars[2]
  sigma_m <- K*sigma_m
  sigma_i <- K*sigma_i # scale noise into units of K
  
  if(sigma_m > 0 && sigma_i > 0){
    g <- function(X) f(X[1], X[2], pars)
    lower <- c(max(x - sigma_m, 0), max(q - sigma_i, 0))
    upper <- c(x + sigma_m, q + sigma_i)
    A <- adaptIntegrate(g, lower, upper)
    out <- A$integral/((q+sigma_i-max(q-sigma_i, 0))*(x+sigma_m-max(x-sigma_m, 0)))
  } else if(sigma_m == 0 && sigma_i > 0){ 
    g <- function(h) f(x, h, pars)
    lower <- max(q - sigma_i, 0)
    upper <- q + sigma_i
    A <- adaptIntegrate(g, lower, upper)
    out <- A$integral/(q+sigma_i-max(q-sigma_i, 0))
  } else if(sigma_i == 0 && sigma_m > 0){ 
    g <- function(y) f(y, q, pars)
    lower <- max(x - sigma_m, 0)
    upper <- x + sigma_m
    A <- adaptIntegrate(g, lower, upper)
    out <- A$integral/(x+sigma_m-max(x-sigma_m, 0))
  } else if (m == 0 && n == 0){
    out <- f(x,q,pars)
  } else {
    stop("distribution widths cannot be negative")
  }
  out
}
<environment: namespace:pdgControl>

If we assume a simple function like logistic growth, , where ,

f <- function(x, h, p) {
    sapply(x, function(x) {
        S = max(x - h, 0)
        p[1] * S * (1 - S/p[2]) + S
    })
}

With parameters

r <- 1
K <- 100
pars <- c(r, K)

r = 1 and K = 100.

Following Sethi et al, we can do this in closed form,

function(x,q, m, n, pars){
  K <- pars[2]
  m <- K*m
  n <- K*n # scale noise by K
  if(m > 0 && n > 0){
  out <- ((q+n-max(0,q-n))*(x+m-max(0,x-m))*(6*x*K-6*q*K-6*n*K+6*m*K+6*max(0,x-m)*K-6*
    max(0,q-n)*K-2*x^2+3*q*x+3*n*x-4*m*x-2*max(0,x-m)*x+3*max(0,q-n)*x-2*q^2-4*n*q+3*m*q+3*
    max(0,x-m)*q-2*max(0,q-n)*q-2*n^2+3*m*n+3*max(0,x-m)*n-2*max(0,q-n)*n-2*m^2-2*max(0,x-m)*m+3*
    max(0,q-n)*m-2*max(0,x-m)^2+3*max(0,q-n)*max(0,x-m)-2*max(0,q-n)^2))/(6*K)/((q+n-max(q-n, 0))*(x+m-max(x-m, 0)))
  } else if(m == 0 && n > 0) {
    y <- x
    out <- (((6*q+6*n-6*max(0,q-n))*y-3*q^2-6*n*q-3*n^2+3*max(0,q-n)^2)*K+(-3*q-3*n+3*max(0,q-n))*
      y^2+(3*q^2+6*n*q+3*n^2-3*max(0,q-n)^2)*y-q^3-3*n*q^2-3*n^2*q-n^3+max(0,q-n)^3)/(3*K*(q+n-max(q-n, 0)))
  } else if(n == 0 && m > 0){
    h <- q
    out <- ((3*x^2+(6*m-6*h)*x+3*m^2-6*h*m+6*max(0,x-m)*h-3*max(0,x-m)^2)*K-x^3+(3*h-3*m)*x^2+
      (-3*m^2+6*h*m-3*h^2)*x-m^3+3*h*m^2-3*h^2*m+3*max(0,x-m)*h^2-3*max(0,x-m)^2*h+max(0,x-m)^3)/(3*K*(x+m-max(x-m, 0)))
  } else if (m == 0 && n == 0){
    S <- max(x - q, 0)
    out <- S * (1 - S/K) + S
  } else {
    stop("distribution widths cannot be negative")
  }
  max(out,0)
}
<environment: namespace:pdgControl>

We can compare timing and equivalence of these two expressions:

require(cubature)
# Confirm that these give the same value, and time performance
system.time(a <- sapply(x_grid, function(x) int_f(f, x, 1, 0.1, 0.1, 
    pars)))

   user  system elapsed 
  7.444   0.000   7.394

system.time(b <- sapply(x_grid, function(x) F(x, 1, 0.1, 0.1, pars)))

   user  system elapsed 
  0.012   0.000   0.015

ggplot(data.frame(x = x_grid, a = a, b = b)) + geom_line(aes(x, a), 
    col = "red") + geom_line(aes(x, b), lty = 2)

plot of chunk unnamed-chunk-6

Note that as the uncertainy gets small, we recover the original transition probability :

F(50, 4, 0.1, 0.1, pars)

[1] 67.01

F(50, 4, 1e-04, 1e-04, pars)

[1] 70.84

f(50, 4, pars)

[1] 70.84

When the noise width is much smaller than the bin width we may have trouble, particularly with the uniform distribution, since part of the bin corresponds to zero density, part to high density.

Numerical implementation

We consider a profits from fishing to be a function of harvest h and stock size x, , conditioned on and ,

price <- 1
c0 <- 0
c1 <- 0
profit <- profit_harvest(price = price, c0 = c0, c1 = c1)

with price = 1, c0 = 0 and c1 = 0.

Additional parameters

delta <- 0.05
xT <- 0
OptTime <- 25
sigma_g <- 0.1
sigma_m <- 0.5
sigma_i <- 0.5

The uniform distribution must be careful for noise sizes smaller than binwidth

pdfn <- function(P, s) dunif(P, 1 - s, 1 + s)
# pdfn <- function(P, s) dlnorm(P, 0, s)

We will determine the optimal solution over a 25 time step window with boundary condition for stock at 0 and discounting rate of 0.05.

Scenarios:

sdp <- SDP_uniform(f, pars, x_grid, h_grid, sigma_g = sigma_g, pdfn, 
    sigma_m = sigma_m, sigma_i = sigma_i, F)

uniform <- find_dp_optim(sdp, x_grid, h_grid, OptTime, xT, profit, 
    delta, reward = 0)

Do the deterministic exactly,


SDP_Mat <- determine_SDP_matrix(f, pars, x_grid, h_grid, sigma_g = sigma_g, 
    pdfn)
det <- find_dp_optim(SDP_Mat, x_grid, h_grid, OptTime, xT, profit, 
    delta, reward = 0)

Determine the policies for each of the scenarios (noise combinations).

plots

require(reshape2)
# policy <- melt(data.frame(stock = x_grid, deterministic=det$D[,1],
# uniform = uniform$D[,1], mc=opt$D[,1]), id = 'stock')
policy <- melt(data.frame(stock = x_grid, deterministic = det$D[, 
    1], uniform = uniform$D[, 1]), id = "stock")

ggplot(policy) + geom_jitter(aes(stock, stock - x_grid[value], color = variable), 
    shape = "+")

dat <- subset(policy, stock < 140)
dt <- data.table(dat)
linear <- dt[, approx(stock, stock - x_grid[value], xout = seq(1, 
    150, length = 15)), by = variable]
ggplot(linear) + stat_smooth(aes(x, y, color = variable), degree = 1, 
    se = FALSE, span = 0.3) + xlab("Measured Stock") + ylab("Optimal Expected Escapement")

Notes

2012-09-12T00:00:00-07:00

{% octokit_commits pdg_control%}

Esa Changes Arxiv Policy Following Community Comments

2012-09-05T00:00:00-07:00

Earlier today, Scott Collins, the president of the Ecological Society of America has announced that the society will now accept articles that have previously been posted on preprint servers. This comes on the heels of a growing discussion in our community. Ethan White has a good summary over on Jabberwocky Ecology.

Many voices have joined the discussion over the past month, and it is exciting and vindicating to see the Society engage and discuss these questions. With this announcement out, I thought I might share my original letter. Here’s the text of an email I sent on August 1st to Don Strong, editor-in-chief of Ecology and a friend to open science known for his bottom-up view that ecologists should show their publication preferences by their actions. Don kindly forwarded this email to other members of the board.

Dear Don,

Perhaps you have already seen the perspective appearing in Nature this week, “Geneticists eye the potential of Arxiv.” I must say it is particularly saddening to see the Ecological Society of America being singled out in the discussion as a professional society that opposes pre-prints, particularly given that ESA is not beholden to corporate publishers, and in the face a rising swell of interest in pre-print server capacity, as evidenced by the Nature article (which quotes our own Graham Coop), the emergence of an NCEAS working group to bring about a preprint server, and the extensive online discussions about ESA’s stance against pre-prints which was no doubt responsible for bringing ESA into the negative limelight of this perspective.

The discussion launched by Professor Ethan White on his blog suggests that ESA has in fact only recently removed a clause in the policies explicitly permitting the use of the arXiv. Meanwhile members of the NCEAS working group had considered approaching ESA to partner in their efforts to bring about a preprint server and culture, hoping to take advantage of the agility ESA has as a respected and independent society publisher. Ecology as a field has been a leading example of pushing for innovative and open publishing practices such as mandatory data archiving, and ESA has led this front for decades with innovations such as Ecological Archives and more recent Ecosphere. It shames me to see our society and our field painted as a backwater of regressive policies in such prominent magazines when even journals such as Nature, Science, & PNAS permit and encourage the use of preprint servers such as the arXiv. I would hate to see our most innovative research migrate away from the Society at a time when ESA could be leading our field through this academic publishing transition that is now discussed broadly in the NY Times, Guardian, Economist, & USA Today. This is a chance for the Ecological Society to both lead and flourish.

I hope that these issues can be discussed in some fashion with the input of our community at the annual meeting in Portland. The British Ecological Society already plans to use the meeting to discuss a digital future of their journals, surely ESA will be doing as much at it’s own meeting, formally or informally?

Thank you for your time and consideration. Please let me know if there is anything I can do to help.

Cheers,

Carl

Notes

2012-09-04T00:00:00-07:00

Fallacy paper done
Dissertation filing
Decision theory from the ROC curve.
Working group paper
Jim/Mike paper
wrightscape paper?
Signature page two copies, archival paper?
ropensci emails: Position paper: Goals
misc emails

Fallacy paper finalizing appendices

Commit images and runs from yesterday (ibm simulation still running…)
clean up appendix code
clean up appendix text

Code tricks for today

vim set sytnax language, useful for getting R highlighting in .Rmd files: setf r, or equivalently, set filetype=r (must be lowercase r). set filetype with no argument gives the current filetype.

Coding

cboettig opened issue SChamberlain/isqaltms#8: section: technical barriers to use 11:14 2013/05/21
cboettig opened issue SChamberlain/isqaltms#7: Section: Provenance 11:07 2013/05/21
cboettig opened issue SChamberlain/isqaltms#6: Section: Putting in context 11:04 2013/05/21
cboettig opened issue SChamberlain/isqaltms#5: Section: Open metrics 11:00 2013/05/21
cboettig opened issue SChamberlain/isqaltms#4: Section: standards and consistency 10:56 2013/05/21
cboettig opened issue SChamberlain/isqaltms#3: Altmetrics data providers section 10:45 2013/05/21
cboettig opened issue SChamberlain/isqaltms#2: cite wherever Jason coined the term altmetrics 10:40 2013/05/21
cboettig opened issue SChamberlain/isqaltms#1: intro doesn’t mention time-lag issue 10:40 2013/05/21

Reading

Signature of ocean warming in global fisheries catch: Nature (2013). Volume: 497, Issue: 7449. Pages: 365-368. William W. L. Cheung, Reg Watson, Daniel Pauly et al. 04:38 2013/05/18
Incorporating uncertainty associated with habitat data in marine reserve design: Biological Conservation (2013). Pages: 41-51. Vivitskaia J. Tulloch, Hugh P. Possingham, Stacy D. Jupiter, Chris Roelfsema, Ayesha I.T. Tulloch, Carissa J. Klein et al. 04:38 2013/05/18
Innovations in capture fisheries are an imperative for nutrition security in the developing world: Proceedings of the National Academy of Sciences (2013). Pages: 1-6. S. J. Hall, R. Hilborn, N. L. Andrew, E. H. Allison et al. 04:38 2013/05/18

Santa Cruz Postdoc

2012-08-29T00:00:00-07:00

Spent Monday and Tuesday visiting Santa Cruz, where I have recently accepted a post-doctoral position with Steve Munch, Marc Mangel, and Alec MacCall. I will be based primarily at the National Marine Fisheries Service building, here: `

Five fantastic years of graduate school are coming to an end: I will start officially later this fall, where I will

join an exciting team of ecologists and applied mathematicians who work collaboratively on cutting-edge problems in ecological and evolutionary modeling through the Center for Stock Assessment Research. I will work with us to develop non-parametric approaches to ecosystem-based management based on semi-parametric Bayesian methods for nonlinear time-series modeling …

or so the job description goes. I am looking forward to it.

Notes

2012-08-23T00:00:00-07:00

Fallacy with May model runs

plot of chunk figure2

Working on unmanaged warning. (see #21)

Warning Signals In Management

2012-08-22T00:00:00-07:00

Tour of existing files (version stable links)

managed warning is the current direct test of this, but needs decently ranged input example where detection has a good chance to succeed.
criticaltransition Statistics across replicate crashes that don’t show much promise for detection.
A heuristic implementation in response to the detection of a signal might use a simple rule like a fraction of the optimal harvest of a stable system. cautious.md simply shows the economic cost associated with this fractional harvest.
resilience_assessment not run

Also compare to

Stability assessment, estimating coefficient ; surprisingly difficult to distinguish the fished from the unfished by this metric.

Revisions on Prosecutors Fallacy

2012-08-22T00:00:00-07:00

Email Alan re: Prosecutor’s fallacy manuscript. Schedule meeting.
Proof corrections for rfishbase manuscript. Done.
Fallacy simulations: calibration range for May model: expect small fraction of runs to transition by chance from stable state.

Here we’re getting a bit less than 2% chance transitions in 5000 steps.

>   sn <- 
+   sapply(1:1000, function(rep){
  +     x <- vector(mode="double", length=n)
  +     x[1] <- 8 # positive equilibrium
  +     z <- rlnorm(n, 0, .1)
  +     r = .75; k = 10; a=1.55; H=1; Q = 3
  +     for(t in 1:n){
    +       x[t+1] = z[t] *  x[t] * exp(r * (1 - x[t] / k) - a * x[t] ^ (Q - 1) / (x[t] ^ Q + H ^ Q)) 
    +     }
    +     x
    +   })
> crashed <- which(sn[n,] < 2)
>   length(crashed)/1000
[1] 0.017
> 
> n
[1] 5000

Fallacy simulation analysis running on farm using May model. See fallacy.Rmd, prosecutor branch
rerun using different point to indicate transition. Tracking runs on Issue 5
add figure plotting b and d curves, or at least b-d curve. Likewise for May model.

Misc

Exploring twitteR finding publications under discussion
Migrated labnotebook plugins to separate repository
Updated vimrc to preview pandoc:

map \md : call PreviewMarkdown()<CR>
func! PreviewMarkdown()
exec "w"
exec "! pandoc -s % -o .temp.html"
exec "! google-chrome .temp.html"
endfunc

Should add css templates from my notebook or github.
Could also parse with github’s own API instead of pandoc, e.g. with ruby:

#!/usr/bin/env ruby

# Example usage 
# ./render_gfm.rb input.md > output.html
# 

require 'octokit'
puts Octokit.markdown(File.read(ARGV.first))

twitteR authentication keys configured in .Rprofile

Pdg Control Updates

2012-08-21T00:00:00-07:00

Resolving S < D

Code updates

octokit

Reading

Macksamie, K., Cockburn, J., & Wagner, J. (2012). Robust control amidst stock and flow benefits: The case of wolf-elk-hunting dynamics in the US Mountain West. 2012 AERE Summer Conference, Asheville, NC (pp. 1–33). Retrieved from http://www.webmeets.com/files/papers/AERE/2012/3/AEREConfPaper05172012.pdf

The Linear parameter varying (LPV) approach which allows this problem to leverage a standard LMI (linear matrix inequality) solver seeems like a clever route, but unclear if this would be restricted to the additive noise models such as they consider?

Some background on LPV:

Becker, G., & Packard, A. (1994). Robust performance of linear parametrically varying systems using parametrically-dependent linear feedback. Systems & Control Letters, 23(3), 205–215. doi:10.1016/0167-6911(94)90006-X
Brock, W. a., & Durlauf, S. N. (2005). Local robustness analysis: Theory and application. Journal of Economic Dynamics and Control, 29(11), 2067–2092. doi:10.1016/j.jedc.2005.06.001

An interesting example handling Knightian uncertainty about the model directly:

Athanassoglou, S., & Xepapadeas, A. (2012). Pollution control with uncertain stock dynamics: When, and how, to be precautious. Journal of Environmental Economics and Management, 63(3), 304–320. doi:10.1016/j.jeem.2011.11.001

Notes

2012-08-20T00:00:00-07:00

Writing

Draft outline of policy costs paper sent to Paul.

Misc

ropensci NCEAS proposal - contacts
MEE supplement follow-up: Wow, MEE promotes video podcasts along with publications.
Hadley to join RStudio full time.
A gist by Hadley on storing things like API keys across R packages.
Joined fisheries-science mailing list

Reading

Warning signals

Critical transitions at Amer Fisheries Society
Ives, A., & Dakos, V. (2012). Detecting dynamical changes in nonlinear time series using locally linear state-space models. Ecosphere, 3(June). Retrieved from http://www.esajournals.org/doi/abs/10.1890/ES11-00347.1
Feng, J., Dakos, V., & van Nes, E. H. (2012). Does predator interference cause alternative stable states in multispecies communities? Theoretical population biology. doi:10.1016/j.tpb.2012.06.003
Katz, R. W., Brush, G. S., & Parlange, M. B. (2005). Statistics of Extremes : Modeling Ecological Disturbances, 86(5), 1124–1134.

More ABC methods in Evolution

Kutsukake, N., & Innan, H. (2012). Simulation-Based Likelihood Approach for Evolutionary Models of Phenotypic Traits on Phylogeny. Evolution, (1), no–no. doi:10.1111/j.1558-5646.2012.01775.x

Methods

Might be a good paper for algorithms group to read.

Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., et al. (2007). Top 10 algorithms in data mining. Knowledge and Information Systems (Vol. 14, pp. 1–37). doi:10.1007/s10115-007-0114-2

And a good one for theory tea?

De Boer, R. J. (2012). Which of Our Modeling Predictions Are Robust? (R. Antia, Ed.)PLoS Computational Biology, 8(7), e1002593. doi:10.1371/journal.pcbi.1002593

Papers following up from ESA

Regime shifts in disease models

Chaves, L. F., Hashizume, M., Satake, A., & Minakawa, N. (2012). Regime shifts and heterogeneous trends in malaria time series from Western Kenya Highlands. Parasitology, 139(1), 14–25. doi:10.1017/S0031182011001685
Davies, T. D., & Baum, J. K. (2012). Extinction Risk and Overfishing: Reconciling Conservation and Fisheries Perspectives on the Status of Marine Fishes. Scientific Reports, 2, 1–9. doi:10.1038/srep00561
Fries, R. S. D., Chapin, F. S., & Syvitski, J. (2012). Planetary Opportunities: A Social Contract for Global Change Science to Contribute to a Sustainable Future. BioScience, 62(6), 603–606. doi:10.1525/bio.2012.62.6.11
Shelton, a. O., & Mangel, M. (2011). Reply to Sugihara et al: The biology of variability in fish populations. Proceedings of the National Academy of Sciences, 108(48), E1226–E1226. doi:10.1073/pnas.1115765108
Shelton, A., Dick, E., & Pearson, D. (2012). species composition and quantifying uncertainty in multispecies fisheries: hierarchical Bayesian models for stratified sampling protocols with missing data. … Journal of Fisheries …, 246(5007), 231–246. doi:10.1139/F2011-152
Shelton, A. O., & Mangel, M. (2011). Fluctuations of fish populations and the magnifying effects of fishing. Proceedings of the National Academy of Sciences of the United States of America, 108(17), 7075–80. doi:10.1073/pnas.1100334108

Esa 2012 Conference

2012-08-10T00:00:00-07:00

My Schedule
Meetings/discussion with Don Strong, Tony Ives, Marten Scheffer, Justin Yeakel, Karen Abbott, Jarrett Byrnes, Christopher Lortie.
Check in with Paul Armsworth on working group
rOpenSci tutorial session (below)
tweetup
My contributed talk (below)

rOpenSci tutorial, Thursday

A year out from our launch at ESA last year, we presented an introduction and tutorial on rOpenSci. Slides and links to more information are all available at esa.ropensci.org.

Slides from my talk, Friday

Twitter Notes (generated from R)

library(twitteR)
hashtag <- "#ESA2012"
user <- "cboettig"

Generate a list of a user’s conference tweets

tweets <- twListToDF(userTimeline(user, n = 500))
tagged <- grep(hashtag, tweets$text, ignore.case = TRUE)

Write to file, making links appear as links

text <- tweets$text[tagged]
text <- gsub("(@[A-Za-z0-9_]+)", "<a href=\"http://twitter.com/\\1\">\\1</a>", 
    text)
text

My twitter notes

[1] “.@luketkelly uses GAMM for species response from time since fire, interval, intensity, etc, then solves SDP for optimal policy #esa2012”
[2] “.@luketkelly optimizes distrib of time since fire to maximize geom mean of biodiversity - better than arithmetic or shannon. #esa2012”
[3] “.@luketkelly on optimal fire histories for conserving biodiversity #esa2012 w/ amazing images from down under”
[4] “Armsworth’s analysis in eco let http://t.co/w5W8Ubsa #esa2012”
[5] “Armswortth shows paying farmers for biodiversity gains is best when spatially targeted & worse with multispecies targets #esa2012”
[6] “Fun at #esa2012 where a physics PhD student presents the experimental study testing the mathematical theories presented by the ecology PhD”
[7] “Yet another Science paper described at #esa2012 resilience session: warning signals in yeast dynamics http://t.co/QRGwSH2x”
[8] “Michael Pace shows early warning in zooplankton levels in lake ecosystem experiment. #esa2012 Why is cv=1 a special value?”
[9] “David Seekell speaking on conditional #heteroskedacity as an early warning signal of sudden shifts #esa2012”
[10] “See Will’s recent paper for correct link to Doak et al http://t.co/fVclkNM9 #esa2012 (link pdfs from homepages & help a tweep out!)”
[11] “Notes that portfolio effect is inevitable, cites Dan Doak paper: http://t.co/3RDcyI8L need different null #esa2012”
[12] “96% of returning CA salmon come from hatcheries #esa2012”
[13] “Portfolio effects in salmon stocks: do fish manage risk better than bankers? Will Satterthwaite up now #esa2012”
[14] “Alex Perkins examines impact of poor mixing in disease dynamics, draws analogy to #esa2012 mixers. Transmission is a very local process!”
[15] “Learn how 2 get data & full text open access literature/metadata through R/#rstats w/ @ropensci 11:30a F150 http://t.co/zkifSTuU #esa2012”
[16] “de Roos just showed how total biomass can increase with increased mortality. #Wow. #esa2012”
[17] “de Roos: asymmetry/differences in age/size responses (e.g. biomass production) leads to alternate stable states #esa2012”
[18] “Andrre de Roos shows how ontogenetic assymetry (size structure) breaks basic pop dyn predictions #esa2012”
[19] “Check out @luketkelly ’s talk at #esa2012 Friday (10:20a?) #notetoself”
[20] “#esa2012 Come to the @nceas & @sesync mixer, 6:30 PM-8:00 PM, in conference center rm F150”
[21] “In case you missed the talk or the links, see @jebyrnes slides online http://t.co/nCcrvAYE #esa2012”
[22] “.@jebyrnes mentions Nielsen’s book on online #openscience. Also see his TED talk http://t.co/UkCYR7xV #esa2012”
[23] “.@jebyrnes reminds us that we live in the 21st century. "Taking the science conversation online" #esa2012”
[24] “Ruckelshaus @NatCapProject echoes Kareiva’s theme of collaborating with groups you don’t agree with: "I’m with stupid" #esa2012”
[25] “Karevia: Best levers for change are private sector, not public. Don’t testify before congress, but before Walmart & DOW #esa2012”
[26] “Karieva cites G. West on cities and creativity, possibly this paper: http://t.co/WBGJtLTl #esa2012”
[27] “Karevia channels Sarah Palin: how’s that "hope-y change-y thing going?" #esa2012 need more than talking to congress about ecology”
[28] “Olsen: problem isn’t information, it’s informatics (making sense of information) #esa2012”
[29] “Olsen: cites Karevia, biggest problem is scientists cease to be good listeners. #esa2012 Buisnesses hire improv actors to learn this”
[30] “Randy Olsen: scientists evolved from humans. Split was pretty recent. Still have vestiges of story-telling. Don’t lose them! #esa2012”
[31] “Can find the room by the line going deep into the hallway RT @carlystrasser: .@se_hampton is TEARING IT UP. join the fun A106 #ESA2012” [32] “to be accessible to reviewers RT @rOpenSci: Ecological Monographs now requires data to be deposited prior to publication #esa2012”
[33] “Ben Bolker on the need for off-the-shelf methods: not enough staticians for every ecologist to have a stats buddy 4 custom methods #esa2012”
[34] “#esa2012 stats discussion: "all modeling is exploratory. If you think AIC confirms your hypothesis, you’re wrong"”
[35] “Yeakel uses perturbations/return rate to estimate elasticity, more senstive to fun. form, see http://t.co/AX3ds7uR #esa2012 #mustread”
[36] “Yeakel cites http://t.co/UYaDso0d on nonpar bayes, compares to generalized models #esa2012”
[37] “Excellent #esa2012 session understand vs predict in modelling. Explores ecologist’s v of Breiman’s "Two Cultures" http://t.co/TIDL87hZ”
[38] “Kellner: Multispecies modelling has biggest impact when including non-consumptive value http://t.co/lj3NN0Be #esa2012”
[39] “Kellner presents rich model for multispecies management of fisheries, see http://t.co/fdopymKB #esa2012”
[40] “Bobby describing disease modeling: humans are essentially widgets that eventually fail #esa2012”
[41] “Not nearly enough room for the crowd outside Karen Abbott’s talk: what’s so special about alternative stable states? #esa2012”
[42] “#esa2012 @sesync panel live peer-review of audience-submitted ideas. Levin: "It should count: I view social science as subset of ecology"”
[43] “Ray Hilborn "fisheries management is people management" need socio-econ context of ecology. @sesync #esa2012”
[44] “Molly discusses "most important number you’ve never heard of" - Social cost of carbon #esa2012 @sesync ($25/ton is current est)”
[45] “Lopez-Hoffman explores equitable water-rights that conserve ecosystem services, e.g. Colarado river use @sesync #esa2012”
[46] “Levin: cites common-pool resources example from African cattle http://t.co/j35iY3MO #esa2012”
[47] “Simon Levin speaking @sesync panel: polycentric appches to international agreements on enviro resources #esa2012”
[48] “#esa2012 @sesync will focus on "wicked problems" (technical term) using theme-driven, interdisciplinary approaches”
[49] “Excellent #esa2012 session intro 2 @sesync: "if you’re familiar with @nceas, you’re familiar w/ synthesis". Now focus on socio-econ + uncert”
[50] “#esa2012 @biocreativity Speaking of art-science collab, don’t forget Dr. Scheffer’s "Critical Transitions, the Movie" http://t.co/2nM6RxL9”
[51] “Lubchenco discusses NOAA’s effort to create climate services like their drought forecasting software: http://t.co/zSRN3Y2U #esa2012”
[52] “Lubchenco’s Q&A brings with disclaimer about Hatch Act: she is speaking for the agency, not the campaign, and can’t answer some Q’s #esa2012”
[53] “At #esa2011 plenary, Pacala encouraged us to spend some time on policy-rel issues. #esa2012 Lubchenco raises the bar: run for office!”
[54] “For instance, to promote outreach, NOAA workers are encouraged to use social media, see their policy: http://t.co/SqSw6UeK #esa2012”
[55] “Lubchenco trumpets success of Magnuson-Stevens Act of 2007 (http://t.co/bOqdasVT) as the policy which will end overfishing #esa2012”
[56] “Lubchenco discusses examples of ecosystem thinking in policy such as the Ocean Policy Task Force: http://t.co/XeyAkQVc #ESA2012”
[57] “#esa2012 Lubchenco on the challenge of scale across ecological, social + econ contexts: highlights DeFries 2012 http://t.co/NEU3AdiQ”
[58] “Jane Lubchenco, opening plenary of #esa2012, dedicates her talk to economics Nobel Laureate Elinor Ostrom http://t.co/XZLEeUnQ”
[59] “.@mickresearch speaker ready rooms are D134 & F152 7a-7p, by reservation(?) (program pg 7). Looking forward to your talk btw #esa2012”
[60] “#esa2012 is trying to facilitate networking for young scientists by labeling cocktail tables by sub-field during tonight’s reception”
[61] “Off to Portland for #esa2012”

Mentions at conference, (other than retweets)

mentions <- twListToDF(searchTwitter(paste("@", user, sep = ""), 
    n = 300))
with_hashtag <- grep(hashtag, mentions$text, ignore.case = TRUE)
rt <- grep("RT\\s", mentions$text)
me_not_rt <- with_hashtag[!(with_hashtag %in% rt)]

Again, write the output

text2 <- mentions$text[me_not_rt]
text2 <- gsub("(@[A-Za-z0-9_]+)", "<a href=\"http://twitter.com/\\1\">\\1</a>", 
    text2)
text2

[1] “. @cboettig building cost of switching policies into models necessary & provide better outcomes that reduce costs of management #ESA2012”
[2] “. @cboettig most optimal management strategies produce fluctuations that policy makers don’t like - we can model their dislike! #esa2012”
[3] “.@cboettig enthusiastically uniting decision theory and resilience theory for management. Making math exciting! #esa2012”
[4] “Go see @cboettig (Carl Boettiger’s) talk today in E143 at 920 - he gives a great engaging talk #esa2012”
[5] “#FF #ESA2012 @DynamicEcology @labroides @qaecology @cboettig @paulrehrlich @tessafrancis @lizabio @gigi_rose @IngaPLaPuma @sarcozona” [6] “Awesome! MT @cboettig: Fun at #esa2012 where physics student presents experiment testing the mathematical theories presented by ecology PhD”
[7] “.@cboettig says readme.txt is the best way to keep track of dataset versions. Can version control that on github #ESA2012”
[8] “@jasonpriem @_inundata @cboettig Don’t worry- you made an appearance earlier when @totalimpactdev was discussed! #ESA2012”
[9] “@carlystrasser @_inundata @cboettig Sorry to be missing that! #ESA2012”
[10] “We’ve reached the part of the workshop where .@_inundata and @cboettig slamming MATLAB now… #ESA2012”
[11] “@cboettig - let db managers know that you would use a repository + API to share your data with them - move ecology data forward #esa2012”
[12] “.@cboettig: R is a pain to learn. @rOpenSci packages and tools take advantage of your current knowledge base. #ESA2012”
[13] “@ropensci @cboettig: scripting analyses makes science reproducible, repeatable, & creates an analytical workflow #esa2012”
[14] “.@cboettig showing Michener’s Info Entropy Graph: more time passing means less remembered about data #ESA2012”
[15] “@jacquelyngill @cboettig @treebiology @jenedavison would you like to write about #ESA2012 for @Scitable http://t.co/tPzaPL2U”
[16] “Fun #ESA2012 tweetup. Nice meeting you! @cboettig @MiriamGoldste @SarahBisbing @bonebraking @JacquelynGill @msanclem @luketkelly @edyong209”
[17] “Check out @cboettig’s talk at #ESA2012 Friday 9:20 in E143. Unknown unknowns: management strategies under uncertainty & alt stable states”
[18] “@cboettig @luketkelly is at 10:30 on Friday in E144 at #ESA2012. Optimal fire history for conserving biodiversity.”
[19] “#ESA2012 tweetup was a great idea. Nice meeting you guys @cboettig @MiriamGoldste @SarahBisbing @bonebraking @JacquelynGill @msanclem and co”
[20] “Rt @cboettig in case you missed the talk or the links, see @jebyrnes slides online http://t.co/cUiwTe23 #esa2012”
[21] “@cboettig @ropensci for all other journals too I mean. Why not share the data? #esa2012”
[22] “@cboettig @ropensci agreed. Or at least a subset of data for referees and the readership to access #esa2012”
[23] “@cboettig which leads to the suggestion that we need stats mentors and editors in eeb. #esa2012”
[24] “MT @forestdim: Inspired by @cboettig ’s stardom to tweet. Ppl mistaking me 4 @edyong209. Need 2 now ride the wave of false ID #ESA2012”
[25] “Yay for Baysian stats! MT @cboettig: #esa2012 stats discussion: "If you think AIC confirms your hypothesis, you’re wrong"”
[26] “Why code should be made avail: MT @cboettig Ben Bolker: not enuf staticians 4 each ecologst to have a stats buddy 4 custom methods #esa2012”
[27] “Conversation at #esa2012 Hastings lab dinner: pole dancing at the 2016 Olympics? http://t.co/slpeBcNA @Dr_Bik @cboettig”
[28] “@cboettig Thanks for the art-science tip! #ESA2012 #biocreativity "Critical Transitions, The Movie" TONIGHT Portland Blrm 251 8-10p”
[29] “Let’s hear it for statistics! MT @ropensci: Check out @rOpenSci developer talks: @_inundata Tues 840a, F150 & @cboettig 920a, E143. #esa2012”
[30] “Not on rOpenSci… MT @rOpenSci: Check out talks by @rOpenSci developers: @_inundata Tues 8:40 am F150 - @cboettig 9:20 am E143 #esa2012”
[31] “Check out talks by @rOpenSci developers: @_inundata Tues 8:40 am F150 - @cboettig 9:20 am E143 #esa2012”