2.1.1 Statistical Functions
Many simple statistics can be thought of in terms of properties of the EDF [empirical distribution function]. For example the sample average $\bar{y} = n^{-1} \sigma y_j$ is the mean of the EDF. More generally, the statistic of interest $t$ will be a symmetric function of $y_1,\ldots,y_n$, meaning that $t$ is unaffected by reordering the data. This implies that $t$ depends on the ordered values $y_{(1)} \leq \cdots \leq y_{(n)}$, or equivalently on the EDF $\hat{F}$. Often this can be expressed simply as $t = t(\hat{F})$, where $t(\cdot)$ is a statistical function - essentially just a mathematical expression of the algorithm for computing $t$ from $\hat{F}$. Such a statistical function is of central importance in the nonparametric case because it also defines the parameter of interest $\theta$ through the "algorithm" $\theta = t(F)$. This corresponds to the qualitative idea that $\theta$ is a characteristic of the population described by $F$...

Forwarded from Frank Harrell:

DEADLINES FAST APPROACHING – 8th Annual International R User Conference useR! 2012, Nashville, Tennessee USA

Registration Deadlines:
Early Registration: Passed
Regular Registration: Mar 1- May 12
Late Registration: May 13 – June 4
On-Site Registration: June 12 – June 15

Please note: Nashville is offering several large entertainment events the month of June, and hotels are quickly selling out. It's imperative that you make your hotel accommodations for the conference as soon as possible. For those of you who have submitted abstracts we will be
notifying you this week regarding acceptance as an oral presentation or as a poster.

Students: A limited number of $500 reimbursements for registration and travel expenses are available, based on merit and need. Please apply by sending an application to Tatsuki Koyama at tatsuki.koyama@Vanderbilt.Edu by April 15. Include a brief CV, a copy of your abstract if one was submitted, a statement that demonstrates your need for financial assistance, and a letter of support from your supervisor.

Please join us at the 8th Annual International R User Conference useR! 2012 in Nashville, Tennessee. For more conference details, please visit http://biostat.mc.vanderbilt.edu/wiki/Main/useR-2012

That's a mouthful! I presented this topic to a group of Vandy statisticians a few days ago. My notes (essentially reproduced in this post) are recorded at the Dept. of Biostatistics wiki: HowToBootstrapCorrelatedData. The presentation covers some bootstrap strategies for hierarchically structured (correlated) data, but focuses on the multi-stage bootstrap; an extension of that described by Davison and Hinkley (ISBN 978-0-521-57471-6).

The multi-stage bootstrap mimics the data generating mechanism by resampling in a nested fashion. For example, resample first among factors at the highest level of hierarchy. Then, for each resampled factor, further resample among factors at the next lower level, and so forth. Each level may be resampled with or without replacement. Furthermore, some levels of hierarchy may be ignored completely, if considered to have little or no effect on the data correlation structure. Whether to ignore a level of hierarchy, or to sample with replacement are important bootstrap design considerations.

The resample function below implements a multi-stage bootstrap recursively. That is, levels of hierarchy are traversed by nested calls to resample. The dat argument is a dataframe with factor fields for each level of hierarchy (e.g., hospital, patient, measurement), and a numeric field of measured values. The cluster argument is a character vector that identifies the hierarchy in order from top to bottom (e.g., c('hospital','patient','measurement')). The replace argument is a logical vector that indicates whether sampling should be with replacement at the corresponding level of hierarchy (e.g., c(TRUE,FALSE,FALSE)).

resample <- function(dat, cluster, replace) {

  # exit early for trivial data
  if(nrow(dat) == 1 || all(replace==FALSE))
      return(dat)

  # sample the clustering factor
  cls <- sample(unique(dat[[cluster[1]]]), replace=replace[1])

  # subset on the sampled clustering factors
  sub <- lapply(cls, function(b) subset(dat, dat[[cluster[1]]]==b))

  # sample lower levels of hierarchy (if any)
  if(length(cluster) > 1)
    sub <- lapply(sub, resample, cluster=cluster[-1], replace=replace[-1])

  # join and return samples
  do.call(rbind, sub)

}

The following block of R code simulates a dataset with 5 correlated (rho = 0.4) repeat measurements on each of 10 patients, from each of 5 hospitals. Hence, there are 250 simulated measurements and 50 patients in total. Patients are simulated independently (i.e., the hospital level of hierarchy has no affect on the correlation structure). The functions covimage and datimage generate a levelplot representations of the covariance and data matrices for the simulated data, respectively.

# simulate correlated data
rho <- 0.4
dat <- expand.grid(
  measurement=factor(1:5),
  patient=factor(1:10),
  hospital=factor(1:5))
sig <- rho * tcrossprod(model.matrix(~ 0 + patient:hospital, dat))
diag(sig) <- 1
dat$value <- chol(sig) %*% rnorm(250, 0, 1)

library("lattice")

covimage <- function(x)
   levelplot(as.matrix(x), aspect="fill", scales=list(draw=FALSE),
      xlab="", ylab="", colorkey=FALSE, col.regions=rev(gray.colors(100, end=1.0)),
      par.settings=list(axis.line=list(col=NA,lty=1,lwd=1)))

datimage <- function(x) {
   mat <- as.data.frame(lapply(x, as.numeric))
   levelplot(t(as.matrix(mat)), aspect="fill", scales=list(cex=1.2, y=list(draw=FALSE)),
      ylab="", xlab="", colorkey=FALSE, col.regions=gray.colors(100),
      par.settings=list(axis.line=list(col=NA,lty=1,lwd=1)))
}

datimage(dat)
covimage(sig)

The images below result from calls to datimage(dat) and covimage(dat) respectively.

The next block of R code generates several boostrap distributions for the sample mean, and approximates the 'true' sampling distribution by Monte Carlo. The final series of boxplots (shown below) illustrate that bootstrap design greatly impacts the inferred distribution of the sample mean (and presumably for other sample statistics). Hence, it's important to think carefully about bootstrap design for hierarchically structured data, and ensure that it closely reflects the 'true' data generating mechanism.

# bootstrap ignoring hospital and patient levels
cluster <- c("measurement")
system.time(mF <- replicate(200, mean(resample(dat, cluster, c(F))$val)))
system.time(mT <- replicate(200, mean(resample(dat, cluster, c(T))$val)))
#boxplot(list("F" = mF, "T" = mT))

# bootstrap ignoring hospital level
cluster <- c("patient","measurement")
system.time(mFF <- replicate(200, mean(resample(dat, cluster, c(F,F))$val)))
system.time(mTF <- replicate(200, mean(resample(dat, cluster, c(T,F))$val)))
system.time(mTT <- replicate(200, mean(resample(dat, cluster, c(T,T))$val)))
#boxplot(list("FF" = mFF, "TF" = mTF, "TT" = mTT))

# bootstrap accounting for full hierarchy
cluster <- c("hospital","patient","measurement")
system.time(mFFF <- replicate(200, mean(resample(dat, cluster, c(F,F,F))$val)))
system.time(mTFF <- replicate(200, mean(resample(dat, cluster, c(T,F,F))$val)))
system.time(mTTF <- replicate(200, mean(resample(dat, cluster, c(T,T,F))$val)))
system.time(mTTT <- replicate(200, mean(resample(dat, cluster, c(T,T,T))$val)))
#boxplot(list("FFF" = mFFF, "TFF" = mTFF, "TTF" = mTTF, "TTT" = mTTT))

# Monte Carlo for the true sampling distribution
system.time(mMC <- replicate(200, mean(chol(sig) %*% rnorm(250, 0, 1))))
#boxplot(list("MC" = mMC))

boxplot(list("MC" = mMC,
             "F" = mF, "T" = mT,
             "FF" = mFF, "TF" = mTF, "TT" = mTT,
             "FFF" = mFFF, "TFF" = mTFF, "TTF" = mTTF, "TTT" = mTTT))

The following figure presents boxplots for the distribution of sample means under the above sequence of bootstrap strategies. The "MC" boxplot summarizes the 'true' distribution of the sample mean (estimated using Monte Carlo). The remaining boxplots are labeled according to the bootstrap strategy used. For instance, the "TF" boxplot corresponds to a multi-stage bootstrap of patients with replacement and measurements-within-patients without replacement (this is commonly called the "cluster bootstrap"), but that ignores the hospital factor. This strategy most closely reflects the data generating mechanism. Notice that sampling all levels of hierarchy without replacement (e.g., "FFF") simply permutes the indices of the resampled data, and does not confer any variability on the sample mean.

The early registration deadline for useR! 2012 is tomorrow! Visit the Online Registration Website. The fees for registration increase March 1^st.

Consider the problem where a certain amount of monthly income $E$ is available to either invest in savings (i.e., savings or money market account, CD, mutual funds, stocks, other unwieldy financial instruments⁵), or to make an additional payment towards a home mortgage. The relevant quantities are denoted

• $B_l$, $B_s$ - Loan or Savings Balance
• $P$ - Initial Loan (Principal)
• $R_l$, $R_s$ - Monthly Interest Rates
• $A$ - Minimum Monthly Mortgage Payment
• $N$ - Number of Payments / Investments
• $E$ - Unallocated Earnings
• $S$ - Monthly Savings ($S \leq E$)

Hence, $S$ is the portion of $E$ that is invested, where the remainder of $E$ is used to reduce the mortgage balance. Our problem is to select $S$.

The balance formulae for savings and loans are, respectively,

$\begin{array}{r c l}B_s & = & P_s(1+R_s)^N + S[(1+R_s)^N - 1]/R_s \\ B_l & = & P_l(1+R_l)^N - (A + E - S)[(1+R_l)^N - 1]/R_l\end{array}.$

$\begin{array}{r c l}W & = & P_l - B_l + B_s \\ & = & P_l - P_l(1+R_l)^N + (A + E - S)[(1+R_l)^N - 1]/R_l + S[(1+R_s)^N - 1]/R_s \end{array}$

Elsevier-funded NY Congresswoman Carolyn Maloney Wants to Deny Americans Access to Taxpayer Funded Research

Fraser has this warning:

The failure to make true assertions with a promised reliability can be extreme with the Bayes use of mathematical priors (Stainforth et al., 2007; Heinrich, 2006). The claim of a probability status for a statement that can fail to be approximate confidence is misrepresentation. In other areas of science such false claims would be treated seriously.

The complaint about coverage of Bayesian confidence regions arises often, I think, because there is a very ingrained notion that correct frequentist coverage is a most desirable quality; that frequentist coverage is a literal statement about a natural phenomenon. Fraser goes further to say that confidence regions with incorrect coverage are misrepresentative, perhaps even fraudulent (i.e., a thing to be 'treated seriously')!

Of course, frequentist coverage is almost never a literal statement about a natural phenomenon, because statistical models almost never fully reflect the truth. In the sentiment of G.E.P. Box, all reported confidence levels are wrong, but some are useful.

More importantly, the criticism of approximate frequentist coverage is readily dismissed from a Bayesian perspective. In response, Christian Robert had this final comment:

Bayesian credible intervals are not frequentist confidence intervals and thus do not derive their optimality from providing an exact frequentist coverage.

On a side note, Fraser's statement above seems to neglect that frequentist probability is different from Bayesian probability in the context of confidence regions! I wrote on a related topic a few weeks ago: Bayesian vs. Frequentist Intervals: Which are more natural to scientists? Unfortunately though, my understanding is upset again due to Robert's reference to Jaynes: "there is only one kind of probability". How can this be true? Aren't Bayesians clear that the posterior distribution on a parameter is not to be interpreted in a frequentist way? And aren't frequentists clear that confidence regions not to be interpreted in a subjective way?