I don't know, of course, because the evidence at hand is based on my experience. But, I'll leave the reader to consider whether these observations generalize.
Proponents of Bayesian statistical inference argue that Bayesian credible intervals are more intuitive than the frequentist confidence intervals, because the Bayesian inference is a probability statement about a parameter. A frequentist uses the term 'confidence' because their intervals are based on the probability that a random interval includes the value of a parameter.
A non-statistician scientist might initially agree that the Bayesian interval is more natural because it reads as a probability statement. But, while scientists do often think and behave (perhaps subconsciously) in a Bayesian fashion (i.e., update their prior beliefs using evidence from current experiments), their conscious notion of probability is often more aligned with the frequentist (roughly, the relative frequency of an event in repeated experiments).
For each type of interval, the 'parameter' most often refers to a fixed population quantity used to index a family of parametric models. Hence, when the scientist is reminded of this, the conflict is apparent between the Bayesian interval interpretation and the frequentist notion of probability. The scientist may ask: 'Since I have assumed that a parameter has fixed value, how can I claim that its value will lie in an interval with some frequency in repeated experiments?'
Of course, the above is a misinterpretation (as was pointed out in a previous discussion), because the Bayesian idea of probability is different than the frequentist idea. The scientist may later learn that for Bayesians, 'probability' refers to ones subjective belief about the fixed value of a parameter, but that this belief may be modified by new evidence. The fully informed scientist, despite any subconscious Bayesian tendencies, will often reject the Bayesian notion of probability in favor of the more 'objective' frequentist probability.
So, how should a Bayesian argue more convincingly?
I suppose the title of this post might have been "Bayesian vs. Frequentist Probabilities: ..." I would be surprised if there were not a literature related to this question that I have neglected. But, isn't a blog a reasonable forum to express ideas without the need for exhaustive research to ensure someone else hasn't had the same idea?
The question in the title of your post needs an empirical answer, as you say. I tried a quick literature search and came up with a few items; one person who has done a lot of work investigating the intuitive interpretation of frequentist confidence intervals is Geoff Cumming in Melbourne, Australia. But after a cursory look at those articles I could not find empirical evidence about the main misinterpretation of confidence intervals I've encountered, which is this: Researchers tend to think of frequentist confidence intervals as providing distributional information instead of merely end points on an interval. For example, if a 95% confidence interval on a mean (mu in a normal distribution) is reported as extending from 1.0 to 3.0, people will think that there is a higher probability that mu falls in the middle near 2.0 than at 1.0 or 3.0, although the CI per se provides no such information. On the other hand, a Bayesian credible interval does provide (or derives from) distributional information. Thus, in my experience, the Bayesian credible interval is more intuitive than the frequentist confidence interval.
John,
Thanks for the lead on Geoff Cumming. He has an new book, The New Statistics, that looks interesting (not too badly priced either at ~ $42).
I take your point, that there may be even more potential for misinterpretation of the confidence interval than for the credible interval. But, for a credible interval with bounds at 1.0 and 3.0, there is likewise no guarantee of greater density at 2.0 (think about a slightly bimodal posterior density with modes at 1.5 and 2.5). Hence, the researcher is at risk of making the same mistake for either type of interval. It is nice that the Bayesian interval confers distributional information, but the researcher must still be careful not to interpret this in a frequentist way.
But my point is that the Bayesian credible interval comes from an explicit posterior distribution, whereas the frequentist confidence interval does not. Sure, if the user throws away the explicit posterior distribution and uses only the limits of the credible interval, then, by definition, the user has thrown away knowledge of the shape of the distribution. My point is that the frequentist confidence interval doesn't even provide any distributional information in the first place. But researchers interpret the CI as if there were distributional information there.