What is this “effects” business?

I’ve got to figure out what people mean when they say “fixed effect” and “random effect”, because I’ve been confused about it for a year and I’ve been hearing it all the time lately.

Bayesian Data Analysis is my starting guide, which includes a footnote on page 391:

The terms ‘fixed’ and ‘random’ come from the non-Bayesian statistical tradition are are somewhat confusing in a Bayesian context where all unknown parameters are treated as ‘random’. The non-Bayesian view considers fixed effects to be fixed unknown quantities, but the standard procedures proposed to estimate these parameters, based on specified repeated-sampling properties, happen to be equivalent to the Bayesian posterior inferences under a noninformative (uniform) prior distribution.

That doesn’t totally resolve my confusion, though, because my doctor-economist colleagues are often asking for the posterior mean of the random effects, or similarly non-non-Bayesian sounding quantities.

I was about to formulate my working definition, and see how long I can stick to it, but then I was volunteered to teach a seminar on this very topic! So instead of doing the work now, I turn to you, wise internet, to tell me how I can understand this thing.


Filed under statistics

6 responses to “What is this “effects” business?

  1. http://www.stata.com/help.cgi?xtmixed is a good guide for what people who use STATA think it means…

  2. Laurence Thayer

    Random and fixed variables are also relevant for Design of experminents- anova. If the variable has only few fixed values that can be set- two conditions, three modes, two processes, then it is fixed. If you can choose a random value like 120 or 115 or 130 degees for high,0,-10, -20 degrees for low than you have a random variable. The nature of the variable can impact some of the analysis calculations

  3. @Laurence: Ooh, I bet this whole notation comes out of experiment design, where you really know the probability distributions you are dealing with. So just like the sample average is a very clearly defined random variable of the population average, maybe there is some easy explanation of the random effect.

  4. Kyle Foreman

    In both Adrian Raftery’s course on multilevel modeling and in Gelmans ARM, random/fixed effects are almost never mentioned. “Varying intercepts” and “varying slopes” normally took the place of random effects. And in the first 50 pages or so of ARM there was even an aside about random effects not being a terribly informative/useful term.

  5. Casey Olives

    One thing that kind of helps me is to think about a random intercept/random slope model

    Y= \beta_0 + \beta_1 X_1 + b_0 + b_1 X_1 + \epsilon

    where the b’s are “random effects” distributed as zero mean normals, and the \beta’s are “fixed”.

    Rewriting it as

    Y= \beta*_0 + \beta*_1 X_1 + \epsilon


    or another way,

    \beta*_0 ~ N(\beta_0, \sigma^2_0)
    \beta*_1 ~ N(\beta_1, \sigma^2_1)

    Now it becomes a little more clear that random effects represent random deviations from some “population mean”, or a fixed effect. I think that this was the original frequentist intuition. A natural estimate of the b’s is then

    E[b | \hat{\beta}, \hat{\sigma}],

    which is exactly what is done in practice.

    Add priors on \beta_0, \beta_1 and the \sigma’s and you have yourself a nice little hierarchical model where nothing is “fixed” or “random” anymore. But the interpretations are essentially the same.

  6. Thanks Casey and Kyle. That’s the sort of approach that makes it quite understandable to me. I’m going to try to bend my mind around this frequentist/experiment design interpretation some more, since it seems like that is an important alternative strain in the FE/RE story.

    The other part that I’m still trying to understand is the way economists apply FE/RE models, which I think is a sort of causal modeling on the sly.