Glenn Beck Rally and the Experimenter Effect

Of course no single test, or in the case of the GCP data and analysis, no single event can prove or discount a proposed hypothesis. But it came to me that the Glenn Beck Rally might be an opportunity to test the "Experimenter Effect," in a half-joking way. (Note: this is an exploration, not a formal event.) I wouldn't have thought more about the infamous rally Beck set up for the steps of Lincoln Memorial on the anniversary of Martin Luther King's "I have a dream" speech. But Mike Breland brought it up, with a tone of distaste in his email voice. And I responded that I guess we could take a look, if I could find the event timing, gritting my teeth, or, rather, suppressing my gorge. It was all too easy to find the times -- 11 to 1 pm on Saturday the 28th of August.

So, here we are. The prediction, based on the notion of an Experimenter Effect, would naturally be, ta ta ... No effect. I would personally hate to see the GCP data rocketing off into significance with persistent increased variance, so of course any experimenter effect I am conscious of would suppress that tendency. And what do we get?

Well, it is either ambiguous or evidence for a moderately successful suppression. As a person who's basically disgusted by Beck, the most pleasing or satisfying result would be a flat trend, no sign of an effect. But the result isn't exactly that. For the first hour there is a persistent decrease of variance, of sufficient magnitude that it is unlikely at about the 5% level, or would be if our prediction was for reduced rather than increased variance, and if the event were just one hour instead of three. (The observant will recognize these "ifs" as the reason event specifications must be made a priori.) If that trend were to continue for the full 3 hours it would be impressive indeed, and might be interpreted as an effect on the network. But it did not continue, and in the end, the event has a non-significant outcome of about Z = -0.777. The result is neither here nor there, and doesn't amount to much as a test of the Experimenter Effect. It also doesn't amount to much as an indication of interest in Beck on the part of Global Consciousness. That's no surprise to me, because the only interest I can muster is the sort that compels people to look at accident scenes. It's not something I can do for very long.

It is important to keep in mind that we have only a tiny statistical effect, so that it is always hard to distinguish signal from noise. This means that every "success" might be largely driven by chance, and every "null" might include a real signal overwhelmed by noise. In the long run, a real effect can be identified only by patiently accumulating replications of similar analyses.