Hurricane Sandy Hits GCP
The GCP technology is designed for relatively short events, usually a few hours. This sharp focus helps identify a particular event with less potential for overlap and confusion with the plethora of others that might have as much effect on the network. When a longer period is considered it becomes more likely that multiple sources of effect will be included. Thus, the GCP cannot usefully assess Long drawn out events like hurricanes and floods. Nevertheless, we do attempt to capture the emotional and physical impact of big natural disasters, usually by identifying a 24 hour period that seems representative.
Hurricane Sandy presented an especially powerful example because it made landfall in New Jersey and because it was so huge (1000 miles across), there was a direct impact on Princeton, NJ, where the GCP server is located. Princeton lost electricity for four full days, during which the server was unavailable. The elegant design of the network protected the data, however, since it is stored on the local computers at each Egg node. Only the Princeton Egg was unable to record data since there was no power. When the electricity came on again, all data from around the world flowed in for archiving.
Lian Siderov proposed that the special situation might be used to learn something about possible experimenter effects. Though we know the S/N ratio is much too small for any conclusive comparisons, I agreed that it made sense to set an event for Sandy's impact on the GCP. I chose the first day of the power blackout as representative, believing that time would be the most potent in terms of surprise and emotional response of an anthropomorphized GCP.
The formal event was set for 6:30 pm local time (22:30 UTC) continuing for 24 hours. The result was Chisquare 86331 on 86400 df for p = 0.565 and Z = -0.164. Although there is a rather consistent positive deviation (the prediction) during the first four hours of the specified period, the overall deviation is very small and slightly negative.
To provide context, we show also a graph of 6 days beginning with landfall and continuing until the storm had weakened to the point it was no longer destructive. The 4-day power outage that shut down the GCP server is indicated. The whole period shows a modest but persistent negative deviation -- opposite to the prediction we make for all formal analyses.
It is important to keep in mind that we have only a tiny statistical effect, so that it is always hard to distinguish signal from noise. This means that every "success" might be largely driven by chance, and every "null" might include a real signal overwhelmed by noise. In the long run, a real effect can be identified only by patiently accumulating replications of similar analyses.