By Sally Raskoff
In the sciences, we use theory and methods to empirically assess
“reality”. While we can often play with data to explore the relationships
between our concepts(our variables), it is important to frame what we’re doing
with good theory.
An interesting graph has made its rounds through social
media lately. It shows a strong relationship between Internet Explorer market
share and murders in the U.S.
I first encountered this graph on Facebook when a friend
sent it to me so I could use it in my statistics classes. When I post graphs or
other forms of data, I like to include the source so that students can assess
the veracity of the data and whether or not to trust that it is accurate.
In my search for its source, I thought it was first posted
on Twitter (1/21/2013) but then traced it to reddit and imgur (1/18/2013) –
which then reference each other as a source. Gizmodo picked it up on 1/22/2013
and it reached me in April 2013.
The comments on the gizmodo site provide other examples of such
spurious relationships:
telephone poles and rapes, temperature and number of pirates, and other
nonsensical pairs that co-exist but are not directly related.
These graphs and other relationships show clearly how
correlation does not equal causation. Showing an apparent relationship with
statistics may illustrate a correlation, yet such a relationship does not prove
that one variable causes a change in the other or that they are causally
related.
This chart is a great example of how data may show similar
trends – which we may interpret as a relationship – although there is no reason
or logic as to why a relationship might exist between them. The relationships
could be spurious or simply coincidence that they have the same trend line. Or
perhaps this apparent relationship could be due to the relationship of these
two variables to other variables that are unmeasured in this analysis.
In any case, I could not find the source of the data used to
create the graph – thus this “data” could be fabricated and it is most likely
false. I did find a site – geek.com – that did post a graph of “real” data and
found that the trend line is not as similar for these two variables.
They, however, did not include a source of the data in the
post. At the end of their post, they included a link to the Twitter post
mentioned previously and to a blog. This graph came from that blog and the
source of the data – cited clearly and with links – came from Wikipedia and
w3schools.com. Some of the Wikipedia data is attributed to the Bureau of
Justice Statistics, but it is unclear what the source was on the “Crime in the
United States” Wikipedia page. The w3schools page is a web development site
that logged their tally of browsers used. Used how? That isn’t clear.
Is murder rate the same as homicide rate? Is Internet
Explorer market share the same as browser usage? These are just some of the
problems with how these concepts were defined and measured.
OK, then. What do we know for sure here? Is murder rate
closely linked with Internet Explorer usage (or market share) or not?
Is there a good reason to find this out? Will it tell us
anything about those two phenomena?
Generally, this question raises the importance of theory to
guide our research and statistical analysis. Theory provides the reasons why
things in society may be connected and helps us understand why those things may
be connected.
You may notice that published high quality research start
with a review of theory and the literature to assess what previous research has
discovered about the phenomena in question. We use that to guide how we
investigate “reality” in that we might replicate a study to ensure what they
found can be found again or to pursue a new angle.
In any case, there must be some logical reason why we
connect two variables – why might they be related and how might that work? To
do that, we need theory and clear definitions of what we’re looking at (or for)
and valid and reliable data. We might find that something as simple as
population density affects both of the original variables.
If we cannot find the source of the data or details of the
research, we should not trust that the illustrated relationship is real and
accurate.
Have you heard of other relationships that sound odd or
illogical? If so, has research uncovered the reason(s) why they might be
related? Was this relationship established by empirical data and is its source
reliable?