Turns out, not much. The statistics for people who died falling out of their bed correlate almost perfectly (0.96) with the number of laywers in Peurto Rico. This stat comes from the rather wonderful tylervigen.com which collects many (many, many) such examples.
Some time spent browsing that site gets the point across that correlation does not imply causation. This means that even if two things vary together, you can't assume they are linked. You definitely can't assume that one is causing the other. It's a logical fallacy known as cum hoc ergo propter hoc, or "With this, therefore because of this".
This fallacy is immensely important to psychology research. The traditional scientific method involves testing causality by changing a single, "independent" variable, then measuring what happens to another variable. If the second, "dependent" variable changes too, then there is a causal relationship between them. Changes in the independent variable create changes in the dependent variable.
This works for some psychology experiments. You can measure relatively simple behaviours, modify some small aspect of a test, and then measure again. An example of this might be priming, a psychology favourite in which people are shown something for a brief period, which makes their subsequent behaviour different to having been shown nothing. These effects are often subtle, and small variations in individual behaviour might mask them. To overcome this, you repeat the experiment with many people, increasing your sample size.
Not all psychology studies are as amenable to a simple lab experiment as this, though. What if you only have one person who has a particular condition you want to study? You can't just have them do the same thing over and over again. Worse, what if the kind of outcome you're looking for defies simple measurement? Or it could entirely unethical, or downright impossible, to replicate the conditions you're studying? Then all you're left with is correlation.
Correlation tells you two things vary together, but it doesn't tell you anything about which causes the other. There are a number of different possibilities, that x causes y, that y causes x, or that there's a third thing, z, that affects both x and y together. There's another even more interesting scenario in that x and y might have a transactional relationship. If this true, when x changes, y changes, which in return changes x, and so on in a spiral.
One of the most awkward examples of this for psychology is the way that psychological knowledge itself can change behaviour. If you take knowledge in physics, for example, when you come up with a new theory and publish, whatever phenomenon you're studying just carries on behaving as it did before. In psychology, however, a new theory might well change the very behaviour it covers.
This post has been pretty theory-heavy, but it does have a major practical application to add to your critical-thinking toolbox. You should treat any research which just shows a correlation with extreme caution. It's one of the tabloid press' favourite headline generators, which leads to all sort of crazy claims. Take a look through today's paper. The Mail Online is probably the best hunting ground for this kind of error, but all papers do it. Look for an article which claims that x causes y, pick it apart and see if it's just correlation. If the original study it's based on just suggests x and y vary together, then you've got your logically fallacy. Let me know what you've found by posting in the comments.
This problem can come up in an engineering context, too. If things in the environment change, and so does user behaviour, it can be easy to see a causal link between the two. This can lead you astray because there are many reasons for users to change behaviour, not just that there was an alteration to some aspect of the product people use. One of the best ways to address this is to use A-B testing, so that you run two different versions (e.g. to different website pages) alongside each other and measure the outcomes of both.
By using this approach and the scientific method you can separate out cause and effect, whilst protecting yourself from seeing relationships between factors that simply aren't there.
Some time spent browsing that site gets the point across that correlation does not imply causation. This means that even if two things vary together, you can't assume they are linked. You definitely can't assume that one is causing the other. It's a logical fallacy known as cum hoc ergo propter hoc, or "With this, therefore because of this".
This fallacy is immensely important to psychology research. The traditional scientific method involves testing causality by changing a single, "independent" variable, then measuring what happens to another variable. If the second, "dependent" variable changes too, then there is a causal relationship between them. Changes in the independent variable create changes in the dependent variable.
This works for some psychology experiments. You can measure relatively simple behaviours, modify some small aspect of a test, and then measure again. An example of this might be priming, a psychology favourite in which people are shown something for a brief period, which makes their subsequent behaviour different to having been shown nothing. These effects are often subtle, and small variations in individual behaviour might mask them. To overcome this, you repeat the experiment with many people, increasing your sample size.
Not all psychology studies are as amenable to a simple lab experiment as this, though. What if you only have one person who has a particular condition you want to study? You can't just have them do the same thing over and over again. Worse, what if the kind of outcome you're looking for defies simple measurement? Or it could entirely unethical, or downright impossible, to replicate the conditions you're studying? Then all you're left with is correlation.
Correlation tells you two things vary together, but it doesn't tell you anything about which causes the other. There are a number of different possibilities, that x causes y, that y causes x, or that there's a third thing, z, that affects both x and y together. There's another even more interesting scenario in that x and y might have a transactional relationship. If this true, when x changes, y changes, which in return changes x, and so on in a spiral.
One of the most awkward examples of this for psychology is the way that psychological knowledge itself can change behaviour. If you take knowledge in physics, for example, when you come up with a new theory and publish, whatever phenomenon you're studying just carries on behaving as it did before. In psychology, however, a new theory might well change the very behaviour it covers.
This post has been pretty theory-heavy, but it does have a major practical application to add to your critical-thinking toolbox. You should treat any research which just shows a correlation with extreme caution. It's one of the tabloid press' favourite headline generators, which leads to all sort of crazy claims. Take a look through today's paper. The Mail Online is probably the best hunting ground for this kind of error, but all papers do it. Look for an article which claims that x causes y, pick it apart and see if it's just correlation. If the original study it's based on just suggests x and y vary together, then you've got your logically fallacy. Let me know what you've found by posting in the comments.
This problem can come up in an engineering context, too. If things in the environment change, and so does user behaviour, it can be easy to see a causal link between the two. This can lead you astray because there are many reasons for users to change behaviour, not just that there was an alteration to some aspect of the product people use. One of the best ways to address this is to use A-B testing, so that you run two different versions (e.g. to different website pages) alongside each other and measure the outcomes of both.
By using this approach and the scientific method you can separate out cause and effect, whilst protecting yourself from seeing relationships between factors that simply aren't there.
Comments
Post a Comment