The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’ but ‘That’s funny…’ —Isaac Asimov
Science depends on replication. Psychology depends on publication. Bullies mar both.
Creative people walk a fine line between brilliance and insanity. We know this, or at least, we think we do. Anecdotal observation tells us the premise is true even if there is no research to support the observation. Science, however, actually does confirm a link between creativity and mental illness. Maybe. That report was published in 2010, but the whole premise was later brought into question when additional studies failed to replicate the results. Arguments between researchers are ongoing, but trust me, we’re all nuts.
Science relies on research that can be duplicated. A scientific theory is just a nice idea until someone can not only prove their concept to a reasonable degree (100% certainty is often impossible), but someone else has to be able to copy that research and achieve the same results. Studies cannot be considered accurate if there is no corroboration. That’s just the way science works.
Well, that’s the way science is supposed to work.
The field of psychology has long had a PR problem and much of that problem revolves around the difficulty that exists in duplicating their research. It’s one thing when scientists are doing research on inert materials or with factors that are relatively absolute. Psychologists, however, have to do their research with humans, which means they’re working with a base set that, at its very core, cannot be duplicated exactly. Sometimes, they can’t even get close. Two different study groups in two different parts of the world may duplicate research methods exactly and still come up with wildly different results, not because the methodology was flawed, but because of basic differences in the humans studied. This makes finding necessary corroboration difficult, and significant when it happens.
So, let’s take a study done some 20 years ago. This study, which involves chocolate chip cookies and self-control, has been the basis for 83 similar studies and almost 200 additional experiments, all coming to the same general conclusion. So, one would think that this passes the scientific requirements for proven theory. Pretty much everyone in the field of psychology accepted this research as fact.
Then, last week an announcement was made that brings all that research into question. There is a paper set to publish next month that directly challenges all the findings from the past 20 years on this particular subject. Most importantly, for this conversation, it challenges how psychological research is performed and the common use of macro-analysis in assessing that research. What the paper seems to infer is that huge bodies of established psychological research are wrong because of errors in macro-analysis.
How does this affect you and me? Everything your therapist told you may be pure bunk. That doesn’t mean psychotherapy isn’t helpful; don’t go cancelling that appointment just yet. But people who rely on the accuracy of that information, which not only includes psychotherapists but aspects of law enforcement, retail planning, marketing, transportation, and everything else that relies on studies of human behaviour, now have more reason than ever to question the veracity of information being presented as fact.
While psychology is the field currently in the hot seat, you should know that questions concerning the accuracy of published research have been around for a while. A study from 2005 claims that most research findings are false. But then, an article for the American Psychological Association questioned whether there is a reproducibility crisis.
There are a crap ton of reasons these fallacies exists, but what they all ultimately come down to are money (no big surprise) and what Harvard psychologist Dan Gilbert calls shameless bullies. There are millions of dollars in funding on the line, not to mention tenure, based on a researcher’s ability to publish their findings, and the greater number of scientific publications have shown a distinct bias for publishing papers that are positive and declare new findings that, at least on the surface, seem to move science forward. And while the scientific method requires that research be duplicated, there is considerable animosity between original researchers and those who would reproduce their work. Katie Palmer’s article in Wired last week, “Psychology Is In Crisis Over Whether It’s In Crisis,” had so much back and forth on the issue that my  head was nearly spinning by the time I waded through everything. Folks with Ph. D.’s can be just as mean as street hoods, they merely use words that cut sharper than a switchblade.
How does one resolve an issue like this when the very methods that might solve the problems are part of the problem?
Dear science: we need you to get your act together and quickly. We’ve trusted you, put faith in your findings, confident that research that was duplicated was accurate. Now, we are beginning to question whether we know anything at all. Finding out our science is skewed is rather on par with finding out those deities we believed in probably don’t exist. Who and what are we supposed to believe? Who and what are we supposed to trust? Is anything real?
I can handle psychology being in flux for a while. After all, scientists are just as crazy as creatives. Just don’t tell me what we know about gravity is wrong; it’s Monday and I might come unglued.