When it comes to understanding our minds, the role of science is new—making it both exciting and, at times, perilous. Can we trust this new science of the mind? Scrutiny of cognitive neuroscience, in particular, has intensified in recent years through the industry of “neurocriticism.” It began as a response to the extraordinary hype given to neuroimaging findings—in both general-audience media and scientific publications. Blogs like Neuroskeptic and Neurocritic, articles in the Guardian, the Globe and Mail, and the New York Times, and entire books like Neuromania, lament extravagant claims made on the basis of imaging the brain. Have we really found the neural basis of “love”, they ask? Can we really say that dogs feel emotions just like we do? But the strongest neuro-critical opinion is not just that findings are misused, but that the very data are flawed: that we can’t find the neural basis of anything with neuroimaging tools like fMRI.

How seriously should we take the neurocritics? Why, exactly, do so many neuroimaging studies fail to support their conclusions? Gary Marcus, a cognitive scientist at NYU, writes of the neurocritics that they “are half right: our early-twenty-first-century world truly is filled with brain porn, with sloppy reductionist thinking and an unseemly lust for neuroscientific explanations. But the right solution is not to abandon neuroscience altogether, it’s to better understand what neuroscience can and cannot tell us, and why.” This is probably right, but it merits understanding what really is wrong, what is right, and how we can fix it.

It’s not even neurons, or—termites inside a log

A take on everything wrong with fMRI was published in Vox last year, and under its list of complaints, it notes that fMRI does not even measure neural activity, but rather “follows oxygenated blood as it flows through the brain.” Let’s be very clear: fMRI is not fast enough to “follow” this flow. Rather, it takes a one-to-two-second average snapshot of the state of oxygenation of the brain—because that’s how long it takes to acquire the measurements. Oxygenated blood flows much faster than this, so we can’t very well follow it along. Ok—but is either a problem, anyway?

No. Blood oxygenation turns out to be a very good proxy for the concerted firing of neurons. It largely reflects the operation of the brain in these kinds of experiments. So in order to relate cognitive abilities to the functioning of the brain, there’s really nothing wrong here. Recording individual neurons is very invasive in humans, and cannot be done except in certain clinical cases, so this is often the best we have.

However, it is true that the spatial resolution of fMRI is much worse than neural recordings. Each measurement with fMRI reflects the aggregate activity of many thousands of neurons. What are the consequences of this? Let’s make an analogy. Imagine we are interested in the activity of termites inside a log, and we can’t see inside this log to identify the termites, but we can get a measure of the holes and dust-piles on the surface of the log. More termite activity means more and bigger holes, so if we are interested in asking how termites respond to say, air humidity, we can ask whether drier air means more holes or less. Of course, under each location on the log, there may be hundreds of termites deep down, and we don’t know what they are individually doing. This means we can miss things: maybe some termites are working very hard but others fight them and stop their work, so we can falsely conclude that none of the termites are working. But we still expect our experiment on air humidity to tell us whether termites tend to work more or less when the air is dry.

Likewise, to the extent that any cognitive phenomena—vision, memory, reasoning—take place at large scales (involve many neurons), we can detect them with fMRI. Researchers have used this method to relate vision to the occipital lobe, and memory to the hippocampus—relations we know are true based on work with neurological patients, who lose vision or memory functions when those areas are damaged. In other words: there are large enough functional areas that we can use fMRI to study them.

We don’t know in advance which mental operations will be detectable at the large scale, but the worst outcome is that we can’t see some of them. That just isn’t a reason to mistrust fMRI. Vox does describe the deeper issues: fMRI lets you “learn what broad areas of the brain are working. But figuring out what, exactly, those brain areas are doing [functionally] is a totally different problem.” Different problem it is, but it’s not a problem that comes from lack spatial resolution: rather, from cognitive resolution.

The cognitive resolution 

It is tempting to imagine an experiment going like this: a subject enters an MRI scanner and the readout shows a quiet, gray brain. Then, some cognitive task begins: the subject is asked to count the number of dots on a screen. All of a sudden, a bright orange blob emerges in the parietal lobe. Aha!

The truth is just the opposite. Signals from the brain look more like tv static—at all times in all regions. The ‘pictures’ are made through careful statistical analyses that separate signal from noise. This point is made by neurocritics (in SciAm, Statnews, Quartz), but the reason it’s a problem warrants exposition. Since fMRI signal reflects the relative oxygenation of the blood, and blood is always oxygenated to some degree, there is always signal—even when a subject lies quietly in an MRI scanner. And it fluctuates with breathing, heartrate, attention, mind-wandering, and scanner noise—all factors unrelated to the experimental task. When the participant begins to count the number of dots on the screen, changes related to this task are a small fraction of those larger fluctuations. To separate them out, we have to use statistical comparison. And what stops us from figuring out what brain areas are doing is what enters into that comparison.

In the termite example, we wanted to know how humidity affects termite activity, so we had to be certain that when we changed humidity, humidity is the only thing that changed—not temperature, not air quality. Same with cognitive variables: does a brain area do memory? Isolate memory. Does a brain area do counting? Isolate counting. This is cognitive resolution—and it’s much harder than issues of statistics or noise.

Take the task of counting. It involves not just what we think is special about it (incrementing numerical quantities) but also much that isn’t special to it: seeing and distinguishing the objects we want to count, retrieving the names of numbers, and articulating those names. Giving subjects these tasks to perform in the scanner requires us to compare them to closely matched tasks—tasks without numerical quantities, but with naming, articulation, and object perception. While a major challenge, cognitive neuroscience has been successful in discovering specialized areas for face recognition, navigation, and reasoning about others’ minds; these have all been products of clever experimental design. (See Nancy Kanwisher’s TED talk for an inspiring review).

 The center for love and other impossibilities

According to the argument above, the reason we can’t find the “love” center of the brain is because love is difficult to isolate from other variables (say, heart rate, positive emotion, familiarity, and so on). But this is not usually the explanation given by neurocritics, who argue against the possibility of such centers in principle.

Vaughan Bell describes the critical flaws of fMRI as this: “All of our experiences and abilities rely on a distributed brain network and nothing relies on a single ‘centre’”. It’s not possible, she argues, to relate any specific cognitive variable to any specific location in the brain, because all brain areas do many things, and each cognitive variable relies on many brain areas. Others advocate giving up on localization entirely: they applaud research which shows how the entire brain responds to different pictures or words without any attempt to distinguish individual areas or relate them to specific cognitive variables. Such views generally suggest that cognitive neuroscience, as described above, is hopeless.

It’s not clear that they are right. Some cognitive variables are linked to specific brain areas: face recognition, spatial navigation, audition, motor control. But not every category that we think of is. If we find that ‘love’ engages many areas, none of them specific to it—we probably have the wrong category. Not the wrong method.

Gary Marcus, quoting Dave Poeppel, argues alone these lines, saying that what we need is “the meticulous dissection of some elementary brain functions, not ambitious but vague notions like brain-based aesthetics, when we still don’t understand how the brain recognizes something as basic as a straight line.” Until we get to the right elementary functions, it will seem like function mapping is impossible. While we can throw out the idea that “love” is one of them, it isn’t worth tossing the rest of cognitive neuroscience along with it.   

Dead salmon and fishy software

So let’s imagine that you’ve found the right cognitive function to target and have designed the perfect experiment to isolate it. No more roadblocks, right? Unfortunately, some of the most recent and toughest criticisms of neuroimaging are about something else entirely: the statistical analyses of the data. For example, a paper last year led to a barrage of news articles suggesting that a software flaw could “invalidate up to fifteen years worth of neuroscientific research”; this was hardly new, as Neuroskeptic noted. Such problems are seen as inherent to fMRI.

Let’s start with an old scandal: that of the dead salmon. Craig Bennett and a team of researchers put a dead fish into the scanner, showed it different pictures, and measured its brain with fMRI. They ran the data through very commonly used analyses, and, lo and behold: they found a spot in the salmon’s brain which was more active for some pictures than others. But the salmon was dead and hadn’t seen any pictures: this result was impossible. The analysis must have been wrong.

The flaw in these analyses was a proliferation of “false positives”: seemingly significant results actually due to chance. Although the salmon’s brain wasn’t functioning, the fMRI scanner was still picking up on data—except this data was pure noise, the “tv static” caused by random magnetic fluctuations in the environment. Normally, random noise should not lead to significant results; tests are designed so that this happens less than 5% of the time. But, if you run enough tests on enough noise, some will by chance turn out “significant”. Avoiding this requires a correction: setting a higher bar for calling something significant. In fMRI data, there are many tests—one at each spatial location—but the correction wasn’t done. This problem is hardly unique to neuroscience: it similarly plagued genome-wide association studies until they, too, recognized the extent of the problem.

This old issue resurfaced recently when it turned out that these corrections weren’t implemented correctly in some common software packages. The conclusion in the news: fMRI is broken.

Take this account, in Quartz: “fMRI scanners rely heavily on software and statistical tests to eliminate background noise […] these software packages and statistical tests have to make a lot of assumptions, and sometimes use shortcuts.” Or this account of the problem: “Researchers […] need to learn how to use the software, but most neuroscientists are not software engineers and so they have to trust that the software works as advertised.” Either the scanners or the software engineers are to blame, it is suggested; the scientists can’t fix their analyses. Not only is this belittling and false, it also makes the problems seem inherently “built-in” to fMRI scanners themselves. That is far from the truth: already the fixes are being made, by the scientists themselves.

So what is wrong with fMRI? Fingers point to the scanners, the statistical software, the inherent impossibility of mapping the mind to the brain, the limitations of spatial scale. But these are not the real culprits. Rather, the problems are the growing pains of a young science ironing out its flaws—which are not inherent to the method, but ones that with a healthy dose of skepticism can be circumvented.