Snowflakes and Random Effects, or, Get Your Lawyering Hands Off My Giant Magnet
It has been said that like snowflakes, no two people are alike. Actually, that’s not really provable, at least from the snowflake side, but the analogy still implies that everyone is a unique and special floating water crystal. We are incomparable, because we are all unique, and we are all special. Yet the fact that we are all unique snowflakes defines each of us as a type: snowflake. And thus, we are all comparable within the set ‘snowflakes’. Are you a big snowflake? Are you a little snowflake? Are you a simple snowflake? Are you still with me?
Well, brains are the same. Each person is in possession of a unique and varied combination of cerebral wrinkles that defines them, at least neurologically, as unique. Learning anything about what that variation means, or even why we should care about brain data at all, rests on the ability to compare brains of different people. In fact most of our assumptions about what the brain ‘does,’ and the findings so far, rely on the ability to say that there is something common between the brains of the world and how they work. Jeffrey Rosen’s recent article in the New York Times Magazine explores what many of these findings may mean for the legal system, and how lie-detection, death-row punishments, and other possible life-or-death situations may swing based on the evidence from brain data. The alluring power of analyzing brains promises to be an important determinant in our Matrix-as-Destiny futures. But while no neuroscientist worth his neurons will debate the responsibility of the brain in determining behavior, it’s unlikely that he would currently be able to provide reliable evidence on which to base the sorts of life-or-death decisions humans endure on a regular basis. Legal cases, after all, are often based on singular precedent and decided on a case-by-case basis. Making comparisons between the people of the world in order to say something about what our brains might have to do with our behaviors is a complex magic of variables and equations and operates under an ever-developing process of scrutiny and experimentation. Using the trends of the extremely nascent field of cognitive neuroscience in order to resolve legality is, to say the least, premature.
So how would you go about quantifying variations in snowflakes? Stretching my analogy to the unreasonable hilt, there is now evidence that the dynamics of falling is actually what determines the uniqueness of individual snowflakes. So too in brains it’s not just the structure but the dynamics of their ‘falling’ that makes them special. We all have different ways of dealing with the same problems, and our brains reflect this. The disparity between individual patterns of brain activity is high even when performing the same boring behavioral task1, but there is a consistency over time that characterizes each individual.2 And we (our brains and our minds, if you want to get needlessly dualist) grew up learning by an infinite variation of behaviors and so there’s really no way to predict how any one brain learned to do what it does. The human brain is perhaps the most complex organ ever to arise; whether this is useful to you depends on your susceptibility to mystical statements, like the fat man in Amélie who read in a magazine that “there are more links in his brain than atoms in the universe.”
(I feel a little guilty about that last reference. A little demystification is in order: yes, there are lots of neurons, and lots of connections between them, but there are about 1012 in your brain and about 109 in your spinal cord, and your spinal cord’s not solving any math problems. And Amélie wasn’t correct by a long shot; there are actually about 1013 synapses in humans and about 1080 atoms in the universe. It’s also a logical impossibility, since each link must surely require at least one atom, and all the atoms in the universe cannot be in one brain. So maybe let’s not play around with the big numbers and the mystical statements.)
Anyway, my point is that neuroimaging, like all the sciences, relies on the statistical analysis of empirical data in order to make sense of these terrific individual variations in brain structure and function. But as with any scientific statistic, it all comes down to power. And by power I mean number. And by number I mean size. The size of any particular data set is what allows a researcher to make significant claims about what he has observed. Experiments which rely on controllable and repeatable paradigms can usually be run enough times for scientists to be able to tell you, “when you do X, Y happens.” Think of Galileo, atop the Torre di Pisa. Every time he drops a ball, it falls to the ground with a constant acceleration, minus the friction from the air. Downstairs Vincenzo doesn’t believe it at first, so Galileo drops another ball, and another, hundreds of balls, until his erstwhile brother is satisfied. You should be satisfied too because now we know that all objects fall to the earth at the same rate, which is a Law (of Gravity). Being able to say something with the certainty of a Law like this usually requires a “Fixed Effects” model, where perfectly repeatable trials are possible.
Neuroimagers must take a different approach. Much of the ‘research’ on lie-detection, free will, and the sort are based on neuroimaging experiments conducted with a functional magnetic resonance imaging (fMRI) scanner. The average experimental subject can endure about 2 hours in an MRI scanner, and the average laboratory budget can support about 15 subjects getting scanned for each study (the going rate is about $800/hr). And we most certainly don’t have any Laws yet to govern our science. Methodologically, little has changed in the behavioral methods of human psychology since the time of William James’ sober conclusion that “we don’t even know the terms between which the elementary laws would obtain if we had them. This is no science, it is only the hope of a science.”3 In the case of fMRI, we’re not trying to describe the individuals in our studies, but instead extrapolating to the general population. For this purpose fixed effects models are inappropriate. Instead, most neuroimaging studies turn to the use of “Random Effects” models in order to say something significant about their data.
To understand this difference better, let’s go back to Pisa, and let’s say that Galileo, instead of wanting to describe the descent of balls of various mass, was instead interested in describing the descent of cats. One by one, Galileo starts dropping cats from 200 ft., recording their many desperate behaviors in careful Italian. Some screech, some squeal, and true enough, some of them land on all fours. Some constraints would have likely governed his method. Presumably he can drop each cat only once. And he certainly can’t throw every cat in Pisa, so he has to choose a representative sample. Sooner or later 100 cats have gone over the rails, and Galileo has lab books full of behavioral observations. At this point, if he wants to make inferences beyond the particular cats that he used, he’ll need a random effects model to describe his results. The idea here is that the behaviors he notices represent a random sample of all the possible behaviors from the world population of cats. Thus, when he collects his data and tries to say something significant about the behavior of falling cats, he is trying to generalize the results he obtained to that world population with Random Effects. And as voters in Florida can tell you, a random sample can often sell the farm.
fMRI in particular is highly reliant on this model of statistical analysis (the cats not so much). Neuroimagers average out the individual differences in the spatial pattern of neural activity in order to find the patterns common to an entire sample of subjects. They make inferences about the general population from just a small subset of that population (usually 12-16 subjects), by assessing the variability in brain activity within that group.4 In this analysis, regions that are significantly active for the individual subject but not for the group simply reflect noise, despite evidence that suggests that the variability itself is an important part of neural coding.5, 6
All of this makes the causal inferences drawn from neuroimaging data relatively weak, and legal arguments using these data probably even more so. While meta-analyses of many fMRI studies reveal some striking consistencies in the data7, we’re not getting the kind of results upon which to base life-or-death decisions. Remember that almost every study using fMRI is making general conjectures about the population. The analyses used by most fMRI studies ignore the individual patterns of activity in favor of what is common to their group of subjects. A few studies have countered this trend by using the relatively comfortable electroencephalogram (EEG)8, 9 method to find reliable responses to very, very simple visual stimuli (rotating circles, B&W boxes). Since these studies were able to characterize their subjects’ electrophysiological response and use a fixed effect model of the data, they were able to predict those subjects’ behaviors with a much greater certainty than the more typical random effects models. This is the sort of analysis you can start telling your lawyer friends about, but I doubt they’d be interested.
Rosen cites the well-known evidence for brain regions that selectively respond to faces and places, first described by Nancy Kanwisher and Russell Epstein, respectively, but even these studies are not without controversy.10 The reproducibility of any brain response is common only for the most basic of brain processes; only human brain activity for sensory stimuli is relatively stable and can be predicted with a strong degree of certainty. This is about as far as it goes. Such results are common for viewing faces and viewing scenes, but there aren’t these localized regions of activity for every type of perception or behavior we experience. Rosen is keen to point this out in his radio interview with Terry Gross: ‘there is no chocolate center in the brain’ (but oohh, if there was, I’m sure it would be delicious). The further behaviors get from basic sensory information – for instance vision vs. propensity for violence - the more difficult they become to localize. So, while you may be able to look at someone’s brain activity and tell that they are looking at a baby or a forest, nothing beyond the basic perceptual details - more complex calculations such as if you liked the face, if you recognize the face, or whether you saw the face before – is currently accessible to this sort of easy ”mapping.” In fact, some believe that higher mental functions like language and recognition may be impossible to localize, owing to not only the above-mentioned variation in functional processing, but also the highly dynamic nature of brain communication.
I know what you’re thinking: ‘But wait! In college we learned Broca’s area is for language!’ This is the localizationist speaking through you; careful analyses11 have shown that there is actually considerable variation in where the functional ‘Broca’s area’ is physically located, and this functional variation likely holds for all higher brain regions. The forbears of this localizationist perspective are both noble and embarrassing; it may be argued that phrenology has just as much influence as Aristotle (hopefully more, actually) on the theory underlying what we think is going on in your skulls. Cognitive neuroscience, as it stands today, has located itself somewhere between the complete generalist view that the brain is a continuous matrix and the ‘new phrenology’12 of localizing function to structure.
The legal implications of localizationism come into focus continuously with reference to a brain area called the amygdala. Rosen raises many examples of lawyers, judges, and other non-scientists relying on damage to the amygdala to explain the presence of rage or maniacal behavior. Granted, these are not completely unfounded claims, and the strongest data sets for this claim come from Fixed Effects analyses that rely on lesions and stimulations of this region…in mice. (And rats, and cats, and monkeys.) But new investigations of the role of the amygdala in humans show that this structure is nobody’s scapegoat, and that it may play a crucial role in such well-mannered activities as memory, visual awareness, and napkin folding (ok, maybe not the last one).
So there’s no catch-all brain region that can tell you who’s been a bad, bad snowflake. Patients with specific lesions and other medical case studies have been instrumental in helping us to understand the relative contributions of different brain regions to behavior (trust me, I could have re-written this whole thing using just this data set), but the information garnered from case studies apply to only one person (the subject him or herself) with a high degree of certainty. Looking for neural signatures for murderous rage (or recognition, or lying, or guilt) within single subjects (or defendants) would be fraught with the problems of replicability and strength of inference. To be able to do so would imply that we have some basic understanding of how the human brain works, and that we’re ready to predict its performance and explore the biological nature of responsibility. I’m not so sure cognitive neuroscience is ready for such liability.
References
3. James W. 1892. Psychology: Briefer Course. New York: Henry Holt & Co.
4. Friston KJ, Holmes AP, Worsley KJ. 1999. How many subjects constitute a study? Neuroimage. 10: 1-5.
SIN (Science in the News)
Too often, the media’s reporting on scientific findings and issues becomes a professional game of Telephone, with each re-telling adding an extra layer of misunderstanding, miscommunication, erroneous information, and dumbed down simplification. Each week, we will take a recent news story and attempt to excavate the original truth behind the hype, correcting the errors of the media’s coverage, commenting upon reoccurring issues with the science-to-public pipeline, and analyzing the motives and machinery behind these distortions.