Finding the Right Subjects for Your FMRI Study

When asked, What is the most important part of an experiment?, some will tell you that it lies in careful, considered deliberation about the design of the study, and being able to accurately tease apart alternative explanations of the results; others will say that emphasis should be placed on technical finesse, statistical competence, and strictly adhering to the rules governing good experimental behavior, including correcting for your critical p-value every time you peek at the data - each viewing like another lashing from the scourge of science.

However, what these people fail to mention is the selection of subjects, which, if overlooked or neglected, will render all of the other facets of your experiment moot. Good subjects provide good data; or, at the very least, reliable data, as you will be certain that they performed the task as instructed; that they were alert, awake, and engaged, and that therefore any issues with your results must be attributed to your design, your construct, or technical problems, but that any problems due to the individuals in your experiment must be ruled out.

To honor this observation, I am constantly on the lookout for fresh cerebrums to wheedle and coax to participate in my studies; during my walk to work I observe in a nearby pedestrian a particularly promising yet subtle eminence on the frontal bone, and silently estimate the amount of cubic centimeters that must therefore be located within Brodmann's Area Number Ten; I sidle up to a young girl at the bar, and after a few minutes of small talk and light banter, playfully brush aside a few unruly strands of her hair and place it behind her ear, taking the opportunity to lightly trace the arc of her squamous suture with my finger, feel the faint pulse of her temporal artery, and fantasize about the blood flowing to the auditory association cortex in response to strings of nonsense vowels. "Do you like playing with my hair?" she asks coyly. "Yes," I manage to stammer, roused from my reverie; "It is beautiful - Beautiful!"

There is one qualm I have with selecting good subjects, however. Often they are people I know, or they are referred by reliable friends, so that I have little doubt that they will be able to successfully carry out their charge. Often they are young, college-aged, healthy, right-handed, intelligent, motivated, and desperate for cash; and as I think about the generalizability of my results, I cannot help but conclude that my results are only generalizable to people like this. A great number of people, either not having enough regard to follow the instructions, or not neurotic enough to care about how they do on the task as they would on a test, perform at a suboptimal level and are thereby excluded; else, they are not even recruited in the first place. This becomes more of a concern when moving beyond simple responses to visual and auditory stimuli, and into higher-level tasks such as decision-making, and I begin to question what meaning my results have for the great mass of humanity; but then I simply stir in more laudanum into my coffee, drink deep from the dregs of Lethe, and sink into carefree oblivion.

In any case, once you have found a good subject, odds are that they also know good subjects; and it is prudent to have them contact their friends and acquaintances, in order to rapidly fill up your subject quota. However, when this approach fails me, and I am strapped for participants, I try a viral marketing approach: As each subject is paid about fifty dollars for two hours of scanning time, upon completion of the study and payment of the subject, I request that they convert their money into fifty one-dollar bills, go to some swank location - such as a hockey game, gentleman's club, or monster truck rally - and take a picture of themselves holding the bills spread out like a fan in one hand and a thumbs-up in the other, while underneath the picture in impact font are the words ANDY HOOKED ME UP. This leads to a noticeable spike in requests for participating in my study, although not always from the clientele that I would like.

How to Fake Data and Get Tons of Money: Part 1

In what I hope will become a long-running serial, today we will discuss how you can prevaricate, dissemble, equivocate, and in general become as slippery as an eel slathered with axle grease, yet still maintain your mediocre, ill-deserved, but unblemished reputation, without feeling like a repulsive stain on the undergarments of the universe.

I am, of course, talking about making stuff up.

As the human imagination is one of the most wonderful and powerful aspects of our nature, there is no reason you should not exercise it to the best of your ability; and there is no greater opportunity to use this faculty than when the stakes are dire, the potential losses abysmally, wretchedly low, the potential gains dizzyingly, intoxicatingly high. (To get yourself in the right frame of mind, I recommend Dostoyevsky's novella The Gambler.) And I can think of no greater stakes than in reporting scientific data, when entire grants can turn on just one analysis, one result, one number. Every person, at one time or another, has been tempted to cheat and swindle their way to fortune; and as all are equally disposed to sin, all are equally guilty.

In order to earn your fortune, therefore, and to elicit the admiration, envy, and insensate jealousy of your colleagues, I humbly suggest using none other than the lowly correlation. Taught in every introductory statistics class, a correlation is simply a quantitative description of the association between two variables; it can range between -1 and +1; and the farther away from zero, the stronger the correlation, while the closer to zero, the weaker the correlation. However, the beauty of correlation is that one number - just one! - has the inordinate ability to make the correlation significant or not significant.Take, for example, the correlation between shoe size and IQ. Most would intuit that there is no relationship between the two, and that having a larger shoe size should neither be associated with a higher IQ or a lower IQ. However, if Bozo the Clown is included in your sample - a man with a gigantic shoe size, and who happens to also be a comedic genius - then your correlation could be spuriously driven upward by this one observation.

To illustrate just how easy this is, a recently created web applet provides you with fourteen randomly generated numbers, and allows the user to plot an additional point anywhere on the graph. As you will soon learn, it is simple to place the observation in a reasonable and semi-random location, and get the result that you want:

Non-significant correlation, leading to despair, despond, and death.

Significant correlation, leading to elation, ebullience, and aphrodisia.

The beauty of this approach lies in its simplicity: We are only altering one number, after all, and this hardly approaches the enormity of scientific fraud perpetrated on far grander scales. It is easy, efficient, and fiendishly inconspicuous, and should anyone's suspicions be aroused, that one number can simply be dismissed as a clerical error, fat-finger typing, or simply chalked up to plain carelessness. In any case, it requires a minimum of effort, promises a maximum of return, and allows you to cover your tracks like the vulpine, versatile genius that you are.

And should your conscience, in your most private moments, ever raise objection to your spineless behavior, merely repeat this mantra to smother it: Others have done worse.

Using SPM.mat to Stay on Track

Life, I have observed, is a constant struggle between our civilized taste for the clean, the neat, and the orderly, on the one hand, and the untrammeled powers of disorganization, disorder, and chaos, on the other. We feel compelled to organize our household and our domestic sphere, including the arrangement of books and DVDs in alphabetical order, placing large items such as vacuum cleaners and plungers in sensible locations when we are done with them, and cleaning and putting away the dishes at least once a week. However, this all takes time and effort, which is anathema to our modern tendency to demand everything immediately.

The same is true - especially, painfully true - in analyzing neuroimaging data. Due to the sheer bulk of data collected during the course of a typical study, and the continual and irresponsible reproduction and multiplication of files, numbers, and images for each analysis, dealing with such a formidable and ever-increasing mountain of information can be paralyzing. The other day, for example, I was requested to run an analysis similar to another analysis I had done many months before; but with little idea of how I had done the first analysis in the first place, I was at a complete loss as to where to start. Foreseeing scenarios such as this, I had taken the precaution to place a trail of text files in each directory where I had performed a step or changed a file, in the hopes that it would enslicken my brain and guide me back into the mental grooves of where I had been previously. However, a quick scan of the document made my heart sink like an overkneaded loaf of whole wheat bread, as I realized deciphering my original intentions would baffle the most intrepid cryptologist. Take, for example, the following:


README.txt
---------------
20 July 2011
Input data into cell matrix of dimensions 30x142x73; covariates entered every other row, in order to account for working memory span, self-report measure of average anxiety levels after 7pm, and onset of latest menstrual cycle. Transposed matrix to factor out singular eigenvariates and determinants, then convolved with triple-gamma hemodynamic response function to filter out Nyquist frequency, followed by reverse deconvolution and arrangement of contrast images into pseudo-Pascal's Triangle. I need scissors! 61!


Deepening my confusion was a list of cross-references to handwritten notes I had scribbled and scrawled in the margins of notebooks and journals over the course of months and years, quite valuable back then, quite forgotten now, as leafing through the pages yielded no clue about when it was written (I am terrible at remembering to mark down dates), or what experiment the notes were about. But just as the flame of hope is about to be snuffed out forever, I usually espy a reference to a document located once again on my computer in a Dropbox folder, and I am filled with not so much pride or hope, as gladness at some end descried; which invariably sets me again on a wild goose chase through the Byzantine bowels of our server, which, if not precisely yielding any concrete result, at least makes me feel stressed and harried, and therefore productive.

Imagine my consternation then, during the latest round of reference-chasing, when I came to the point where I could go no further; where there was not even a chemical trace of where to go next, or what, exactly, I was looking for in the first place. My mind reeled; my spine turned to wax; my soul sank faster than the discharge of a fiberless diet. At wit's end, I cast about for a solution to my predicament, as I mentally listed my options. Ask for help? Out of the question; as an eminently and internationally respected neuroscience blogger, to admit ignorance or incompetence in anything would be a public relations disaster. Give up? As fond a subscriber as I am to the notion that discretion is the better part of valor, and as true a believer as any that there is nothing shameful, base, or humiliating about retreating, surrendering, or rearguard actions, this situation hardly seemed to merit my abject capitulation; and deep down I knew that overcoming this obstacle and chronicling my struggle would inspire my children and grandchildren to similar feats of bravery.

And so it was precisely at this moment, at the nadir of my existence, in the slough of despond, that, through either the random firing of two truculent interneurons in my hippocampus or through intervention by the divine hand of Providence, I had a sudden epiphany. The circumstances of my present situation echoed parallels to the gruesome detective stories I used to read as a child straight before bedtime, and I imagined myself standing in the shoes of a fleabitten detective attempting to piece together the origin and denouement of a puzzling murder, as in Gore by the Gallon or Severed Throats; and I therefore reasoned that, as every strangulation, bludgeoning, shooting, stabbing, poisoning, drowning, and asphyxiation leave traces of their author, so too must each analysis bear the fingerprint of its researcher. Straightaway I navigated to the directory of the analysis I was attempting to replicate, loaded the SPM.mat file into memory, displayed its contents, and quickly realized that I had no idea what any of it meant.

Thus, although the output of the SPM.mat file appears to me as hieroglyphs, I have faith that a more experienced user will know what they mean; and it still stands to reason that these files do contain everything that was done to create them, much as the strands of genetic information coursing through our bodies are virtual volumes of the history of the gonads and gametes from whence they came. I encourage the beginning neuroimager to be aware of this, as the designers of these software packages have proved far more prescient than we, and have installed safeguards to prevent us from the ill effects of our own miserable disorganization.

Psychopathy and the Dark Patch of the Brain

Childe Roland to the Dark Patch Came

Recently my attention was grabbed and squeezed by the vitals by a news item linking frontal lobe deficits to psychopathic behavior. While the association between brain abnormalities and anti-social personality disorder is nothing new, this article was noteworthy for throwing around the term "dark patch", where supposedly "evil lurks" in "killers, rapists, and Nickelback fans"; and while some may object that this smacks of sensationalism, unjustified claims, and Popery, I find myself compelled to defend this finding with all of the earnestness and gravity that is equal to such a weighty subject.

Let us begin with the so-called "Dark Patch"; a name which, unfortunately, carries an unsettling connotation with other patches in various locations on our integument. If this is truly the seat of our turpitude and of all that is base, immoral and wicked; if it is the root of all our evil, of all our intemperance, incontinence, gaming, pimping, whoring, murdering, pilfering, bribing, prevaricating, and a thousand other vices inflaming the natural passions beyond their reasonable bounds, and creating yet new ones to decrease our contentment and increase our sorrow; then it is clear that, in order to rid ourselves of the noxious effects of the Dark Patch, it should be identified, targeted, and devitalized and withered, using any of the numerous methods of neural alteration we have perfected in our progressive era, whether surgical, chemical, or aspiratory; or else restored to its proper function through the use of drugs, shunts, sandbox therapy, or a combination of all of the above, as is seen fit.

One possible objection that may be raised against this approach is that perhaps too much emphasis is put on the Dark Patch, and consequently less scrutiny given to our own choices, actions, and the surfeits of our own behavior. An admirable evasion by whore-master man, to lay his goatish disposition to the charge of the Dark Patch! Proponents of this theory will tell me that a person such as Himmler was, by all accounts, a mild-mannered chicken farmer before seizing upon a position allowing free and irresponsible rein to a deep-seated and heinous ideology, resulting in the deaths of millions and untold suffering to scores of millions more; and likewise, I have heard from a reliable but old-fashioned prison psychiatrist that he never ceases to be amazed that individuals with supposedly uncontrollable addictions to murder and mayhem are somehow able to restrain themselves when meeting with him, a man they detest, given an adequate threat of punishment.

While I understand these concerns, antiquated though they may be, I am pleased to learn that we are trending toward a deeper understanding and appreciation of circumstances outside of our control, of which the Dark Patch is only the most recent and terrible declension. I envision a reformed world in which punishment is meted out, for example, by taking into account the size of one's Dark Patch; and that, instead of inflicting greater and undeserved punitive measures against an individual with an abnormally large Dark Patch, instead care is taken to reform and resocialize them through the techniques mentioned above. For it is the height of hypocrisy and ungenerous in the extreme for those with smaller Dark Patches to rail against and excoriate those who merely happen to have bigger Dark Patches; and I have good reason to believe, based on my observations of all individuals being predisposed to licentiousness, cruelty, and cheating, when it is advantageous to them, that we all possess a Dark Patch to some degree, and that those unfortunate souls who happen to get caught are simply victims of the untimely caprice and vicissitudes of their own Dark Patch. However, I do admit to being at a loss to explain the behavior of otherwise perfectly well-adjusted and functioning individuals, who can commit acts of the most wanton stupidity and barbarity, be it stealing from or maiming another, or sapping the foundations of an otherwise happy relationship through cheating and abuse. I am confident, however, that the root causes of these unfortunate incidents will be pinpointed at some location within our system, and with time, perhaps even whittled down to an individual cell, observed to cause a cascade of anti-social impulses culminating in a febrile desire to listen to Nickelback.

My Confession

Comrades,

I just had a terrible thought - I realized that it has been more than one week since I lasted posted, which violates the promise I made early on to all of my readers, that there would be updates at least once a week, barring exceptional circumstances. However, I assure all of you that there is good reason for my neglect, which has left so many of you puzzled, confounded, bemazed, and deeply hurt; indeed, I have had at least one female reader threaten that, unless I produced new material presently, she would throw herself from the parapet á la Tosca, where she would surely dash out her brains on the stones below; and I have received similar threats by the score, involving dramatic and often bizarre acts of suicide, including severing one's aorta with a jewel-encrusted dagger, throwing oneself under a train, and ingesting arsenic. On the whole, not a very comfortable week.

However, as I said, there is good reason for my long absence, which is enumerated below:

One, I am lazy. With the beginning of the new school year, I have taken the easier path of fulfilling my more pressing duties, such as teaching, collecting data for my dissertation, preparing for a series of concerts, and peddling my limited knowledge of mathematics, statistics, and life, by tutoring a select group of pupils. In addition, I have taken up an outrageous and self-serving routine of self-betterment, involving the gradual acquisition of healthy eating and sleeping habits, making all of my sustenance from scratch, almost totally abjuring alcohol, and running at least sixty miles a week. Obviously this cannot long be sustained, and I look forward to when my will is compromised and I become much the way I was before.

Two, beginning around the start of the New Year, I made the rash and impulsive decision to sever my home internet connection, limiting my contact with the outside world to a series of terminals that I must travel to through the cold and the snow in order to send emails, check the news, and watch videos of obese cats attempt to jump into laughably small cardboard boxes. While this may seem a foolish endeavor, over the past month I have noticed a marked change in my mood, physical symptoms, and attention span; I spend more time at the piano bench and more time at my desk poring over textbooks and volumes of poetry; more easily lose myself in deep thought, and, when I desire to do so, lose consciousness faster and sleep deeper; and in general have gradually lost the craving to compulsively check particular websites, leading to less agitation, reduced nervousness, increased libido, and significantly fewer discharges of the necessities of nature.

Last, as I have been teaching a couple of colleagues all that I know about my trade, I have concomitantly discovered that I know far less than I presumed at first. This has led me to reevaluate both what I know and how I teach myself, so that I may more effectively transmit my meager store to the next generation; for in my view nothing acquired easily is worth having, and without the broad foundation to understand why one does what one does, along with all of its possibilities and all of its horrors, one's profession becomes increasingly mundane, monotonous, quotidian, and dull; qualities which begin to seep into and corrode one's soul, until there is nothing left but an automaton, and whatever spark of life and curiosity there once was, lies entombed within that coffin of devitalized flesh.

In short, I feel as though I am starting again from the ground up; and instead of pumping out videos and tutorials without fully understanding what I am doing, I wish to take a step back and think about this some more. All good things to those who wait.

Guidelines for Conducting FMRI Studies: Reliability Issues

Since FMRI data is (mostly) crap - but extremely expensive crap - there is much debate over how experiments should be designed in order to maximize both power and efficiency. My opinion is that most of these issues would be null if we simply precluded doing any experiments which study trivial or useless things. For example, I have in front of me a paper discussing the neural correlates of heterosexual attraction among females, which used a sample of thirty-nine subjects. Assuming that experiment took about an hour and each scanning hour cost about $500, we can guess that this study cost upwards of $20,000. And all this to study a question that has been answered long ago, as common sense and my own observations suggest that females are irresistibly attracted to the soft, slightly pudgy build of the neuroscience blogger.

For those who must conduct such experiments, however, there are guidelines for balancing the tradeoff between efficiency and reliability (or, the probability that an independent study will replicate your results). In a study conducted by Thirion et al (2007), a large sample of 81 subjects was partitioned into different numbers of subgroups: 2 groups of 40 each, 3 groups of 27 each, 4 groups of 20 each, and so on, to test whether there is a noticeable cutoff for reproducible effects below a specific group size. Further parameters were tested, such as group-level variability and and sensitivity of different p-thresholds.

Figure 1 reproduced from Thirion et al (2007). The top two rows in (a) represent activation maps for disjoint groups of subjects from the same sample; (b) is the group-level statistical map. Note the spread in activation profiles between each of the disjoint groups.

The authors found that an optimal number of subjects to balance both reliability and statistical sensitivity (that is, the ability to detect an effect that is actually present) is about N=25-27, with diminishing returns after that. In addition, the authors counsel the use of mixed-effects models, which take into account variance from first-level analyses (i.e., individual subjects), and downweight subjects with high variability. This procedure is similar to the one employed in FSL's FLAME and AFNI's 3dMEMA.

Figure 8 reproduced from Thirion et al (2007). Both Kappa (a measure of reliability) and activated voxels increase significantly up to around 27 subjects, with a plateau shortly after that.

As a side note, besides merely testing for statistical significance (which is virtually guaranteed with a large enough sample), effect sizes should also be calculated to measure the...well, size of your effect. Essentially all an effect size is, is quantifying the magnitude of the difference between your calculated mean and the null hypothesis mean, in terms of standard deviations. The following table will help you qualify how big your effect is when describing the result:

0-0.3: Wee
0.3-0.5: Not so wee
0.5+: Friggin' HUGE


More details, along with a description of why cluster-thresholding is a better methods than whole-brain corrected voxels, can be found in the original paper.

Andy's Brain Blog Brain Food: Mom's Homemade Granola

You can really taste the vitamins

Back in the good old days, when I first started graduate school, I had a hard time coming up with a good snack for those long afternoon stretches between lunch at noon and going home around two-thirty. Fortunately, I recently rediscovered this classic recipe, which is sweet, delicious, nutritious, and packed with enough fiber to keep your hunger at bay for long periods of time.

You may ask, why is it called Mom's Homemade Granola? Maybe because it's my Mom's recipe, and because she makes it at home. If it were called Goathumper Bill's Granola, you would expect it to be made by some crazy guy out on a farm somewhere diddling livestock. Pay attention.


Anyway, the ingredients are:

• 1 cup old-fashioned oats
• 1/4 cup wheat germ (not toasted)
• 1/4 cup sweetened flaked coconut
• 1/4 cup coarsely broken cashews
• 1/4 cup coarsely broken walnuts or pecans
• 3 teaspoons sesame seeds (or sunflower seeds)
• 1/4 cup pure maple syrup
• 2 tablespoons vegetable oil
• 1 tablespoon brown sugar
• 1 tablespoon light molasses
• 1/2 teaspoon ground cinnamon
• 1/4 cup raisins
• 1/4 cup chopped dried apricots

Preheat the oven to 350 degrees, and then mix the first six ingredients in a large bowl. Whisk the maple syrup and next four ingredient in a medium bowl. Add wet ingredients to dry ingredients; stir to coat evenly. Transfer to a large rimmed baking sheet and spread out the mixture in an even layer. Bake for 8 minutes and then stir, bringing the bottom layer to the top. Bake for 8 minutes longer until golden brown, and then mix in the raisins and apricots. Bake until the fruit is heated through and the granola is slightly darker, and cool completely on the sheet.

I tend to put aluminum foil on the baking sheet and then spread the mixture on top of that; that way, it prevents any of the mixture from sticking to the pan, and you can pick up the foil and funnel the entire mixture easily into a container. Lastly, I also like to substitute agave nectar for the syrup; its viscosity is somewhere in between maple syrup and honey, and I think it tastes better after it is baked. In any case, whatever you do, don't tell Goathumper Bill, or his childhood friend, Melvin the Melon Mounter.