In my whole world-wide wandering, I have concluded that nobody really knows what multivoxel pattern analysis (MVPA) is. For example, a couple of days ago I asked the checkout cashier at Target what MVPA was; he looked at me like some loutish knight beriddled by a troll.
I had nearly given up trying to understand what it was, until I stumbled upon this blog dedicated to vivisecting MVPA so that you can peer into the inner workings of this gruesome monstrosity. After having read this blog for the better part of an afternoon, I have concluded that it is able to make your wildest dreams come true and can answer any questions you could possibly have about MVPA, including discussions on the effect of different experiment parameters on classification accuracy, what a searchlight algorithm is, and what MVPA detects, exactly. It also includes, as far as I can tell, the only convincing argument in favor of allowing people to keep midgets as pets.
The author is a far more dedicated instructor than I am, with large swaths of R code accompanying most of the tutorials so that you can get a better feel for what's going on. I give this baby a 10/10.
For any of those reading this who live in the Bloomington area, tomorrow I will be accompanying for a cello recital which includes, among other pieces, the Grieg cello sonata in A Minor, Op. 36. This one is a beast, and by far the largest and most demanding cello piece I have yet accompanied (although the Strauss and Mendelssohn cello sonatas are close contenders); but it's also epic in scope, features both displays of dazzling virtuosity and moments of heartstopping intimacy, and has a finale guaranteed to bring down the house. In short, it has all of the qualities I idolize about classical music; which, I hope, will awaken some powerful longing for the beautiful and the sublime within even the most naive listener.
Time: 7:00pm, Monday, February 25th
Location: Ford Hall (2nd Floor of the Simon Music Center, near the corner of 3rd & Jordan)
====Program====
Mendelssohn: Lied Ohne Worte, Op. 109
Bach: Cello Suite #4 in E-Flat Major, BMV 1010
Grieg: Cello Sonata in A Minor, Op. 36
I. Allegro Agitato - Presto - Prestissimo
II. Andante Molto Tranquilo
III. Allegro Molto e Marcato
Previously I discoursed at length upon the fatalistic nature of FMRI analysis, and how it leads to disorientation, depression, anxiety, and, finally, insanity. Those who are engaged in such a hopeless enterprise are easily identified by the voids that swirl like endless whorls within their eyes. I once knew a man driven out of his wits by just such an interminable cycle; he was found five days later on the sandy deserts of southern Indiana, ragged and filthy and babbling in alien tongues, scraping with pottery shards the boils that stippled his skin as though they had been painted on. Curse SPM, and die.
For those willing to prolong their agony, however, it is useful to know exactly what is contained within the SPM structure. Although not well-documented (if at all) on the SPM website, there are vast riches of information you can assay within the depths of the scripts themselves, which can be found in your spm8/spm5 library. (If they are in your path, you should be able to open them from anywhere; e.g., using a command such as "open spm_getSPM".) I have copied and pasted below the information from three such scripts, as well as a brief description about what they do. Good fortune to you.
Model-based FMRI analysis is so hot right now. It's so hot, it could take a crap, wrap it in tin foil, put hooks on it, and sell it as earrings to the Queen of England.* It seems as though every week, I see another model-based study appear in Journal of Neuroscience, Nature Neuroscience, and Humungo Garbanzo BOLD Responses. Obviously, in order to effect our entry into such an elite club, we should understand some of the basics of what it's all about.
When people ask me what I do, I usually reply "Oh, this and that." When pressed for details, I panic and tell them that I do model-based FMRI analysis. In truth, I sit in a room across from the guy who actually does the modeling work, and then simply apply it to my data; very little of what I do requires more than the mental acumen needed to operate a stapler. However, I do have some foggy notions about how it works, so pay heed, lest you stumble and fall when pressed for details about why you do what you do, and are thereupon laughed at for a fool.
Using a model-based analysis is conceptually very similar to what we do with a basic univariate analysis with the canonical Blood Oxygenation Level Dependent (BOLD) response; with the canonical BOLD response, we have a model for what we think the signal should look like in response to an event, either instantaneous or over a longer period of time, often by convolving each event with a mathematically constructed gamma function called the hemodynamic response function (HRF). We then use this to construct an ideal model of what we think the signal at each voxel should look like, and then increase or decrease the height of each HRF in order to optimize the fit of our ideal model to the actual signal observed in each voxel.
HRF convolved with a punctate response. This approximate shape can also be plotted by calling upon an SPM.mat file using a canonical HRF by loading the SPM.mat file into memory and typing "plot(SPM.xBF.bf)"
Model-based analyses add another layer to this by providing an estimate of how much the height of this HRF can fluctuate (or "modulate") in response to additional continuous (or "parametric") data for each trial, such as reaction time. The model can provide estimates for how much the BOLD signal should vary on a trial-by-trial basis, which are then inserted into the general linear model (GLM) as parametric modulators; the BOLD response can then correlate either positively or negatively with the parametric modulator, signaling whether more or less of that modulator leads to increased or decreased height in the BOLD response.
To illustrate this, a recent paper by Ide et al (2013) applied a Bayesian model to a simple stop-go task, in which participants either had Go trials in which participants made a response, or Stop trials in which participants had to inhibit their response.The stop signal appeared only on a fraction of the trials, but after a variable delay, which made it difficult to predict when the stop signal would occur. The researchers used a Bayesian model in order to update the estimated prior about the occurrence of the stop signal, as well as the probability of committing an error. Think of the model as representing what an ideal subject would do, and try to place yourself in his shoes; after a long string of Go trials, you begin to suspect more and more that the next trial will have a Stop signal. When you are highly certain that a Stop signal will occur, but it doesn't, according to the model that should lead to greater activity, as captured by the parametric modulator generated by the model on that trial. This is applied to each subject and then observed where it is a good fit relative to the observed timecourse at each voxel.
Model-based regressors applied to FMRI data (Ide et al, Figure 3). The magenta region in (A) shows the contrast of parametric modulators for the probability of a stop trial, P(stop), on Go trials as opposed to Stop trials. In graph C, note the close correspondence of the model predictions to observed FMRI activity in response to each combination of trials.
In addition to neuroimaging data, it is also useful to compare model predictions to behavioral data. For reaction time, to use one example, RT should go up as the expectancy for a stop signal also increases, as a subject with a higher subjective probability for a stop signal will take more time in order to avoid committing an error. The overlay of model predictions and behavioral data collected from subjects provides a useful validation check of the model predictions:
A) Relationship of RT to the probability of a stop trial, P(stop). As P(stop) increases, so does RT, presumably in order to prevent errors of commission on these trials. B) Relationship of P(stop) to error rates on stop trials. Looking at the left side of the graph, if there is a subjectively low probability of receiving a stop trial, the actual occurrence of a stop trial will catch the subject relatively unprepared, leading to an increased error rate on those trials. Taken from Figure 2 of Ide et al, 2013.
Note, however, that this is a Bayesian model as applied to the mind; it's an estimation of what the experimenters think the subject is thinking during the task, given the trial history and what happens on the current trial. In this study, the methods for testing the significance and size of the parameter estimates are still done using null hypothesis significance testing methods.
Those of you who know me, know that I like to stay organized. The pencils on my desk are arranged in ascending order, neatly as organ pipes; the shoes in the foyer of my apartment are placed according to when they were last used, so that I never run in the same pair on consecutive days; the music scores on my bookshelf are stacked so that the first one I take off the top is the one I love best - which, incidentally, always happens to be Liszt's Hungarian Rhapsodies.
However, the game is different when attempting to organize and stay on top of your neuroimaging analyses, as each experiment usually requires dozens, many of them unforeseen but nevertheless pursued, as if by sheer compulsion, until either the waste of your body or until their final endarkenment. At the times that names are given we think them apt; but return weeks, months later, and find to your horror that you have little idea what you did. This collective misery is shared, I think, by many.
AFNI tries to mitigate this by providing a history of each dataset, which traces, step by step, each command that led to the birth of the current dataset. This is useful for orienting yourself; but if you wish to go a step further and append your own notes to the dataset, you can do so readily with the command 3dNotes.
This is a simple program, but a useful one. The options are -a, to add a note; -h, to add a line to the history; -HH, to replace the entire history; and -d, to delete a note. This can also be done through the Plugins of the AFNI interface, all of which is shown in the video below.
Power is like the sun: Everybody wants it, everybody finds in it a pleasant burning sensation, and yet nobody really knows what it is or how to get more of it. In my introductory statistics courses, this was the one concept - in addition to a few other small things, like standard error, effect size, sampling distributions, t-tests, and ANOVAs - that I completely failed to comprehend. Back then, I spoke as a child, I understood like a child, I reasoned like a child; but then I grew older, and I put away childish things, and resolved to learn once and for all what power really was.
1. What is Power?
The textbook definition of statistical power is rejecting the null hypothesis when it is, in fact, false; and everyone has a vague sense that, as the number of subjects increases, power increases as well. But why is this so?
To illustrate the concept of power, consider two partially overlapping distributions, shown in blue and red:
The blue distribution is the null distribution, stating that there is no effect, or difference; the red distribution, on the other hand, represents the alternative hypothesis that there is some effect or difference. The red dashed line represents our rejection region, beyond which we would reject the null hypothesis; and we can see that the more density of the alternative distribution that lies outside of this cutoff region, the greater probability we have of randomly drawing a sample that leads to a rejection of the null hypothesis, and therefore the greater our statistical power.
However, the sticking point is this: How do we determine where to place our alternative distribution? Potentially, it could be anywhere. So how do we decide where to put it?
One approach is to make an educated guess; and there is nothing wrong with this approach, given that it is solidly based on theory, and this may be appropriate if you do not have the time or resources to run an adequate pilot sample to do a power calculation. Another approach may be to estimate the mean of the alternative distribution based on the results from other studies; but, assuming that those results were significant, they have a greater probability of being sampled from the upper tail of the alternative distribution, and therefore have a larger probability of being greater than the true mean of the alternative distribution.
A third approach is to estimate the mean of the alternative distribution based on a sample - which is the logic behind doing a pilot study. This is often the best estimate we can make of the alternative distribution, and, given that you have the time and resources to carry out such a pilot study, is the best option for estimating power.
Once the mean of the alternative distribution has been established, the next step is to determine how power can be affected by changing the sample size. Recall that the standard error, or standard deviation of your sampling distribution of means, is inversely related to the square root of the number of subjects in your sample; and, critically, that the standard error is assumed to be the same for both the null distribution and the alternative distribution. Thus, increasing the sample size leads to a reduction in the spread of both distributions, which in turn leads to less overlap between the two distributions and again increases power.
Result of increasing the sample size from 4 to 10. Note that there is now less overlap between the distributions, and that more of the alternative distribution now lies to the right of the cutoff threshold, increasing power.
2. Power applied to FMRI
This becomes an even trickier issue when dealing with neuroimaging data, when gathering a large number of pilot subjects can be prohibitively expensive, and the funding of grants depends on reasonable estimates from a power analysis.
Fortunately, a tool called fmripower allows the researcher to calculate power estimates for a range of potential future subjects, given a small pilot sample. The interface is clean, straightforward, and easy to use, and the results are useful not only for grant purposes, but also for a sanity check of whether your effect will have enough power to warrant going through with a full research study. If achieving power of about 80% requires seventy or eighty subjects, you may want to rethink your experiment, and possibly collect another pilot sample that includes more trials of interest or a more powerful design.
A few caveats about using fmripower:
This tool should not be used for post-hoc power analyses; that is, calculating the power associated with a sample or full dataset that you already collected. This type of analysis is uninformative (since we cannot say with any certainty whether our result came from the null distribution or a specific alternative distribution), and can be misleading (see Hoenig & Heisey, 2001).
fmripower uses a default atlas when calculating power estimates, which parcellates cortical and subcortical regions into dozens of smaller regions of interest (ROIs). While this is useful for visualization and learning purposes, it is not recommended to use every single ROI; unless, of course, you correct for the number of ROIs used by applying a method such as Bonferroni correction (e.g., dividing your Type I error rate by the number of ROIs used).
When selecting an ROI, make sure that it is independent (cf. Kriegeskorte, 2009). This means choosing an ROI based on either anatomical landmarks or atlases, or from an independent contrast (i.e., a contrast that does not share any variance or correlate with your contrast of interest). Basing your ROI on your pilot study's contrast of interest - that is, the same contrast that you will examine in your full study - will bias your power estimate, since any effect leading to significant activation in a small pilot sample will necessarily be very large.
For your final study, do not include your pilot sample, as this can lead to an inflated Type I error rate (Mumford, 2012). A sample should be used for power estimates only; it should not be included in the final analysis.
Once you've experimented around with fmripower and gotten used to its interface, either with SPM or FSL, simply load up your group-level analysis (either FSL's cope.feat directory and corresponding cope.nii.gz file for the contrast of interest, or SPM's .mat file containing the contrasts in that second-level analysis), choose an unbiased ROI, your Type I error rate, and whether you want to estimate power for a specific subject number or across a range of subjects. I find the range much more useful, as it gives you an idea of the sweet spot between the number of subjects and power estimate.
Thanks to Jeanette Mumford, whose obsession with power is an inspiration to us all.
Yes, life is very, very rum. Imagine a young man of my station, lavished with all the blessings of Nature: eyes grey as gunsteel, hair brown as walnut, body and flesh endowed with the doughy, slightly pudgy form that is the unadulterated delight of every girl; soft to the touch and a joy to caress, leaving for some time the shallow imprint from a laid-upon hand or from the pressure of a firm kiss, as well as providing clear evidence of a refined manner of living, far above the tedious drudgery of the lower classes.
But if I asked you to imagine this same person, the cynosure of a thousand young maiden's eyes, the most sensitive of aesthetes, the most perfectly formed ball of rotundity ever formed by design or by accident, was currently bereft of a lover on this inauspicious day, surely that image would be as incongruous as picturing a slice of toast without chocolate hazelnut spread. "Surely," you would say, "Attempting to keep a girl from you would be as fruitless as trying to restrain a Chihuahua from pouncing upon a porkchop." Yet it is true, every word of it; and as I gaze out upon the cheerless world on this dismal day, idly dipping my fingers into a lightly microwaved bowl of Nutella and mindlessly transferring the liquid bliss to my moistened lips, I cannot help but reflect upon the train of unfortunate accidents which have brought me to this juncture.
There is one day that stands out in particular - the day that I had my first real, mature, passionate, full-fledged longing for a girl; which, incidentally, happened during an anatomy course in college. That last detail is of considerable interest when I recall the letter I later sent to her, into which I poured all of my hopes, fears, doubts, aspirations, anxieties, and rawest emotions. I told her that her hair was as perfect as the softest, most velvety branches of telodendria, a brilliant fan of cauda equina radiating from her scalp; that her eyes reminded one of a delicately placed nucleolus in a magic sea of cytoplasm; that the mere thought of her silky epithelium was enough to engage my cremasteric reflex. These opening lines were unquestionably the greatest verse I have ever composed, wholly without precedent and never equaled since; and it is a tragic loss for humanity and future scholars that I destroyed all copies of this letter in a paroxysm of fury. (While I cannot remember the rest of the missive I sent her, I do have the
feeling that it was, although intensely felt, mere doggerel.)
But as deep and genuine as my emotions were, however, I quickly realized that it was not meant to be; as the day after I sent my letter, she began to look at me the same way you would regard a tupperware container filled with nose hairs. I attempted some small consolation by telling myself that she was probably getting her minge rocked by some water polo player anyway; which, in fact, turned out to be the case. And every love interest I have had since then has merely been a slight variation on the same theme - passionate love letter, bitter rejection, minge getting rocked by a water polo player, everything.
