We now turn to a slightly more sophisticated way to run your analyses, that is, through the command line as opposed to the pointing and clicking through the graphical user interface. When I mentioned in the last post that this factors out human error, what I meant was that it eliminates any error due to faulty clicks in the GUI, or fat-finger dialing of any additional parameters. The probability that you would enter everything in by hand the exact same way for every subject is essentially zero. Learning the basics of scripting is essential for any neuroimaging researcher these days, even if it is just to acquire enough familiarity to know what is going on when reading other people's scripts.
Whenever you set up and run an analysis, a file called design.fsf is output into each results directory; this contains everything you specified in the GUI, but in text format. This file can also be generated at any time by using the "Save" option within the FEAT GUI, and conversely, can be loaded using the "Load" option; this will fill in all of the fields as they were when you saved the design.fsf file.
The power of design.fsf comes from its commandline use. Simply type in "feat design.fsf" and it will execute every command inside the design.fsf file, the same as it would if you were to load it into the FEAT GUI and press the Go button.
Honestly, I am surprised that this feature is not showcased more in the FSL documentation; a couple of lines are devoted to its commandline usage in the FEAT basics section of the manual, but really they should emphasize this much more. (There is a good tutorial here about how to swap code within the design.fsf file and execute it with feat.) If you are new to Unix and need to get your feet wet, I suggest going through the following tutorials: One for basic Unix usage, the other for basics of shells scripting. This will be a new language to you, and as with any language, the beginning can be disorienting and overwhelming at times; however, stick with it, and I promise that the fog will begin to clear eventually, and you will discern what exactly it is you need to know when searching the help manuals, and what it is that you can disregard.
For those of you who have enough Unix background to feel comfortable reading scripts, here is something I created for a recent study I analyzed. This is a special case, since there were some runs where the participant was not exposed to a condition; in this case, for that regressor FSL requires you to specify the shape as "All Zeroes", something that we did not cover yet, but something that you should be aware of. The following script will check for whether the timing file is empty or not, and adjust the shape specification accordingly; however, it will also work for studies which have all regressors accounted for and do not have any missing data.
Here is the script, which can also be downloaded here; I apologize that the formatting is a little screwy with the margins:
#!/bin/bash
for run in 01 02 03 04
do
#Loops through all runs and replaces "ChangeMe" with run number
\cp design.fsf tmpDesign.fsf
sed -i -e 's/run01/runChangeMyRun/' tmpDesign.fsf
sed -i -e 's/FSL_01/FSL_ChangeMyRun/' tmpDesign.fsf #Replaces run placeholder with variable "ChangeMyRun"; this will be swapped later with the appropriate run number
iter=1 #Counter for running loops below
while [ $iter -le 11 ]
do
for timingFile in FSL_"$run"_EmotionRegAM.txt FSL_"$run"_EmotionRegAS.txt FSL_"$run"_EmotionRegNM.txt FSL_"$run"_EmotionRegNS.txt FSL_"$run"_EmotionResp1.txt FSL_"$run"_EmotionResp2.txt FSL_"$run"_EmotionResp3.txt FSL_"$run"_EmotionResp4.txt FSL_"$run"_EmotionResp5.txt FSL_"$run"_Instructions.txt FSL_"$run"_Relax.txt
do
if [ -s ../Timing/$timingFile ]
then
echo "File found; timing file is " $timingFile
sed -i -e 's/fmri(shape'$iter') 3/fmri(shape'$iter') 3/' tmpDesign.fsf
else
echo "File not found; timing file is " $timingFile
sed -i -e 's/fmri(shape'$iter') 3/fmri(shape'$iter') 10/' tmpDesign.fsf
fi
iter=$(( $iter + 1))
done
done
\cp tmpDesign.fsf design_$run.fsf #Make a copy for each run
sed -i -e 's/ChangeMyRun/'$run'/' design_$run.fsf #Swap "ChangeMyRun" with run number
\rm *-e #Remove excess schmutz
feat design_$run.fsf #Run feat for each run
done
Note that a template script is generated by copying a script output by FEAT and replacing "run01" with "runChangeMyRun". ChangeMyRun serves as a placeholder for the run number, being updated on each successive iteration of the loop. The script then executes a while loop, checking each timing file for emptiness, and if it is empty, assigning the appropriate shape in the model. Also note that my timing files, in this example, are located in a directory one level above called "Timing"; this may be different for you, so adjust accordingly if you plan on adapting this script for your purposes.
After that, a new design.fsf is generated for each run, and then executed with feat. If you have more than one subject, then it is a matter of adding another loop on top of this one and going through subjects; with higher-level analyses, replacing single runs of data with .gfeat directories; and so on.
For the beginner, I would recommend opening up a generic design.fsf file with a text editor and using search and replace to update it to the next run. After you get a feel for what is going on, spend some time with Unix and monkey around with the "sed" search and replace tool, until you feel comfortable enough to use it on your design.fsf files; then experiment with loops, and test it on a single subject in a copy directory, so that if something blows up (which it inevitably will on your first go-around), you have a backup.
That is it for the basics of FEAT, and, really, all you need to know for carrying out basic analyses in FSL. Interpreting the results is another matter entirely, but once you have the techniques down, a major part of the drudge work is out of the way, and you can spend more time thinking about and looking at your data; which, really, is what we are trained to do in the first place.
More tutorials will be up soon about the basics of Unix, with an eye on assisting the beginning neuroimaging researcher understand what the hell is going on.
Hey Andrew, this is my first time visiting your blog and I find this post quite helpful. I know how to do FEAT analysis via the GUI and using the command line feat design.fsf but I don't know how to script things. Actually I don't know how to understand a bash script. It seems like a foreign language to me, and the syntax looks kinda complex (compared to MATLAB)
ReplyDeleteHow long do you think it would take to write one's own first shell script to automate FEAT analysis? Do you think before writing one's own script, we gotta be able to understand someone else's script first?
Hi thecuriousmind,
ReplyDeleteI have a brief set of tutorials on Unix covering the fundamentals of the syntax and some shell scripting. I also have a sample script on my homepage under the link "Automated FEAT Analysis."
The script was written for a colleague, and I decided to post it so that others could modify it for their purposes. It may be difficult to parse if you are new to Unix, but it may be a good place to start. As always, let me know if you have any questions!
Best,
-Andy
Hello,
ReplyDeleteI'm planning to write some scripts of my own and in the process stumbled across yours. I have two questions:
1. In the beginning of the for loop you copy the design.fsf
(\cp design.fsf tmpDesign.fsf)
and then later you copy this again
(\cp tmpDesign.fsf design_$run.fsf #Make a copy for each run)
Is this really necessary? Can't you just copy it once and then alter that?
Why do you need the tmpDesign.fsf?
2. In the if-condition you have
sed -i -e 's/fmri(shape'$iter') 3/fmri(shape'$iter') 3/' tmpDesign.fsf
Does this actually do anything?
Best,
a confused mind
Hi Anonymous,
DeleteYou could just make one copy and loop over each run; my method is slightly more verbose, but I find it easier in case there are differences between runs (e.g., different numbers of TRs).
The sed command was to change the type of waveform without creating a different file (by using the -i option). Again, you may want to alter it to suit your purposes.
Best,
-Andy
Hi Andy,
ReplyDeleteThanks for such a fun and informative blog! I see that this script deals with empty EVs at the first level of analyses, but what do you do for them at the second level (combining across sessions for a single subject)? I've tried a few things but they're all pretty suboptimal.
P.S. I think the last sed line need a "g" added after the last "/" so that it does a global search and replace.
Hi Rita,
DeleteI haven't made a script to deal with that yet, but I would assume it is a similar process; what have you done so far, and what have you found hasn't worked that well? I'd like to take that into consideration when I make future tutorials on that.
Also, thanks for the heads-up on the sed script; however, I think it should be taken care of by going through the for loop. If you find that it doesn't work as you think it should when applying it to your own data, let me know.
Best,
-Andy
Is setting an EV shape to 10 and having an empty .txt file for that condition the same as not having an EV for that condition? I'm glad I came across this as I've been thinking about how to deal with having different EVs for different subjects. I was thinking about writing an "if" statement where only .txt files present in the subject's directory would be written into the .fsf file, but this solution seems much simpler if it works in a similar way.
ReplyDeleteHey M,
DeleteI'm not sure about that; I haven't tried it. I would assume that if there are no onsets in there, it would be treated as an empty file, but you would have to run it both ways to check for sure.
Best,
-Andy
Hi
ReplyDeletethanks for your helpfull post, I have a question about entering the data in FSL, my data consists of 34 subjects and each subject has 6runs with different number of volumes!!! how can I enter them to FSL for group analysis ICA?
Hello,
DeleteWhat kind of error does it give you when you enter runs with different numbers of volumes? I haven't encountered a situation with different numbers of volumes per run, so I can't give you any advice off the top of my head.
-Andy
acutally there is no error on it but it takes a long time to run!!! for 1 subject about 2days! and its still running now
Delete"sed -i -e 's/ChangeMyRun/'$run'/' design_$run.fsf"
ReplyDeleteWhat is the implication of using both -i and -e arguments? I was under the impression that -i replaces the original text with the new variable, and that -e temporarily replaces which then vanishes when the script ends.
I think I was screwing around for a while, and using both the -i and -e arguments were what worked. It's been a while since I've used it, so I'm sure there's a more elegant coding solution.
Delete-Andy
Hi Andrew Jahn,
ReplyDeletethanks for writing such a great post. Is there any such bash automation of visualizing the functional connectivity matrices in the 3D brain space view?
My work deal a lot with working with these visualizations. Currently, I am using a tool called BrainNet Viewer. But it is very slow and manual.
Your response may help me.