Saturday, September 1, 2012

Unix for Neuroimagers: For Loops and Shell Scripting

With this tutorial we begin to enter into AFNI territory, as the more advanced Unix commands and procedures are necessary for running an AFNI analysis. First, we cover the basics of for loops. For loops in fMRI analysis are used to loop over subjects and execute preprocessing and postprocessing scripts for each subject individually. (By the way, I've heard of preprocessing and I've heard of postprocessing; but where does plain, regular processing take place? Has anyone ever thought of that? Does it ever happen at all? I feel like I'm taking crazy pills here.)

Within the t-shell (and I use the t-shell here because it is the default shell used by AFNI), a for loop has the following syntax:


foreach [variableName] ( [var1] [var2]...[varN])
     
     [Executable steps go here]

end



So, for example, if I wanted to loop over a list of values and have them assigned to variable "x" and then return those values, I could do something like the following:

foreach x (1 2 3 4 5)
    
     echo $x

end

Which would return the following:

1
2
3
4
5

Notice in this example that the for loop executes the line of code "echo $x", and assigns a new value to x for each step of the for loop. You can imagine how useful this can be when extended to processing individual subjects for an analysis, e.g.:

foreach subj ( subj1 subj2 subj3...subjN )

    tcsh afni_proc.py $subj

end


That is all you need to process a list of subjects automatically, and, after a little practice, for loops are easy and efficient to use. The same logic of for loops applies to FSL analyses as well, if you were to apply a feat or gfeat command to a list of feat directories. In the next batch of tutorials, we will begin covering the basics of AFNI, and eventually see how a typical AFNI script incorporates the use of for loops in its analysis.

The second part of the tutorial covers shell scripting. A shell script executes a series of commands within a text file (or, more formally, shell script), allowing the user more flexibility in applying a complex series of commands that would be unwieldy to type out in the command line. Just think of it as dumping everything you would have typed out explicitly into the command line, into a tidy little text file that can be altered as needed and executed repeatedly.

Let us say that we wish to execute the code in the previous example, and update it as more subjects are acquired. All that would be added would be a shebang, in order to specify the shell syntax used in the script:

#!/bin/tcsh

foreach subj (subj1 subj2 subj3)

     tcsh afni_proc.py $subj

end


After this file has been saved, make sure that it has executable permissions. Let's assume that the name of our shell script is AFNI_procScript.sh; in order to give it executable permissions, simply type

chmod +x AFNI_procScript.sh

And it should be able to be run from the command line. In order to run it, type the "./" before the name of the script, in order to signify that it should be executed from the shell, e.g.:

./AFNI_procScript.sh

Alternatively, if you wish to override the shebang, you can type the name of the shell you wish to execute the script. So, for example, if I wanted to use bash syntax, I could type:

bash AFNI_procScript.sh

However, since the for loop is written in t-shell syntax, this would return an error.


Due to popular demand, an optseq tutorial will be uploaded tomorrow. Comparisons between the output of optseq and AFNI's 1d_tool.py is encouraged, as there is no "right" sequence of stimulus presentations you should use; only reasonable ones.


9 comments:

  1. HELLO,
    I WANT WRITE A FOR LOOP (FOR NII IMAGE) THAT MULTIPLY 7 IMAGE (INVARIANT) TO 4 IMAGE (VARIANT) FOR 37 SUBJECT(IN MATLAB OR BASH),IT IS VERY VITAL,IF IT IS POSSIBLE PLEASE GUIDE ME.
    THANK YOU FOR YOUR ATTENTION

    ReplyDelete
  2. Hi Andrew,

    How do you go about executing AFNI commands inside a for loop in say a bash script? I am having difficulty escaping the '[]' parts of the dataset name that are used to select a sub-brick.

    For example:

    for sub in ${sub_list}
    do
    3dbucket -aglueto outdata+tlrc indata+tlrc'[2]'
    done

    ReplyDelete
    Replies
    1. Hey there,

      In Bash, you don't need to escape the brackets; remove the single quotes, and it should work.

      -Andy

      Delete
    2. Thanks for the reply!

      I have the for loop inside a bash script.

      When I remove the single quotes, AFNI says:
      ** ERROR: selector syntax error 5
      can't decipher index codes

      Delete
    3. When you use the way you originally were doing, what error did it give you?

      Delete
  3. This comment has been removed by the author.

    ReplyDelete
  4. If i want to write a for loop for running multiple higher levels at once(unix); would the same concept apply? so -

    foreach higherlevel (higherlevel1 higherlevel2)

    feat higherlevel1.fsf $higherlevel

    end

    ReplyDelete
    Replies
    1. Hi Teodora,

      I believe so; sorry for the late reply!

      -Andy

      Delete