Preprocessing#

In this section I will outline what attempts to preprocess the data were made. I will link to the corresponding files with errors and I will outline my reasoning for these failures!

The (data-set) was previously obtained via data-lad (also repated in the first step of this notebook).

Before attempting to run preprocessing pipelines using docker, one should have already installed docker. I want to mention a few ressources that helped me in setting up docker correctly:

  • Various sites from the docker installation guidances - for example I had issues by not doing the post-installation steps correctly, as it’s clearly outlined here. I used the steps as they are outlined for ubuntu, as thats my OS. For the installation guidance for your OS, check the docker docs

  • the fMRI-prep documentation on how to use it with docker: Here

  • the MRI-QC documentation on how to use it with docker: Here

Import of the raw Data#

Using datalad, the data of the first person is imported below.

#! cd /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/
#!datalad install https://github.com/OpenNeuroDatasets/ds003707.git
#! cd ./ds003707
!datalad get sub-01
#!datalad get sub-*

Preprocessing via fMRI-Prep#

Multiple attempts to preprocess the data on my device failed. In the following they are outlined.

Attempet 1#

!docker run -it --rm -v  /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/ds003707:/data:ro \
-v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives:/out -v  /home/tchaase/licenses/freesurfer/license.txt:/opt/freesurfer/license.txt nipreps/fmriprep:21.0.4  \
/data /out participant --participant_label 01 --error-on-aroma-warnings --ignore t2w --fs-license-file /opt/freesurfer/license.txt

This produced an error related to the resample workflow. The respective error-message (bash output and error-log) is in the folder attempt_1.

Following this attempt, I specified output-spaced. Also the correct argument regarding aroma (--use-armoa) was inserted.

Attempt 2#

!docker run -it --rm -v  /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/ds003707:/data:ro \
    -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives:/out -v  /home/tchaase/licenses/freesurfer/license.txt:/opt/freesurfer/license.txt nipreps/fmriprep:21.0.4  \
    /data /out participant --participant_label 01 --use-aroma  --ignore t2w --output-spaces MNI152NLin6Asym:res-2 T1w --fs-license-file /opt/freesurfer/license.txt

This still produced errors, it was suspected it might be due to memory limitations. The respective error-message is in the folder attempt_2. After deleting contents of the work-dir, the following was attempted.

Attempt 3#

!docker run -it --rm -m 10GB --cpus="3" -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/ds003707:/data:ro \
-v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives:/out -v  /home/tchaase/licenses/freesurfer/license.txt:/opt/freesurfer/license.txt nipreps/fmriprep:21.0.4  \
/data /out participant --participant_label 01 --use-aroma  --ignore t2w --output-spaces MNI152NLin6Asym:res-2 T1w --fs-license-file /opt/freesurfer/license.txt

This lead to early abruption of the process, abrupton of the process. The respective error message is in the folder attempt_3.

This error persists unless the -m argument is removed. As an attempt to solve this issue, the working directory was specified as shown below:

Attempt 4#

!docker run -it --rm -m 10gb --cpus="3" -v  /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/ds003707:/data:ro \
    -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives:/out -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives/workdir:/workdir  \
    -v  /home/tchaase/licenses/freesurfer/license.txt:/opt/freesurfer/license.txt nipreps/fmriprep:21.0.4  \
    /data /out participant --participant_label 01 --use-aroma  --ignore t2w --output-spaces MNI152NLin6Asym:res-2 T1w -w /workdir

The respective error message is in attempt_4.

Attempt 5#

The docker command above was run without the memory related arguments. The respective files are in the folder for attempt_5.

I further attemepted the above arguments with multiple versions of docker and across participants. Using different participants did not result in different errors. Changing to much older fmriprep-versions did result in altered error-messages, but didn’t provide success.

Attempt 6#

This attempt involved using not only the --low-memory function but also only preprocessing anatomical data (--anat-only). Respective documents are in attempt_6.

!docker run -it --rm -m 14gb --cpus="16" -v  /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/ds003707:/data:ro \
    -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives:/out -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives/workdir:/workdir  \
    -v  /home/tchaase/licenses/freesurfer/license.txt:/opt/freesurfer/license.txt nipreps/fmriprep:21.0.4  \
    /data /out participant --participant_label 01 --use-aroma  --low-mem --anat-only --ignore t2w --output-spaces MNI152NLin6Asym:res-2 T1w -w /workdir

This did not yield success. I also attempted the above with only one task and also got stuck mid running, waiting for extended durations did not progress the processing. During this, the CPU wasn’t used much and load on the RAM was minimal.

Following this, I attempted to reuse what the authors used (fmri-prep version 1.5.0)

Attempt 7#

!docker run -it --rm -m 14gb --cpus="16" -v  /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/ds003707:/data:ro     \
    -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives:/out -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives/workdir:/workdir      \ 
    -v  /home/tchaase/licenses/freesurfer/license.txt:/opt/freesurfer/license.txt poldracklab/fmriprep:1.5.10    \
    /data /out participant --participant_label 01 --use-aroma  --low-mem --output-spaces MNI152NLin6Asym:res-2 T1w -w /workdir

This resultes in a git-annex error. The respective error logs are in the folder attempt_7

Quality Controle with MRI-QC#

Using docker, I also tried to compute quality measures for each participant.

More information on the pipeline that is used to compute these measures is available here

To ensure that the pipeline would work, the following command is specified to only run for one participant.

!docker run -it --rm -v  /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/ds003707:/data:ro \
        -v /home/tchaase/Documents/Universitaet/Forschungsmodul/project/data/derivatives:/out nipreps/mriqc:16.0.0 \
        /data /out participant --participant_label 01

This did not work. As errors were similar to fMRI-prep and I documented how I tried to handle those erros in depth I will not describe those steps for MRI-QC.

The Solution#

I have now managed to get the data preprocessed. While the various attempts failed as outlined above, I was able to preprocess most participants using brainlife. Here are some notes on what was requires to get the data preprocessed on brainlife.

  • Multiple pipelines were created for every run, else the pipelines would fail.

  • brainlife created new anatomic files for each of these pipelines. As every new pipeline that is initiated will also use the files, if preprocessed anatomical files already exist, the pipeline might not start with the following warning: `Ambiguous: Multiple input objects match the criteria and can’t submit a task’.

    • I also manually added an anat tag to the unpreprocessed images, in order to be able to specify that I am referring to the not preprocessed data.

  • Downloading a lot of data lead to fails. I therefore only downloaded a few files at a time, else it was very hard to keep track of what data I have and what I still need.

Brainlife therefore was the plattform that enables me to continue with this project, as I did not have access to the computational ressources to do the preprocessing myself. However, I want to mention two things regarding brainlife that I did not like:

  1. Brainlife probably does not expect me to want to analyze the data on my own. When downloading the data, the data is no longer in the BIDS formate. Reformatting the data took a lot of time and movement of files.

  2. Brainlife did not provide group-level outputs for MRI-QC. As a newbie in cognitive neuroscience, interpreting the quality measures without reference of the other subjects is tedious. But one has to mention again, that I would not have access to the data quality metrics if it weren’t for brainlife!