Merge branch 'master' of github.com:mvdoc/hyperface-data-paper

* 'master' of github.com:mvdoc/hyperface-data-paper:
  Prepare repository for public release (#1)

Author: mvdoc

CLAUDE.md

@@ -1,173 +1,98 @@
 # CLAUDE.md
 
-This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+This file provides guidance to Claude Code when working with this repository.
 
 ## Essential Commands
 
 ### Environment Setup
 ```bash
-# Create virtual environment and install package
 uv venv
-source .venv/bin/activate  # On Windows: .venv\Scripts\activate
-uv pip install -e .
-
-# Install development dependencies
-uv pip install -e ".[dev]"
-
-# Install documentation dependencies
-uv pip install -e ".[docs]"
+source .venv/bin/activate
+uv sync
 ```
 
-**Important:** Always activate the virtual environment before running any Python scripts:
+Always activate the virtual environment before running scripts:
 ```bash
 source .venv/bin/activate && python scripts/qa/...
 ```
 
-### Quality Assurance Pipeline
+### QA Pipeline
 
-#### tSNR Analysis
 ```bash
-# Generate tSNR plots and HTML reports (volume space)
-python scripts/qa/qa-plot-tsnr.py                    # All subjects
-python scripts/qa/qa-plot-tsnr.py --subjects sub-001 # Specific subjects
-python scripts/qa/qa-generate-html-reports-tsnr.py   # Generate HTML reports
-
-# Example scripts for reference (from Budapest dataset)
-python scripts/examples/compute-tsnr-volume.py sub-001
-python scripts/examples/compute-tsnr-fsaverage.py sub-001
-python scripts/examples/compute-isc-fsaverage.py
-python scripts/examples/plot-tsnr-fsaverage.py
-python scripts/examples/plot-isc-fsaverage.py
-```
+# tSNR analysis
+python scripts/qa/qa-save-tsnr-volume.py              # Compute tSNR maps
+python scripts/qa/qa-plot-tsnr.py                      # Generate plots
+python scripts/qa/qa-generate-html-reports-tsnr.py    # HTML reports
 
-#### Motion Analysis
-```bash
-# Generate motion QA plots and HTML reports
-python scripts/qa/qa-plot-motion.py                           # All subjects
-python scripts/qa/qa-plot-motion.py --subjects sub-001        # Specific subjects
-python scripts/qa/qa-generate-html-reports-motion.py          # Generate HTML reports
-
-# Outputs:
-# - Motion parameter traces (translation/rotation) per run
-# - Framewise displacement (FD) traces per run
-# - FD violin plots across runs and subjects
-# - Interactive HTML reports with session navigation
-# - Files saved to: data/derivatives/qa/motion/
-```
+# Motion analysis
+python scripts/qa/qa-plot-motion.py                    # Generate plots
+python scripts/qa/qa-generate-html-reports-motion.py  # HTML reports
 
-### Documentation
-```bash
-# Build Jupyter Book documentation
-jupyter-book build docs/
+# ISC analysis
+python scripts/qa/qa-save-isc.py                       # Compute ISC
+python scripts/qa/qa-plot-isc.py                       # Surface plots
 
-# Clean previous builds
-jupyter-book clean docs/
+# Process specific subjects
+python scripts/qa/qa-plot-tsnr.py --subjects sub-sid000005
 ```
 
 ### Code Quality
 ```bash
-# Format code with Black
-black src/ scripts/qa/ notebooks/
-
-# Sort imports
-isort src/ scripts/qa/
-
-# Type checking
-mypy src/
-
-# Linting
-flake8 src/ scripts/qa/
+ruff check src/ scripts/qa/
+ruff format src/ scripts/qa/
 ```
 
 ## Code Architecture
 
-### Package Structure
-- **`src/hyperface/`**: Core analysis package with two main modules:
-  - `utils.py`: fMRI data processing utilities including `compute_tsnr()` and `clean_data()` functions
-  - `viz.py`: Visualization utilities for creating mosaic plots of brain volumes
-
-### Data Organization (BIDS Compliant)
-- **`data/`**: Git submodule containing BIDS-formatted dataset (managed separately with datalad)
-  - `data/derivatives/fmriprep/`: Preprocessed fMRI data from fMRIprep
-  - `data/derivatives/freesurfer/`: FreeSurfer structural processing outputs
-  - **`data/derivatives/qa/`**: All QA metrics and figures generated by this pipeline
-
-### Analysis Pipeline Architecture
-The QA pipeline follows a structured workflow:
-1. **Data cleaning**: Uses fMRIprep confounds (motion, global signal, aCompCor, framewise displacement) + polynomial regressors
-2. **tSNR computation**: Temporal signal-to-noise ratio after confound regression
-3. **Motion analysis**: Extract and visualize motion parameters and framewise displacement from fMRIprep confounds
-4. **Surface mapping**: Projects volume data to fsaverage cortical surface
-5. **Inter-subject correlation**: Leave-one-out analysis for stimulus consistency
-6. **Visualization**: Volume mosaics, cortical surface plots, motion traces, and interactive HTML reports
-
-### Script Organization
-- **`scripts/qa/`**: Quality assurance pipeline scripts that output to `data/derivatives/qa/`
-- **`scripts/stimuli/`**: Stimuli analysis and visualization scripts
-- **`scripts/presentation/`**: Stimulus presentation code
-- **`scripts/examples/`**: Example scripts from Budapest dataset (for reference)
+### Package Structure (`src/hyperface/`)
+- `utils.py` - fMRI processing (`compute_tsnr()`, `clean_data()`)
+- `viz.py` - Visualization (mosaic plots, surface plots)
+- `io.py` - Data loading and BIDS I/O
+- `isc.py` - Inter-subject correlation
+- `qa/` - QA pipeline (config, BIDS utilities, plotting)
+
+### Data Organization (BIDS)
+- `data/derivatives/fmriprep/` - Preprocessed fMRI data
+- `data/derivatives/freesurfer/` - FreeSurfer outputs
+- `data/derivatives/qa/` - QA outputs (generated)
+
+### Scripts
+- `scripts/qa/` - Main QA pipeline scripts
+- `scripts/presentation/` - Legacy stimulus presentation (do not modify)
 
 ### Writing New Scripts
-New scripts should use the centralized configuration system for maintainability:
+
+Use the centralized configuration:
 
 ```python
 from hyperface.qa import create_qa_argument_parser, get_config
 
 def main():
-    parser = create_qa_argument_parser(
-        description="Script description here.",
-        include_subjects=False,  # Set True if processing per-subject
-    )
+    parser = create_qa_argument_parser(description="Script description.")
     args = parser.parse_args()
-
-    # Load configuration (handles --config and --data-dir CLI args)
     config = get_config(config_path=args.config, data_dir=args.data_dir)
 
-    # Use paths from config instead of hardcoding
-    input_path = config.paths.stimuli_labels_dir / "behavioral-ratings.tsv"
-    output_dir = config.paths.stimuli_dir / "figures"
+    # Use config.paths for all paths
+    output_dir = config.paths.qa_base_dir / "output"
 ```
 
-**Available config paths** (see `src/hyperface/qa/config.py`):
-- `config.paths.data_dir` - Root BIDS dataset directory
-- `config.paths.derivatives_dir` - BIDS derivatives directory
-- `config.paths.fmriprep_dir` - fMRIprep output directory
-- `config.paths.qa_base_dir` - Base QA output directory
-- `config.paths.tsnr_dir`, `motion_dir`, `isc_dir`, `stimuli_dir` - QA subdirectories
-- `config.paths.stimuli_labels_dir` - Stimuli labels input directory
+Config paths available: `data_dir`, `derivatives_dir`, `fmriprep_dir`, `qa_base_dir`, `tsnr_dir`, `motion_dir`, `isc_dir`, `stimuli_dir`
 
-**Config file**: `src/hyperface/assets/qa_config.yaml` - Add new paths here as needed
+## Key Details
 
-### Jupyter Book Integration
-- **`docs/`**: Jupyter Book configuration for web documentation
-- **`notebooks/`**: Analysis notebooks referenced in the book's table of contents
-- Notebooks execute with `execute_notebooks: "off"` to avoid re-execution during builds
-
-## Key Implementation Details
-
-### fMRI Data Processing
-The `hyperface.utils.compute_tsnr()` function implements a standardized confound regression approach:
-- Removes 6 motion parameters + derivatives, global signal, framewise displacement
-- Uses 6 aCompCor components for physiological noise
-- Adds polynomial regressors (up to 2nd order) for trend removal
-- Computes tSNR as mean/std after cleaning
-
-### Surface Data Handling
-Scripts distinguish between volume (T1w space) and surface (fsaverage) processing:
-- Volume scripts work with NIfTI files and create mosaic visualizations
-- Surface scripts work with GIFTI files and create cortical surface plots using pycortex
-- Both output standardized BIDS derivative filenames
+### fMRI Processing
+`compute_tsnr()` uses confound regression:
+- 6 motion parameters + derivatives
+- Global signal, framewise displacement
+- 6 aCompCor components
+- Polynomial regressors (2nd order)
 
 ### Path Conventions
-All scripts assume BIDS derivatives structure:
-- Input paths: `data/derivatives/fmriprep/{subject}/func/`
-- Output paths: `data/derivatives/qa/{analysis_type}/`
-- Figure outputs: `data/derivatives/qa/{analysis_type}/figures/`
-
-## Data Dependencies
+- Inputs: `data/derivatives/fmriprep/{subject}/func/`
+- Outputs: `data/derivatives/qa/{analysis_type}/`
+- Figures: `data/derivatives/qa/{analysis_type}/figures/`
 
-The repository expects preprocessed fMRI data from fMRIprep with specific naming conventions:
-- Volume data: `*_space-T1w_desc-preproc_bold.nii.gz`
-- Surface data: `*_space-fsaverage_hemi-{L,R}_bold.func.gii`
+### Expected File Naming (BIDS)
+- Volume: `*_space-T1w_desc-preproc_bold.nii.gz`
+- Surface: `*_space-fsaverage_hemi-{L,R}_bold.func.gii`
 - Confounds: `*_desc-confounds_timeseries.tsv`
-- Brain masks: `*_space-T1w_desc-brain_mask.nii.gz`

README.md

@@ -1,102 +1,90 @@
-# Hyperface fMRI Dataset: Quality Assurance & Analysis
+# Hyperface fMRI Dataset
 
-This repository contains preprocessing scripts, quality assurance analyses, and data visualization tools for the Hyperface functional MRI dataset. The Hyperface dataset consists of fMRI data collected while participants viewed dynamic facial stimuli.
-
-## Repository Structure
-
-```
-hyperface-data-paper/
-├── data/                    # BIDS dataset (git submodule/datalad)
-│   ├── [raw BIDS data]     # Raw functional and structural data
-│   └── derivatives/        # BIDS derivatives
-│       ├── fmriprep/      # fMRIprep preprocessed data
-│       ├── freesurfer/    # FreeSurfer outputs
-│       └── qa/            # Quality assurance metrics & figures
-├── src/hyperface/         # Python analysis package
-├── notebooks/             # Jupyter notebooks for QA analyses
-├── scripts/               # Processing scripts
-├── docs/                  # Jupyter Book source
-└── pyproject.toml         # UV dependency management
-```
+This repository contains analysis code and quality assurance tools for the Hyperface fMRI dataset. The dataset consists of fMRI data collected while participants viewed dynamic face stimuli in a visual memory task.
 
 ## Installation
 
-This project uses [uv](https://github.com/astral-sh/uv) for fast, reliable dependency management.
+This project uses [uv](https://github.com/astral-sh/uv) for dependency management.
 
-### Prerequisites
-- Python 3.8+
-- uv (install with: `pip install uv`)
-
-### Setup
 ```bash
 # Clone the repository
 git clone https://github.com/mvdoc/hyperface-data-paper
 cd hyperface-data-paper
 
-# Create virtual environment and install dependencies
+# Create virtual environment and install
 uv venv
-source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+source .venv/bin/activate
 uv sync
 ```
 
-### Data Setup
-The dataset will be managed as a git submodule with datalad:
+## Data Access
+
+The dataset is managed with [datalad](https://www.datalad.org/) and follows the [BIDS](https://bids.neuroimaging.io/) standard:
+
 ```bash
-# Add data submodule (when ready)
-git submodule add <datalad-dataset-url> data
+# Initialize the data submodule
+git submodule update --init
 cd data && datalad get .
 ```
 
-## Quality Assurance Pipeline
+## Quality Assurance
+
+The repository includes a comprehensive QA pipeline for assessing data quality.
+
+### Motion Analysis
 
-The QA pipeline includes comprehensive analyses:
+```bash
+# Generate motion parameter plots
+python scripts/qa/qa-plot-motion.py
 
-1. **Motion Assessment**: Head motion parameters during scanning
-2. **Temporal SNR**: Signal quality after confound regression  
-3. **Inter-Subject Correlation**: Stimulus-driven response consistency
-4. **Surface Visualization**: Cortical surface mapping with pycortex
-5. **Outlier Detection**: Identification of problematic scans
+# Generate interactive HTML reports
+python scripts/qa/qa-generate-html-reports-motion.py
+```
 
-### Running QA Scripts
+### Temporal SNR Analysis
 
-Scripts are organized by analysis type:
 ```bash
-# Compute tSNR for volume data
-python scripts/quality-assurance/compute-tsnr-volume.py sub-001
+# Compute tSNR maps
+python scripts/qa/qa-save-tsnr-volume.py
 
-# Compute tSNR for surface data  
-python scripts/quality-assurance/compute-tsnr-fsaverage.py sub-001
+# Generate visualizations
+python scripts/qa/qa-plot-tsnr.py
 
-# Compute inter-subject correlation
-python scripts/quality-assurance/compute-isc-fsaverage.py
+# Generate HTML reports
+python scripts/qa/qa-generate-html-reports-tsnr.py
 ```
 
-All outputs are saved to `data/derivatives/qa/` following BIDS conventions.
+### Inter-Subject Correlation
 
-## Documentation Website
+```bash
+# Compute ISC
+python scripts/qa/qa-save-isc.py
 
-This repository generates a Jupyter Book website with QA results:
+# Generate surface visualizations
+python scripts/qa/qa-plot-isc.py
+```
 
-```bash
-# Install docs dependencies
-uv sync --extra docs
+All outputs are saved to `data/derivatives/qa/` following BIDS conventions.
 
-# Build the book
-jupyter-book build docs/
+## Repository Structure
 
-# Serve locally
-jupyter-book build docs/ --builder=html
 ```
+hyperface-data-paper/
+├── data/                    # BIDS dataset (datalad)
+│   └── derivatives/
+│       ├── fmriprep/       # Preprocessed fMRI data
+│       ├── freesurfer/     # FreeSurfer outputs
+│       └── qa/             # QA outputs
+├── src/hyperface/          # Python analysis package
+└── scripts/qa/             # QA pipeline scripts
+```
+
+## Citation
 
-## Key Features
+If you use this dataset, please cite:
 
-- **BIDS Compliance**: Follows neuroimaging data standards
-- **Modern Dependency Management**: Uses uv for fast, reproducible environments  
-- **Comprehensive QA**: Motion, tSNR, ISC, outlier detection
-- **Web Documentation**: Interactive Jupyter Book with all analyses
-- **Surface Visualization**: High-quality cortical surface plots
-- **Automated Pipeline**: Scripts for reproducible QA workflow
+> [Citation information to be added]
 
-## Contributing
+## License
 
-See `TODO.md` for current development tasks and contribution opportunities.
+BSD 2-Clause License. See [LICENSE](LICENSE) for details.