feat(tasks): generalized note section extraction by note type

[Rian354]

PR: Generalized Note Section Extraction by Note Type

Problem

The current _extract_admission_sections() method in BaseMultimodalMIMIC4Task is discharge-note specific. It targets headers like "Chief Complaint", "HPI", and "Past Medical History": sections that are meaningless in radiology notes.

When ClinicalNotesICDLabsMIMIC4 processes radiology notes today, it passes the raw text through w/o section-aware filtering. Radiology reports have a very different structure:

CLINICAL INDICATION:
  65F with shortness of breath.

TECHNIQUE:
  PA and lateral chest radiograph.

FINDINGS:
  Lungs are clear bilaterally...

IMPRESSION:
  No acute cardiopulmonary process.

The "Impression" and "Findings" sections contain the actual diagnostic signal. The rest (technique, demographics) is noise that increases token count w/o clinical value.

Discharge notes also have more valuable admission-time sections than the current target set: Social History, Family History, Allergies, and Review of Systems are all clinically available at admission.

Solution

Introduce a general-purpose extract_note_sections(text, note_type) static method that:

1. Routes to note-type-specific target sets (_DISCHARGE_SECTION_TARGETS vs _RADIOLOGY_SECTION_TARGETS)
2. Shares a single _parse_note_sections() parser (header regex + line accumulator)
3. Falls back to text[:1024] when no matching headers are found

The old _extract_admission_sections(text) becomes a backward-compatible alias calling extract_note_sections(text, note_type="discharge").

Changes

pyhealth/tasks/multimodal_mimic4.py

Symbol	Change
_ADMISSION_SECTION_TARGETS	Renamed to _DISCHARGE_SECTION_TARGETS. Added Social History, Family History, Allergies, Review of Systems.
_RADIOLOGY_SECTION_TARGETS	New. Impression, Findings, Clinical Indication, Indication, Clinical History, History, Comparison, Technique, Conclusion, Summary.
_parse_note_sections(text)	New. Shared parser. Returns {lowered_header: content} dict.
extract_note_sections(text, note_type)	New. Routes to correct target set. Public API.
_extract_admission_sections(text)	Now an alias to extract_note_sections(..., note_type="discharge").

Target Header Rationale

Discharge notes: admission-context sections:

Header	Signal
Chief Complaint	Primary reason for visit
History of Present Illness / HPI	Detailed symptom timeline
Past Medical History (+ variants)	Chronic conditions, prior procedures
Medications on Admission / Home Medications	Baseline medication burden
Social History	Smoking, alcohol, substance use, living situation
Family History	Genetic risk factors
Allergies	Medication safety
Review of Systems	Systematic symptom inventory

Radiology notes: diagnostic output sections:

Header	Signal
Impression	Radiologist's conclusion. Highest information density.
Findings	Detailed image description
Clinical Indication / Indication	Why the study was ordered
Clinical History / History	Background context
Comparison	Prior study referenced
Technique	Modality and protocol
Conclusion / Summary	Used instead of Impression in shorter studies

Usage

from pyhealth.tasks.multimodal_mimic4 import BaseMultimodalMIMIC4Task

# Discharge note
text = BaseMultimodalMIMIC4Task.extract_note_sections(discharge_text, note_type="discharge")

# Radiology report
text = BaseMultimodalMIMIC4Task.extract_note_sections(radiology_text, note_type="radiology")

# Backward-compatible (unchanged):
text = BaseMultimodalMIMIC4Task._extract_admission_sections(discharge_text)

Testing

Run existing unit tests. All 18 should pass unchanged (backward-compatible alias):

python -m pytest tests/core/test_notes_labs_mimic4.py -v

Impact on Downstream Tasks

• NotesLabsMIMIC4: no change (uses discharge only, alias is backward-compatible).
• ClinicalNotesICDLabsMIMIC4: should call extract_note_sections(text, note_type="radiology") for radiology events. Tracked in a follow-up.
• All existing caches remain valid (task UUID is derived from class vars, not method bodies).

Files Changed

File	Lines	Change
pyhealth/tasks/multimodal_mimic4.py	+55 / -11	Implementation