feat(MedTsLLM): add MedTsLLM model with LUDB/BIDMC/MIT-BIH datasets and ECG/respiratory tasks

[antonbarchukov]

Contributor

• Name: Anton Barchukov
• Email: anton@barchukov.com
• NetID: abarc3

Type of Contribution

Option 4 — Full Pipeline (Dataset + Task + Model, extra-credit eligible)

Further Extensions / Experiment Logs / Ablations

Outside of pyhealth, and yes ik the repo name '-pyhealth'
https://github.com/antonbarchukov/cs598-pyhealth

Paper

Chan, N., Parker, E., Bennett, W., Wu, T., Jia, R. A., Liu, J., Wong, E.
MedTsLLM: Leveraging LLMs for Multimodal Medical Time Series Analysis.
Proceedings of Machine Learning for Healthcare (MLHC) 2024.
arXiv: https://arxiv.org/abs/2408.07773
Original repo: https://github.com/flixpar/med-ts-llm

Summary

Adds the MedTsLLM model — a frozen-LLM time-series encoder that reprograms ECG / respiratory patches into language-model embedding space via cross-attention over learned word-prototype tokens — together with three PhysioNet datasets and four tasks demonstrating the full pipeline end-to-end.

Datasets (pyhealth/datasets/)

• LUDBDataset — 12-lead ECG with P / QRS / T wave annotations (PhysioNet LUDB 1.0.1)
• BIDMCDataset — respiratory + ECG with breath annotations (PhysioNet BIDMC PPG & Respiration)
• MITBIHDataset — arrhythmia beat annotations (PhysioNet MIT-BIH)

Each dataset inherits from BaseDataset, ships a configs/*.yaml schema, supports a preprocess=True cache, and offers an 80/20 paper-split via a seed=0 record-name shuffle.

Tasks (pyhealth/tasks/)

• ECGWaveSegmentation — per-timestep P / QRS / T classification (LUDB)
• ECGBoundaryDetection — QRS boundary detection (MIT-BIH)
• ECGAnomalyDetection — normal / abnormal beat classification (MIT-BIH)
• RespiratoryBoundaryDetection — breath onset detection (BIDMC)

Model (pyhealth/models/medtsllm.py + pyhealth/models/_medtsllm/)

• Frozen backbone via HuggingFace AutoModel — GPT-2, Qwen2.5, Phi-4, LLaMA
• ReprogrammingLayer (multi-head cross-attention) with a LinearProjection ablation
• RevIN normalization, PatchEmbedding, FlattenHead output head
• Ablation knobs: prompt_dataset / prompt_task / prompt_patient / prompt_stats; defaults match the reference implementation's dataset+task+patient (dtp) config

Ablation Study / Examples (examples/)

One example script per task, each with CLI flags for prompt-component ablation and LLM-backbone swap:

• examples/ludb_ecg_segmentation_medtsllm.py
• examples/mitbih_ecg_boundary_medtsllm.py
• examples/mitbih_ecg_anomaly_medtsllm.py
• examples/bidmc_respiratory_boundary_medtsllm.py

Tests (tests/core/)

208 tests across four files. All tests run on synthetic wfdb records generated by tests/core/_synthetic_wfdb.py — no real patient data is committed to the repo. Regenerate the committed fixtures with:

python -m tests.core._synthetic_wfdb

File Guide

Area	Files
Model	pyhealth/models/medtsllm.py, pyhealth/models/_medtsllm/{layers,prompt}.py
Datasets	pyhealth/datasets/{ludb,bidmc,mitbih}.py, pyhealth/datasets/configs/{ludb,bidmc,mitbih}.yaml
Tasks	pyhealth/tasks/{ecg_wave_segmentation,ecg_boundary_detection,ecg_anomaly_detection,respiratory_boundary_detection}.py
Examples	examples/*_medtsllm.py
Tests	tests/core/test_{medtsllm,ludb,bidmc,mitbih}.py, tests/core/_synthetic_wfdb.py
Docs	docs/api/{datasets,tasks,models}.rst + docs/api/{datasets,tasks,models}/pyhealth.*.rst
Test fixtures	test-resources/core/{ludb,bidmc,mitbih}/ (fully synthetic)