Skip to content

Add InHospitalMortalityTemporalMIMIC4 task for EMDOT temporal evaluation#1025

Open
nskath wants to merge 18 commits intosunlabuiuc:masterfrom
FatihIllinois:feat/temporal-mortality-mimic4
Open

Add InHospitalMortalityTemporalMIMIC4 task for EMDOT temporal evaluation#1025
nskath wants to merge 18 commits intosunlabuiuc:masterfrom
FatihIllinois:feat/temporal-mortality-mimic4

Conversation

@nskath
Copy link
Copy Markdown

@nskath nskath commented Apr 19, 2026

Contributors

Type of Contribution

Standalone Task (Option 3)

Original Paper

Zhou, H.; Chen, Y.; and Lipton, Z. C. 2023. Evaluating Model Performance in Medical Datasets Over Time. CHIL 2023. PMLR 209:498-508.

High-Level Description

This PR adds InHospitalMortalityTemporalMIMIC4, a standalone task that tags each admission sample with its real (de-shifted) calendar year so that callers can partition data chronologically using the EMDOT framework.

The EMDOT paper shows that standard random train/test splits overestimate model performance by ignoring temporal distribution shift in clinical data. This task enables reproducing that finding on MIMIC-IV by:

  • Extracting diagnosis codes, procedure codes, and drug names as bag-of-codes features
  • Generating binary in-hospital mortality labels from hospital_expire_flag
  • De-shifting MIMIC-IV's anonymized dates using anchor_year and anchor_year_group to recover real calendar years
  • Supporting both all-historical and sliding-window training regimes via the admission_year field

The ablation script demonstrates that random splits yield ~0.986 AUROC while temporal splits starting from 2012 yield ~0.937, confirming the paper's core finding. Window size ablation (w=1,2,3,5) shows the recency vs sample-size tradeoff.

File Guide

File Description
pyhealth/tasks/in_hospital_mortality_temporal_mimic4.py Core task implementation
pyhealth/tasks/__init__.py Added InHospitalMortalityTemporalMIMIC4 import
tests/core/test_mimic4_mortality_temporal.py 16 unit tests using synthetic mock patients
examples/mortality_prediction/mortality_mimic4_temporal_emdot.py EMDOT ablation study script
docs/api/tasks/pyhealth.tasks.InHospitalMortalityTemporalMIMIC4.rst API documentation
docs/api/tasks.rst Added entry to Available Tasks toctree

fmatlamaz and others added 18 commits April 19, 2026 13:11
- Replace docstring-only example with full runnable ablation script
  covering all-historical vs sliding-window regimes, window size
  ablation (w=1,2,3,5), and LR/GBDT/MLP model comparison
- Add test suite using synthetic mimic4demo data covering schema
  validation, label generation, feature extraction, and edge cases
  (minors excluded, admissions with missing features skipped)
Remove GBDT and MLP model variants from the ablation script to match
the project scope (LR baseline, extensible to other models later).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants