(Full path) Racial Disparities and Mistrust in End-of-Life Care#1061
Open
aaronx2-illinois wants to merge 9 commits intosunlabuiuc:masterfrom
Open
Conversation
Commit Detail Add EOL mistrust workflow tasks and target helpers - Implemented task definitions for the EOL mistrust , including: - Left AMA prediction - In-hospital mortality prediction - Code status prediction - Added helper functions for data normalization, age calculation, length of stay calculation, and mapping of ethnicity and insurance. - Created a base task class for downstream predictions and specific task wrappers for each target. - Included necessary validation for input data and defined schemas for input and output.
Commit Detail Refactor code structure for improved readability and maintainability
… Logic commitDetail - Added a new function to load example modules for integration tests. - Updated test data in `test_eol_mistrust_Integration.py` for clarity and accuracy. - Enhanced insurance mapping to include fallback for unrecognized plans. - Modified cohort building tests to reflect changes in admission criteria. - Renamed tests for clarity and updated assertions to match new logic. - Improved note corpus filtering to ensure only relevant discharge summaries are included. - Adjusted noncompliance and autopsy label tests to better reflect case sensitivity and context. - Added new tests for race-based treatment analysis by acuity and ensured proper binning. - Updated model training tests to verify correct hyperparameter settings. - Enhanced module implementation tests to ensure accurate cohort and label processing.
CommitMsg: 1.Update test_eol_mistrust_module.py for new scoring logic and data handling 2.Add assertions for discharge categories and sentiment analysis 3.Add test_eol_mistrust_task.py for task module coverage 4.Test code status target building and task mapping consistency 5.Use dummy patient and event classes for test cases
…rsion 1. Added tests for EOLMistrustClassifier to confirm it extends BaseModel and accepts task-style inputs. 2. Added end-to-end classifier tests for both normal and paper-like dataset preparation. 3. Updated assertions to match new expected treatment totals and code status outputs. 4. Cleaned up dataset setup code in tests for better clarity.
- PEP8 88-char cleanup across 6 core files + example - Added synthetic-data reproducibility note in example docstring - Added 3 RST stubs + index updates for dataset/model/tasks - Added conftest.py with `--run-slow` opt-in; tagged slow tests - Refactored models/eol_mistrust.py helper for readability
Additional updates to PyHealth - EOL project - Amy Hwang
- Expand module-level docstrings in dataset, model, and task modules with Boag et al. 2018 paper citation and URL - Add missing docstrings to private helper functions and methods: _path_variants, _table_assets_exist, _discover_optional_tables, _infer_tensor_input_size, _mean_pool_sequence, _project_tensor, _embed_text_field, _require_columns, _coerce_timestamp, _normalize_token, _normalize_code_status_mode, _normalize_dataset_prepare_mode, _calculate_age_years, _calculate_los_days, _calculate_paper_like_los_days, _build_code_status_target_normal, _build_code_status_target_paper_like - Add section comments to eol_mistrust.yaml config for table groups - Enhance RST documentation with paper links and modality descriptions - Add paper reference to example script header - Add docstrings to four public wrapper functions in pyhealth/models/eol_mistrust.py that bind the generic proxy helpers to specific label columns
add paper references, missing docstrings, and YAML comments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contributor:
Aaron Xing (Email: aaronx2@illinois.edu / NetID:aaronx2)
Amy Sui Hwang (Email: ahwang22@illinois.edu / NetID:ahwang22)
Patrick Yuxuan Wu (Email: pywu4@illinois.edu / NetID:pywu4)
Contribution Type:
Full path (Dataset + Task + Model + Paper Replication)
Original Paper:
Boag et al. (2018), Racial Disparities and Mistrust in End-of-Life Care
https://proceedings.mlr.press/v85/boag18a.html
Description:
This PR adds our replication of Racial Disparities and Mistrust in End-of-Life Care to PyHealth.
We implemented the full pipeline, including the dataset, downstream tasks, model, and tests.
Main additions:
EOLMistrustDataset(BaseDataset): a dataset wrapper built on the combined MIMIC-III CSV filesEOLMistrustDownstreamMIMIC3(BaseTask): a downstream task base class with 3 binary prediction tasks:EOLMistrustClassifier(BaseModel): a multimodal classifier using sequence pooling, linear projection, hashed-text embedding, and an MLPThis PR supports two research paths, plus a PyHealth-native demo, all from the same entry point:
(
BaseDataset -> set_task -> BaseModel -> Trainer.train/evaluate)python examples/eol_mistrust_mortality_classifier.py --root EOL_Workspace/eol_mistrust_required_combined --task-demo --task-demo-train-evalpython examples/eol_mistrust_mortality_classifier.py --root EOL_Workspace/eol_mistrust_required_combined --repetitions 10python examples/eol_mistrust_mortality_classifier.py --root EOL_Workspace/eol_mistrust_required_combined --paper-like-dataset-prepare --repetitions 10Synthetic/unit tests use generated pseudo data and do not require MIMIC-III credentials. Slow full-pipeline tests are marked with
@pytest.mark.slow.Results are written under:
EOL_Workspace/EOL_Result/EOL_(normal|Paperlike)_<timestamp>/We also included:
--ablation-study) to compare the two preparation modes@pytest.mark.slowFiles to Review:
Main implementation:
pyhealth/datasets/eol_mistrust_dataset.pypyhealth/datasets/eol_mistrust.pypyhealth/datasets/configs/eol_mistrust.yamlpyhealth/tasks/eol_mistrust.pypyhealth/models/eol_mistrust_classifier.pypyhealth/models/eol_mistrust.pyExample:
examples/eol_mistrust_mortality_classifier.pyDocumentation:
docs/api/datasets/pyhealth.datasets.EOLMistrustDataset.rstdocs/api/tasks/pyhealth.tasks.eol_mistrust.rstdocs/api/models/pyhealth.models.EOLMistrustClassifier.rstdocs/api/datasets.rstdocs/api/tasks.rstdocs/api/models.rstTests:
tests/core/test_eol_mistrust_dataset.pytests/core/test_eol_mistrust_task.pytests/core/test_eol_mistrust_model.pytests/core/test_eol_mistrust_Integration.pytests/core/test_eol_mistrust_TrainingAndEvaluation.pytests/core/test_eol_mistrust_module.pyRelation to paper replication:
This PR directly implements our replication of Boag et al. (2018). The
paper_likepath is designed to reproduce the original notebook behavior, while theNormalpath provides a corrected PyHealth-native implementation after reviewing the original pipeline assumptions.