Summer Discussion MI2

                  August 26, 2021

Sickest-first policy & Predictive Models

for Liver Transplant Candidates in the US

Hoang Thien Ly   

Outline

1/ Transplant problem and sickest-first policy

2/ MELD-Score and optimistic results

3/ Applying ML Techniques
- Process of cleaning datasets
- Models and obtained results from a paper

4/ Related works and Q&A section

Transplant problem & policy

I am patient with end-stage

liver disease. I need a liver-
transplantation.
 Alright. Beforehand, we
need to explain to you and
I am patientdecide
withwhether
end-stage
you may
benefit
liver disease. from a
I need a transplant.
liver-
tranplanatation.
 I will clarify to you, firstly,
I am patient
about thewith end-stage
policy of liver
liver disease.
transplantI need
in theaUS.
liver-
tranplanatation.
- One national waiting list for the
entire US.IYou
amcanpatient with end-stage
register at
liverin different
medical centers disease. I need a liver-
regions.

-
tranplanatation.
Waiting time cannot be
predictable and may vary from
regions of the country and from
the illness levels.
UNOS.org (United Network for Organ Sharing)
- The deceased donor livers are assigned
I am patient
to matches among with end-stage
patients according to
sickest-first policy.
liver disease. I need a liver-
- When get top of the waiting list, we
tranplanatation.
consider compatibility of patient to
donor liver to lower risk of failure:
+ Blood type,
+ Body size,
+ Geographical considerations…
- November 12, 2019:
I am patient
12 965 patients withlist
are on waiting end-stage
for
liver disease. I need a liver-
liver transplantation.
tranplanatation.
- In 2018:
12 700 patients were added to the list.
8 250 liver transplants were performed.
609 patients died while waiting.
627 patients were removed due to
being too sick.
 Problem of assessing
disease severity of a patient
Outline

1/ Transplant problem and sickest-first policy

2/ MELD-Score and optimistic results

3/ Applying ML Techniques
- Process of cleaning datasets
- Models and obtained results from a paper

4/ Related works and Q&A section

 MELD-score and its variants

 The Model for End-stage Liver Disease (MELD-score) estimates the chance
of surviving during the next 3 months of patients with chronic liver disease.

 MELD-score is ranged from 6 to over 40 

 The higher the score, the sicker the patient is, and the higher his/her position
is in waiting list

 It has variants: MELD, MELD-Na, MELD-exception (for liver cancer, or

hepatopulmonary syndrome,…) and PELD (for pediatric patients)
MELD-score and its variants
3-Month Mortality Based on MELD Scores

MELD Score Mortality Probability

40 71.3% mortality

30-39 52.6% mortality

20-29 19.6% mortality

10-19 6.0% mortality

9 or less 1.9% mortality

https://www.hepatitisc.uw.edu/go/management-cirrhosis-related-complications/liver-transplantation-referral/calculate-meld-score
MELD-score and its variants
1 Year Survival Rate Based on MELD Scores

MELD Score Waiting list Post Transplant

Score 10 90% survival 83% survival

Score 15 81% survival 80% survival

Score 20 63% survival 78% survival

Score 25 42% survival 74% survival

Score 30 21% survival 71% survival

Calculating the MELD Score [UndergroundMed]

MELD-score and its variants
As conditions change, MELD changes.
The table below shows how often MELD score gets updated.

MELD Score Recalculation

25 or higher Every week

19-24 Every 30 days

11-18 Every three months

10 or less Once a year

https://www.upmc.com/services/transplant/liver/process/waiting-list/meld-score
Formula of MELD-Score

MELD = 3.78 × ln [Bili (mg/dL)] + 11.2 × ln [INR] +

+9.57 × ln [Creati (mg/dL)] + 6.43

MELD-Na=MELD + 1.32 x (137 - Na) - [0.033 x MELD*(137 - Na)]

Bilirubin:   how well liver clears substance “bile” (żółć) )

 INR: how well liver makes proteins needed for blood to clot (krzepnięcie krwi)

Creatinine: how well kidneys work

 Na: serum sodium, recently added, how well body regulates fluid balance.
Ranged from 125 to 137
Formula of MELD-Score
MELD = 3.78 × ln [Bili (mg/dL)] + 11.2 × ln [INR] +
+9.57 × ln [Creati (mg/dL)] + 6.43

MELD-Na=MELD + 1.32 x (137 - Na) - [0.033 x MELD*(137 - Na)]

 If Bili, INR, Creati < 1, use 1

 If Creati > 4, use 4

Optimistic results from MELD-Score

The MELD-based allocation system was immediately successful,

leading to the first ever reduction in the number of waiting list
candidates and a 15% reduction in mortality among those on the waiting
list.

Freeman, R., Wiesner, R., Edwards, E., Harper, A., Merion, R., Wolfe, R.: Results of the first year of the new liver
allocation plan. Liver Transplant, 10, 7-15 (2004)
Drawbacks of MELD-Score based system

- The log-transformed values of Bili, INR, Creati at 1.0 can be problematic, as a

large percentage waiting list candidates possess Creati levels below, and values
below this threshold can reflect different levels of kidney function.

Sharma, P., Schaubel, D., Sima, C., Merion, R., Merion, R., Lok, A.: Re-weighting the model for end-stage liver disease score components.
Gastroenterology, 135, 1574-1581 (2008)

- Correlation between MELD and outcome is not equally strong for all patients. For
some patients, MELD may not accurately reflect the severity of their condition.
It’s high time for ML Models!!
Outline

1/ Transplant problem and sickest-first policy

2/ MELD-Score and optimistic results

3/ Applying ML Techniques
- Process of cleaning datasets
- Models and obtained results from a paper

4/ Related works and Q&A section

ML methods from a paper of 2021 
ML Pipeline

  Data: waiting list histories from June 30, 2004 to 2016

  Data division: 50-25-25% for train-val-test

 Out of sample test set.

 Metric: ROC AUC (concerning with giving livers to patients who most need them than accurately
predicting mortality risk) 

  Number of features: 50
o 31 known at registration
o 19 updated over time
Data preprocessing

  Cat features ← dummy variables

  Num features ← standardization (zero mean, unit variance in training set)

 MELD & MELD-Na Score ← by formulae

  Missing values:
o Numerical time-series features: forward-filled by last known value
o Other numerical missing values: by median from training set

After pre-processing, data has 241 columns

Training model

  Train models using four different feature sets:

o First set: all available features

o Second set: excludes demographic features: citizenship, education, gender, ethnicity, blood type, income

(age, height, and weight are kept)

o Third set: excludes diagnosis, functional status,… (can be manipulated by doctors)

o Fourth set: only has: Bilirubin, INR, Creatinine, Na and dialysis twice in previous week

 Models: Logistic Regression and Gradient Boosting ensembles with decision trees

 Hyperparameters: detailed in the paper

 Obtained results
 Obtained results
 Conclusions

  Gradient boosting ensembles outperform MELD and MELD-Na for AUC ROC
o 0.935 (grad-boost) vs 0.831 (MELD-Na) for same-day prediction
o 0.834 (grad-boost) vs 0.730 (MELD-Na) for 3-months prediction

 Removing demographic features (race, education, gender,…) and subjective features does not have a large effect
on model performance

 Both model and MELD-Na slightly underestimate mortality in female patients, but no similar trends when
comparing across ethnicities
 References
• Predicting Mortality in Liver Transplant Candidates Explainable AI in Healthcare and Medicine, 2021, Volume 914 ISBN : 978-
3-030-53351-9 Jonathon Byrd, Sivaraman Balakrishnan, Xiaoqian Jiang, Zachary C. Lipton

• Waiting List, MELD Score, Liver Transplants - Dr. Robert S. Brown

• Liver Transplant Waitlist, Part 1 | UCLA Transplantation Services

• https://columbiasurgery.org/liver/liver-transplant-waiting-list

• https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6902892/#:~:text=MELD%2DNa%20was%20calculated%20through,United%2
0Network%20for%20Organ%20Sharing
)

• https://www.organdonor.gov/learn/organ-donation-statistics

• UNOS.org
Outline

1/ Transplant problem and sickest-first policy

2/ MELD-Score and optimistic results

3/ Applying ML Techniques
- Process of cleaning datasets
- Models and obtained results from a paper

4/ Related works and Q&A section

Discussion Time