[FAQ] Module 1: Why can filling missing values change the median? #423

Question

In Homework 1, we calculate the median of horsepower, fill missing values with the mode, and calculate the median again.

Why can the median change after using fillna()?

Answer

fillna() adds values to the distribution. The median may change when the inserted values alter the position of the middle observation(s) after the values are ordered.

To compare both values without modifying the original DataFrame:

median_before = df["horsepower"].median()
mode_value = df["horsepower"].mode()[0]

horsepower_filled = df["horsepower"].fillna(mode_value)
median_after = horsepower_filled.median()

fillna() returns a new Series by default. Assigning it to horsepower_filled keeps the original df["horsepower"] unchanged.