FAQ: Homework 1 Q6 - why does the horsepower median change after fillna with the mode? #420

Course

machine-learning-zoomcamp

Question

Homework 1 Q6: why does the median of horsepower change after filling NAs with the most frequent value?

Answer

df.horsepower.mode()[0] returns 252.0, which is below the original median of 254.0. Filling the
877 missing values with 252 adds a large block of below-median values, which pulls the median down
to 252.0 - so the answer is "Yes, it decreased".

df.horsepower.median()                       # 254.0
mode = df.horsepower.mode()[0]               # 252.0
df.horsepower.fillna(mode).median()          # 252.0

Note that mode() returns a Series (there can be ties), so you must index it with [0]. Also,
fillna does not modify the DataFrame in place unless you assign the result back.

Checklist

• I have searched existing FAQs and this question is not already answered
• The answer provides accurate, helpful information
• I have included any relevant code examples or links