From 2b6ff29a7a4908db816c7622fb3ec9ebcb06ac43 Mon Sep 17 00:00:00 2001 From: FAQ Bot Date: Mon, 5 Oct 2026 20:27:25 +0000 Subject: [PATCH] NEW: Why do we combine the training and validation datasets before evaluating --- ...ine-train-and-validation-before-test-eval.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 _questions/machine-learning-zoomcamp/module-4/031_6862614ca5_combine-train-and-validation-before-test-eval.md diff --git a/_questions/machine-learning-zoomcamp/module-4/031_6862614ca5_combine-train-and-validation-before-test-eval.md b/_questions/machine-learning-zoomcamp/module-4/031_6862614ca5_combine-train-and-validation-before-test-eval.md new file mode 100644 index 00000000..876f469a --- /dev/null +++ b/_questions/machine-learning-zoomcamp/module-4/031_6862614ca5_combine-train-and-validation-before-test-eval.md @@ -0,0 +1,17 @@ +--- +id: 6862614ca5 +question: Why do we combine the training and validation datasets before evaluating + the final model on the test dataset? +sort_order: 31 +--- + +The validation set is used during model selection and hyperparameter tuning (i.e., to estimate performance and decide which model configuration to keep). + +Once you’ve selected the final model, you no longer need the validation set for tuning—so it’s common to retrain the final model on the combined data (train + validation) to let it learn from as much labeled data as possible. + +You should still keep the test set untouched until the very end, because the test set is reserved for the final, unbiased evaluation on unseen data. + +Example: +- `df_train_full = pd.concat([df_train, df_val])` +- Train the final model using `df_train_full` +- Report performance only once on the held-out test dataset \ No newline at end of file