From 0605a4172057bdf6a596d6353e62680b767bacf8 Mon Sep 17 00:00:00 2001 From: FAQ Bot Date: Mon, 5 Oct 2026 15:47:47 +0000 Subject: [PATCH 1/2] =?UTF-8?q?UPDATE:=20np.linalg.inv(XTX)=20throws=20Lin?= =?UTF-8?q?AlgError:=20Singular=20matrix=20=E2=80=94=20what=20does=20this?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...003_0a7e9c723a_linalgerror-singular-matrix.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md b/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md index 0900b901..49112b87 100644 --- a/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md +++ b/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md @@ -1,9 +1,19 @@ --- id: 0a7e9c723a -question: 'LinAlgError: Singular matrix' +question: 'np.linalg.inv(XTX) throws LinAlgError: Singular matrix — what does this + mean?' sort_order: 3 --- -It’s possible that when you follow the videos, you’ll get a Singular Matrix error. This will be explained in the Regularization video. Don’t worry, it’s normal to encounter this. +A `LinAlgError: Singular matrix` from `np.linalg.inv(XTX)` means `XTX` is not invertible—typically because its rows/columns are linearly dependent (often due to redundant features or insufficient independent information). -You might also receive this error if you invert matrix `X` more than once in your code. \ No newline at end of file +To diagnose, check whether the matrix is rank-deficient: + +- Compute `np.linalg.matrix_rank(XTX)` and compare it to the number of columns of `XTX` (if the rank is smaller, it’s singular). + +Common fixes: + +- Drop or combine redundant features so you get a full-rank `XTX`. +- Instead of the exact inverse, use the pseudo-inverse: `np.linalg.pinv(XTX)`. The pseudo-inverse works even when the matrix is singular, and is often the preferred approach in ML when you can’t guarantee full rank. + +If you still get the error, also verify you’re forming `XTX` correctly (e.g., for linear regression it should be based on `X.T @ X`, not `X @ X.T`). \ No newline at end of file From 49d56573b2b1358a3458fed47283e8c385224877 Mon Sep 17 00:00:00 2001 From: Alexey Grigorev Date: Wed, 14 Oct 2026 07:19:49 +0200 Subject: [PATCH 2/2] Restore regularization fix in singular matrix FAQ --- ...03_0a7e9c723a_linalgerror-singular-matrix.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md b/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md index 49112b87..7c43027a 100644 --- a/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md +++ b/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md @@ -5,15 +5,16 @@ question: 'np.linalg.inv(XTX) throws LinAlgError: Singular matrix — what does sort_order: 3 --- -A `LinAlgError: Singular matrix` from `np.linalg.inv(XTX)` means `XTX` is not invertible—typically because its rows/columns are linearly dependent (often due to redundant features or insufficient independent information). +`np.linalg.inv` raises `LinAlgError: Singular matrix` when `XTX` is not invertible, which happens when some columns of `X` are linearly dependent (duplicated or redundant features). -To diagnose, check whether the matrix is rank-deficient: +It's normal to hit this while following the Module 2 videos. The regularization lesson fixes it by adding a small value to the diagonal before inverting: -- Compute `np.linalg.matrix_rank(XTX)` and compare it to the number of columns of `XTX` (if the rank is smaller, it’s singular). +```python +XTX = X.T @ X +XTX = XTX + r * np.eye(XTX.shape[0]) +w_full = np.linalg.inv(XTX) @ X.T @ y +``` -Common fixes: +To diagnose it, compare `np.linalg.matrix_rank(XTX)` with `XTX.shape[0]`. A smaller rank means redundant columns that you can drop or combine. `np.linalg.pinv` avoids the error, but for homework use `inv` with regularization as in the lessons, because the answer options were computed that way. -- Drop or combine redundant features so you get a full-rank `XTX`. -- Instead of the exact inverse, use the pseudo-inverse: `np.linalg.pinv(XTX)`. The pseudo-inverse works even when the matrix is singular, and is often the preferred approach in ML when you can’t guarantee full rank. - -If you still get the error, also verify you’re forming `XTX` correctly (e.g., for linear regression it should be based on `X.T @ X`, not `X @ X.T`). \ No newline at end of file +Also check that you compute `X.T @ X` (not `X @ X.T`) and that you don't invert the matrix twice.