diff --git a/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md b/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md index 0900b901..7c43027a 100644 --- a/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md +++ b/_questions/machine-learning-zoomcamp/module-2/003_0a7e9c723a_linalgerror-singular-matrix.md @@ -1,9 +1,20 @@ --- id: 0a7e9c723a -question: 'LinAlgError: Singular matrix' +question: 'np.linalg.inv(XTX) throws LinAlgError: Singular matrix — what does this + mean?' sort_order: 3 --- -It’s possible that when you follow the videos, you’ll get a Singular Matrix error. This will be explained in the Regularization video. Don’t worry, it’s normal to encounter this. +`np.linalg.inv` raises `LinAlgError: Singular matrix` when `XTX` is not invertible, which happens when some columns of `X` are linearly dependent (duplicated or redundant features). -You might also receive this error if you invert matrix `X` more than once in your code. \ No newline at end of file +It's normal to hit this while following the Module 2 videos. The regularization lesson fixes it by adding a small value to the diagonal before inverting: + +```python +XTX = X.T @ X +XTX = XTX + r * np.eye(XTX.shape[0]) +w_full = np.linalg.inv(XTX) @ X.T @ y +``` + +To diagnose it, compare `np.linalg.matrix_rank(XTX)` with `XTX.shape[0]`. A smaller rank means redundant columns that you can drop or combine. `np.linalg.pinv` avoids the error, but for homework use `inv` with regularization as in the lessons, because the answer options were computed that way. + +Also check that you compute `X.T @ X` (not `X @ X.T`) and that you don't invert the matrix twice.