Though we are winding down the OPE project, I wanted to document an issue in case the project is ever deployed again. When running for a longer time, the webhooks sometimes fails, preventing students from accessing their pods.
The webhook pod gets a new certificate signed by a new certificate authority (CA), but the Kubernetes API server still trusts the old CA stored in:
webhooks:
- clientConfig:
caBundle:
Then the API server tries calling the webhook and gets:
x509: certificate signed by unknown authority
because the presented cert chains to a different CA.
It can usually be solved by deleting and redeploying the webhook, but if the project is ever run again, it would make sense to find a long term fix to this issue
Though we are winding down the OPE project, I wanted to document an issue in case the project is ever deployed again. When running for a longer time, the webhooks sometimes fails, preventing students from accessing their pods.
The webhook pod gets a new certificate signed by a new certificate authority (CA), but the Kubernetes API server still trusts the old CA stored in:
Then the API server tries calling the webhook and gets:
because the presented cert chains to a different CA.
It can usually be solved by deleting and redeploying the webhook, but if the project is ever run again, it would make sense to find a long term fix to this issue