What happens
HttpProtocol.configure() builds the browser context options and, when http.proxy is set, it calls setIgnoreHTTPSErrors(true) right after setProxy(). The two calls are tied together: there is no configuration key that controls the flag, and no way to keep certificate validation on while fetching through a proxy. The flag is a NewContextOptions setting, so it applies to the whole context: every page, every navigation and every subresource, not just the leg that terminates at the proxy. Without a proxy the browser validates normally, so the behaviour flips on an unrelated setting.
Where
external/playwright/src/main/java/org/apache/stormcrawler/protocol/playwright/HttpProtocol.java:162-170, config keys http.proxy, http.proxy.username, http.proxy.password.
final Proxy globalProxy = getProxy(proxyServer, proxyUser, proxyPwd);
if (globalProxy != null) { b_c_options.setProxy(globalProxy); b_c_options.setIgnoreHTTPSErrors(true);
}Why it matters
An operator who configures a proxy gets a browser that renders any HTTPS page whose certificate does not verify, and the browser executes the JavaScript on that page. Nothing in playwright-conf.yaml or the module README mentions the behaviour, so the operator has no reason to expect it and no key to turn it off. Core has a comparable setting for okhttp (http.trust.everything, documented in configuration.adoc), which is exactly what is missing here: a named, documented switch rather than a side effect. The exposure needs someone able to intercept or answer for the target host, so it only bites deployments where the network path between the proxy and the origin is not trusted.
Reproduction
No automated test. configure() launches or connects to a real Chromium instance, and the flag only shows up in the browser's TLS handshake, so reproducing it needs a browser and a TLS endpoint. The module's own ProtocolTest already requires a running Chrome for the same reason.
Manual steps:
- Serve any page over HTTPS with a certificate the container does not trust (for example a self-signed one).
- Configure the playwright protocol without a proxy and fetch the URL. The fetch fails on the certificate.
- Set
http.proxy to a proxy that forwards to the same host and fetch again. The page is fetched and rendered.
Suggested fix
Introduce a dedicated key, for example playwright.ignore.https.errors, read it in configure() and pass it to setIgnoreHTTPSErrors() independently of whether a proxy is set. Default it to false and log a warning when it is enabled. Document it in playwright-conf.yaml next to the other playwright keys. Note the compatibility impact: deployments running behind a TLS-intercepting egress proxy currently depend on the implicit behaviour and will need either the new key or the proxy CA installed in the browser image.
What happens
HttpProtocol.configure()builds the browser context options and, whenhttp.proxyis set, it callssetIgnoreHTTPSErrors(true)right aftersetProxy(). The two calls are tied together: there is no configuration key that controls the flag, and no way to keep certificate validation on while fetching through a proxy. The flag is aNewContextOptionssetting, so it applies to the whole context: every page, every navigation and every subresource, not just the leg that terminates at the proxy. Without a proxy the browser validates normally, so the behaviour flips on an unrelated setting.Where
external/playwright/src/main/java/org/apache/stormcrawler/protocol/playwright/HttpProtocol.java:162-170, config keyshttp.proxy,http.proxy.username,http.proxy.password.Why it matters
An operator who configures a proxy gets a browser that renders any HTTPS page whose certificate does not verify, and the browser executes the JavaScript on that page. Nothing in
playwright-conf.yamlor the module README mentions the behaviour, so the operator has no reason to expect it and no key to turn it off. Core has a comparable setting for okhttp (http.trust.everything, documented in configuration.adoc), which is exactly what is missing here: a named, documented switch rather than a side effect. The exposure needs someone able to intercept or answer for the target host, so it only bites deployments where the network path between the proxy and the origin is not trusted.Reproduction
No automated test.
configure()launches or connects to a real Chromium instance, and the flag only shows up in the browser's TLS handshake, so reproducing it needs a browser and a TLS endpoint. The module's ownProtocolTestalready requires a running Chrome for the same reason.Manual steps:
http.proxyto a proxy that forwards to the same host and fetch again. The page is fetched and rendered.Suggested fix
Introduce a dedicated key, for example
playwright.ignore.https.errors, read it inconfigure()and pass it tosetIgnoreHTTPSErrors()independently of whether a proxy is set. Default it to false and log a warning when it is enabled. Document it inplaywright-conf.yamlnext to the other playwright keys. Note the compatibility impact: deployments running behind a TLS-intercepting egress proxy currently depend on the implicit behaviour and will need either the new key or the proxy CA installed in the browser image.