df.http() permits credentials over plaintext HTTP to allow-listed domains

[pinodeca]

Summary

df.http() accepts http:// URLs for every allow-listed endpoint, and forwards caller-supplied headers verbatim. An Authorization header (or an Azure Storage SAS token embedded in the query string) sent to an http:// URL is therefore transmitted in cleartext.

This affects all allow-listed hosts under http-allow-azure-domains — every Azure suffix as well as api.github.com. It is not specific to any one domain.

Details

validate_url_scheme in src/ssrf.rs is a single global check with no per-host policy:

pub fn validate_url_scheme(url: &str) -> Result<(), String> {
    let scheme = url.split("://").next().unwrap_or("").to_ascii_lowercase();
    match scheme.as_str() {
        "http" | "https" => Ok(()),
        other => Err(format!(
            "Blocked: unsupported URL scheme '{other}'. Only http and https are allowed."
        )),
    }
}

validate_url_allowlist never inspects the scheme, so http:// and https:// are treated identically for allow-list purposes. src/activities/execute_http.rs then forwards every string-valued entry of config.headers onto the request unchanged.

Concretely, all of these are currently accepted under http-allow-azure-domains:

• http://myvault.vault.azure.net/secrets/... with a bearer token header
• http://myaccount.blob.core.windows.net/container/blob?<SAS> — the SAS credential is in the URL itself, so it leaks in the request line, not just a header
• http://mycog.cognitiveservices.azure.com/... with an API key header
• http://api.github.com/... with an Authorization header

The Azure cases are arguably the more serious exposure, since Key Vault, Storage, and Cognitive Services credentials are higher-value than a GitHub token, and the Storage SAS case leaks via the URL rather than a header.

Mitigating factors

• The redirect policy is Policy::none() (build_client in src/activities/execute_http.rs), so a plaintext request that receives a 301 to HTTPS is not followed and the credential is not re-sent. The first plaintext transmission has still already occurred, however.
• Many Azure services reject plaintext outright or redirect rather than serving over HTTP. This reduces the practical blast radius but does not prevent the credential from leaving the host in the clear.
• Reaching df.http() at all requires EXECUTE privilege on the function (Layer 0, check_http_privilege), so this is not reachable by arbitrary unprivileged users.

Suggested fix

Reject the http scheme whenever an allow-list feature is active, keeping plaintext permitted only under http-allow-all for local development:

• Enforce HTTPS in validate_url_scheme (or a new check) gated on not(feature = "http-allow-all").
• Apply the check at DSL time as well as execution time, so users get the error when defining the function rather than at run time.
• Add regression tests asserting http:// is rejected for both an Azure suffix host and api.github.com, and that it remains permitted under http-allow-all.
• Update docs/http-security.md §1 and §6.1 to state the HTTPS requirement.

This is a behaviour change for any existing workflow that uses a plaintext URL, so it warrants a CHANGELOG entry and should land as its own PR.

Notes

This is pre-existing behaviour, not a regression. It was surfaced while reviewing #341 (which adds api.github.com to the production tier) but is independent of that change — validate_url_scheme predates it and is unmodified by it.