Skip to content

Distributor signing keys: empty entry in -distributor.sign-write-requests-keys silently downgrades V01 stream auth to forgeable #7579

@friedrichg

Description

@friedrichg

Describe the bug

SecretStringSliceCSV.Set() in pkg/util/flagext/secretstringslicecsv.go:23 uses strings.Split(s, ","), which preserves empty entries. A stray or trailing comma in -distributor.sign-write-requests-keys (e.g. newkey,, or a fat-fingered newkey, ,oldkey) leaves a "" in the key list.

The ingester's NewStreamSigningServerInterceptor(keys...) then accepts the stream when subtle.ConstantTimeCompare(sig, computeStreamHMAC("", orgID)) == 1. Because orgID here is the worker name (ingester-<addr>-stream-push-worker-<N> — documented in pkg/ingester/ingester.go), it isn't a secret. So an empty key collapses HMAC verification to HMAC-SHA256("", <guessable>), i.e. forgeable with no secret.

The distributor still signs with keys[0], so legitimate traffic flows and the misconfiguration is invisible. Net effect: a single dangling comma silently downgrades V01 stream authentication (PR #7475) to no-auth, re-opening the tenant-impersonation surface that V01 closed.

To Reproduce

  1. Start Cortex with -distributor.sign-write-requests=true -distributor.sign-write-requests-keys=newkey, (note the trailing comma).
  2. From a client that knows the worker-name format, sign a PushStream request with HMAC-SHA256("", "ingester-<addr>-stream-push-worker-0").
  3. The ingester accepts the forged signature.

Expected behavior

SecretStringSliceCSV.Set() should TrimSpace each entry and drop or reject empty entries. If trimming leaves zero valid keys, return an error from Set(). Equivalently, the ingester's stream interceptor should refuse to construct over an empty-string key.

Environment:

Additional Context

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions