feat: obtain the Consul ACL token by a Kubernetes auth method - #192
Open
TaurMorchant wants to merge 20 commits into
Open
feat: obtain the Consul ACL token by a Kubernetes auth method#192TaurMorchant wants to merge 20 commits into
TaurMorchant wants to merge 20 commits into
Conversation
…L token Introduce ConsulLoginCredentials as the seam between the login protocol and the way the bearer token is obtained, and build the core of the new way on top of it: - ConsulLoginCredentials with M2MLoginCredentials and KubernetesLoginCredentials - ConsulClient.login(ConsulLoginCredentials) added with a default implementation; login(String) is deprecated for removal - ConsulOkHttpClient lets a transport IOException out instead of wrapping it, ConsulRestClient wraps MicroserviceRestClientException into IOException - ConsulLogin as the single-operation interface, TokenProvider implements it - SelfTokenReader takes reading an already issued token out of TokenProvider - TokenUpdater works through ConsulLogin, schedules a relogin at 80% of the remaining lifetime and pauses between retries - ProbingConsulLogin probes the new way, falls back to the old one on any failure and sticks to the choice until the pod restarts - ConsulLoginMode plus mode, authMethod and audience inputs in CreateOptions, with per-mode validation - TokenStorageFactory.from builds the chain in one place The auth method default is a placeholder until infrastructure confirms the real name.
…ints Wire the four places that build CreateOptions to the new login mode, auth method and audience, and document the properties: - Spring property names live in ConsulM2MConfigDataLocationResolver; the RestTemplate and WebClient autoconfigurations reference them - values are read through Binder, which converts the lowercase mode written in configuration regardless of the environment conversion service - the ConfigData phase builds the chain through TokenStorageFactory.from and resolves M2MManager from the bootstrap registry only when the bearer token is actually needed, so the kubernetes mode never touches the registry - the Quarkus producer takes the three values as @ConfigProperty, so the mode can be switched without rebuilding the application - the WebClient module gets src/test and its first tests - the properties are documented for both stacks The Quarkus tests do not run yet: cloud-core-quarkus-bom-internal pins core-rest-libraries to the released 7.4.0, so the extension compiles against the published consul-config-provider-common instead of the reactor.
- ConsulLoginCredentials exposes getAuthMethod and getBearerToken, matching the other getters in the module - the fallback mode is named KUBERNETES_WITH_M2M_FALLBACK instead of AUTO, so the property value says what the mode does - retries in TokenUpdater and in the probe are built on failsafe (LoginRetryPolicies) instead of a hand-rolled loop around Thread.sleep, and gain exponential backoff with jitter; the library was already on the classpath through k8s-utils - the delay fraction is one constant, and the method computing it is named after what it returns - the three Spring properties are bound into ConsulLoginProperties, shared by the ConfigData phase and both autoconfigurations; the duplicated name constants are gone - ConsulRestClient says why a transport failure is wrapped into IOException - the m2m mode gets coverage in the entry points, where only kubernetes and the default were checked before @cf_ignore covers the deferred russian TODO in ProbingConsulLogin, kept until the naming pass agreed with the owner.
ConsulLogin read as a verb phrase and told the reader nothing about what the type holds: - ConsulLogin becomes ConsulTokenProvider, and perform becomes getToken, the name already used for a token fetch that goes over the network in M2MManager and KubernetesAudienceToken - TokenProvider becomes LoginTokenProvider: it obtains a token by performing a Consul login with one set of credentials - ProbingConsulLogin becomes KubernetesWithM2MFallbackTokenProvider, matching the name of the mode it serves, and its fields are named after the two ways they hold, so the class no longer mixes generic field names with hardcoded log constants The type stays deleted whole once the m2m way goes away, so a generic fallback decorator would carry generality nothing uses.
Javadoc for the types and methods whose contract is not visible from the signature: what IOException means for a caller, why the delay is a share of the remaining lifetime, why the fallback choice sticks until the pod restarts, why the defaults live in the builder, and why the ConfigData phase binds the mode instead of injecting it. Getters, constructors and the plain value types are left alone. Also folds the review note on the retry supplier: the custom CheckedFunction had lost its argument and became a supplier, so failsafe's own CheckedSupplier replaces it.
A service can be on the new library and the new projected tokens while Consul has not registered the auth method yet. Such a pod fell back to m2m and stayed there until someone restarted it. Now it moves over on its own once Consul is ready: - the fallback is temporary and the switch is permanent, so the state is a ratchet: probe, fall back, recheck, kubernetes for good - the recheck rides on the scheduled relogin instead of a timer of its own, throttled by the new fallback-recheck-interval property, five hours by default - the relogin schedule now follows the expiration of the token just received rather than of the first one: the two auth methods carry different MaxTokenTTL, and a fixed period would leave the new token unrefreshed after the switch - a second INFO record marks the switch; failed rechecks stay silent so an unmigrated fleet does not log on every relogin Rescheduling outlives the migration and fixes an ordinary operation on its own: changing MaxTokenTTL on an auth method without restarting the pods. Without MaxTokenTTL nothing is scheduled and nothing is rechecked, which is acceptable: such a token never expires and needs no new one.
github-actions
Bot
added
documentation
… stack ConsulRestClient logged the login at debug while ConsulOkHttpClient logged it at info, so a Quarkus pod named its auth method and a Spring pod did not. Pods that reuse a token obtained in the ConfigData phase never log in at all, so raising the level alone would still leave them silent; SelfTokenReader now names the auth method taken from the answer of Consul, which is the method that actually issued the token in use. Consul omits AuthMethod for a token no login produced, and that case reads as unknown rather than failing the parse.
A failed relogin repeated the previous delay, so on a short-lived token the retry landed well past the expiration. Every delay is now measured against the expiration of the token the pod holds right now. The spring stack built a second provider for the TokenStorage bean and lost the way the ConfigData phase had already picked, so one transient failure sent a migrated pod back to m2m. Reading the existing token moved onto ConsulTokenProvider, which recovers the way from its AuthMethod.
…Error The scheduled task caught Exception, so an Error walked out of the lambda, landed in a Future nobody reads, and the task never ran again. The pod then kept a token that eventually expired and every Consul read started failing, with nothing in the log to say why. The SPI behind the kubernetes bearer token throws exactly such an Error when no TokenSource provider is on the classpath. The task now catches Throwable, logs it and rearms. That is a deliberate exception to the rule of the module that lets an Error through: the rule earns its keep where a caller can see the failure, and on a pool thread nobody can. Also from review: probe becomes probeKubernetesWay and says why it unwraps the failsafe failure back into an IOException; the SelfTokenReader and Token.getAuthMethod comments catch up with the code, the latter having claimed that a login answer carries no auth method when it does; and both READMEs say the m2m mode does not use the auth method and the audience rather than not reading them, which it does through the builder defaults.
Infrastructure named the kubernetes auth method, so the default stops being a placeholder and a service no longer has to set login.auth-method to migrate.
The login retries compile against net.jodah:failsafe, which reached the module only through k8s-utils, and TokenUpdater reached commons-lang3 the same way for a single StringUtils.isEmpty call. Both would break on the next k8s-utils upgrade, and the repository is already split between net.jodah and dev.failsafe. failsafe is now declared, pinned to the 2.4.4 the other modules use, and the emptiness check goes back to the plain comparison it replaced, so commons-lang3 is not needed at all. mvn dependency:analyze no longer reports either as used-undeclared.
… hit The phase caught IOException only, so the failure type decided the outcome: a 403 from Consul was logged and the application started without an ACL token, while an answer without a SecretID threw PathNotFoundException past the catch and ended the startup. Same cause, two behaviors, neither of them chosen. The catch now covers Exception, so every login failure ends the phase the same way in every mode, and the TokenStorage bean is left to obtain the token. SelfTokenReader gains the empty-body check LoginTokenProvider already had; a null body used to reach JsonPath and come out as a NullPointerException instead of the IOException the callers retry on.
Three duplications, all of them mechanical: - both consul clients built the login request twice, once for the deprecated login(String) and once for login(credentials). The building moves into a private login(authMethod, bearerToken); the two public methods keep the exception handling that genuinely differs between them - the four login inputs were mapped onto CreateOptions in three places. They move into ConsulLoginProperties.toOptionsBuilder, which the ConfigData phase and both autoconfigurations now start from - the two autoconfiguration tests were the same file twice. The mapping is covered once in ConsulLoginPropertiesTest, and each module keeps only the wiring it owns Also from the review: - TokenStorageFactory.from lists KUBERNETES_WITH_M2M_FALLBACK explicitly and throws in the default branch, so a new mode without a branch fails loudly instead of silently becoming the fallback - isKubernetesConfirmed drops volatile: both fields are read and written under the instance lock only, and one of them claiming otherwise misled the reader - the relogin delay is computed once per schedule rather than twice, so the logged value is the one the executor gets - unknownModeIsNotCreated asserted Enum.valueOf and tested the JDK. It becomes everyModeBuildsAProvider, which walks ConsulLoginMode.values() and fails when a new constant has no branch - the quarkus side gains the binding test for fallback-recheck-interval, which only mode had - TestClock moves out of the middle of the test methods
The recheck landed after most of this text was written, and the text still described the behavior it replaced: - ConsulLoginMode claimed a pod keeps one way for its whole life, and the fallback constant claimed it tries the kubernetes way once and stays on m2m for the rest of that life. Both stopped being true when the recheck arrived - the m2m constant said the auth method and the audience are not read. They are: the builder defaults them. They are not used - ConsulTokenProvider promised IOException on a transport failure of the self read, while SelfTokenReader documents the opposite, because the client does not wrap that one - getToken described the probe as happening after a recheck interval, leaving out the first call - LoginRetryPolicies said anything other than IOException means the input is wrong. A missing projected token is the environment, not the input The READMEs gain what an operator has to know and could not read anywhere: - fallback-recheck-interval is the lower bound, not the period. The recheck waits for the next relogin, which runs at 80% of MaxTokenTTL, so a fleet migrates on the MaxTokenTTL clock - a failed login ends differently on the two stacks: the spring ConfigData phase survives it and the TokenStorage bean does not, and on quarkus the bean cannot be produced at all - the fallback record is one per decision, but every login attempt still logs one of its own - cloud.microservice.namespace stays required on quarkus in the kubernetes mode, unlike the namespace on the spring side Also drops the ratchet metaphor, which named the mechanism after a machine part instead of describing it.
TaurMorchant
changed the title
…type Infrastructure is considering an auth method of type jwt instead of type kubernetes. The login request carries the same two fields either way, so the README should not pin the way to one type of auth method on the Consul side.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A second way to obtain the Consul ACL token: the pod exchanges its projected Kubernetes service account token at a
Consul auth method of type
kubernetes. The M2M way keeps working for the whole migration and nothing is removed here.The two halves of the migration ship separately. A service picks up the new library whenever it releases, and the
kubernetesauth method appears in Consul whenever the platform registers it, in either order. A pod that starts beforeConsul is ready therefore begins on the M2M way and moves over by itself later, without a restart.
How the token is obtained
One property picks the way:
kubernetes-with-m2m-fallback(default)kubernetesm2mAn unknown value fails the startup. Both stacks read the mode at runtime, so switching it needs no rebuild.
A failed login is a different matter, and the two stacks part here. The Spring ConfigData phase never ends on one: it
logs the failure and the application starts without an ACL token, in every mode and whatever the failure was. The
TokenStoragebean is stricter, on both stacks — a login failure its retries do not fix ends the startup.Behavior in the main scenarios
The auth method already exists in Consul. The first login goes the kubernetes way and succeeds. No fallback happens,
and the pod logs one
INFOrecord naming the way it uses.The auth method is not registered yet. The kubernetes login fails, and the pod serves tokens through the M2M way. It
logs the reason, the response code, and a truncated body in a single
INFOrecord. That record marks the decision anddoes not repeat on retries; the login attempts themselves keep logging one record each, as they always did.
The auth method appears later. Every scheduled relogin retries the kubernetes way, no more often than
fallback-recheck-interval. The first success switches the pod over and logs a secondINFOrecord. The switch isfinal: the pod never returns to the M2M way, and a later kubernetes failure is a plain failure.
Because the recheck rides on the relogin,
fallback-recheck-intervalis a lower bound and not a period. The reloginruns at 80% of the remaining lifetime, so with a
MaxTokenTTLof 24 hours a pod retries about every 19 hours whateverthe interval says. A fleet migrates on the
MaxTokenTTLclock; loweringMaxTokenTTLon the auth method is what makesit migrate sooner.
The token expires. The pod relogins at 80% of the token's remaining lifetime. The period follows the token the pod
holds right now rather than the first one, because the two auth methods can carry different
MaxTokenTTL. A failedlogin is retried with an exponential delay and jitter, and the schedule survives a failure instead of stopping.
The auth method has no
MaxTokenTTL. Consul issues a token that never expires, so no relogin is scheduled, and norecheck happens either. Such a pod keeps the way it picked at startup.
Spring starts in two phases. The ConfigData phase logs in once and hands the token to the
TokenStoragebean. Thatbean reads
/v1/acl/token/self, sees which auth method issued the token, and continues with that way instead of probinga second time.
Configuration
Four properties, under
spring.cloud.consul.config.login.andquarkus.consul-source-config.login.:modekubernetes-with-m2m-fallbackauth-methodapplications-k8s-m2maudiencenetcrackerfallback-recheck-intervalDurations take the format of their stack:
5hin Spring,PT5Hin Quarkus.The defaults match what the platform registers, so a service normally sets nothing. They live in the options builder,
which also serves callers that build the options directly instead of going through properties.
Set these properties in the service's own configuration:
application.yaml, an environment variable, or a systemproperty. They cannot come from Consul, because reading Consul needs the token they produce.
In
kubernetesmode the Spring side needs neitherM2MManagernor the namespace. The Quarkus producer still readscloud.microservice.namespacein every mode, even though the namespace never reaches Consul.spring.cloud.consul.config.m2m.enabled=falseandquarkus.consul-source-config.m2m.enabled=falsestill turn the wholeexchange off, and then none of the four are used.
Entry points
All four are wired: the Spring ConfigData phase, the RestTemplate and WebClient autoconfigurations, and the Quarkus CDI
producer. In
kubernetesmode the ConfigData phase no longer reaches into the bootstrap registry forM2MManager. Inthe fallback mode it resolves
M2MManageronly when a bearer token is actually needed.Built to be removed
The M2M way goes away in a later major version, and this change is shaped so that removal is a deletion rather than a
rewrite. Everything specific to the M2M way sits behind one seam and carries
M2Min its name: the credentials, thefallback provider, and the mode enum are separate files that get deleted whole. Below the seam the code sees a single
way of obtaining a token and needs no edits. What remains is dropping the deprecated login method, the client
constructors that take an M2M token supplier, and the two builder inputs the M2M way needs.
Compatibility
Nothing breaks here.
ConsulClientgains a login method taking credentials, with a default implementation thatdelegates to the deprecated one, so an outside implementation keeps compiling and working. The options builder gains
methods. Existing methods, bean names, and property names are untouched.
For the release notes:
one log record;
the current token;
IllegalArgumentExceptioninstead of an unmessagedNullPointerException;IOExceptionand endedthe startup on a malformed answer from Consul.
Open