redact OBS/OSS/COSN/TOS/BOS secrets in sensitive-config-keys#8613
Open
nishat-06 wants to merge 1 commit into
Open
redact OBS/OSS/COSN/TOS/BOS secrets in sensitive-config-keys#8613nishat-06 wants to merge 1 commit into
nishat-06 wants to merge 1 commit into
Conversation
|
🎊 +1 overall
This message was automatically generated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
hadoop.security.sensitive-config-keyslists credential keys for the s3a, azure, adl and gs connectors only, so the obs, oss, cosn, tos and bos connectors have their secret keys and session tokens matched by no pattern.ConfigRedactor.configIsSensitivereturns false for all of them, andConfiguration.dumpConfiguration/writeXmlthen emit the values verbatim, so the/confservlet on the NameNode, DataNode, ResourceManager, NodeManager and HttpFS hands out object store secrets and session tokens in cleartext, as does reconfiguration logging throughReconfigurableBase.The thirteen missing patterns are added to both
core-default.xmlandCommonConfigurationKeysPublic.HADOOP_SECURITY_SENSITIVE_CONFIG_KEYS_DEFAULT, since the latter is what applies whencore-default.xmlis off the classpath. Bucket-scoped forms follow the existingfs.s3a.secret.key/fs.s3a.*.secret.keyshape. Access key ids are deliberately left unredacted, matching howfs.s3a.access.keyis treated today.How was this patch tested?
TestConfigRedactorcovers this through its existingtestRedactWithCoreDefaultandtestRedactNoCoreDefaultpair, so the added keys are asserted against both the XML default and the Java constant. Each of the thirteen keys is redacted with the patch and returned in cleartext without it.Local surefire did not run to completion on my machine (the build host ran out of disk), so I verified by driving
ConfigRedactordirectly against the builthadoop-commonjar over both config paths, and separately replayed the pre-patch pattern list from trunk to confirm the same thirteen keys come back unredacted. I also matched the new patterns against every<name>in the tree's*-default.xmlfiles to check for over-matching, which returned no hits. Full CI results here would be welcome.For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?No JIRA id yet and no new dependencies; no connector code is touched, so the object store integration suites do not apply. Happy to open a JIRA and retitle if that is preferred before review.
AI Tooling
If an AI tool was used:
where is the name of the AI tool used.
https://www.apache.org/legal/generative-tooling.html
Contains content generated by BugQore, a multi-agent AI code review system. The finding, the patch and this description were produced with it and reviewed by me before submitting.