Fixes 33170: fail fast when DQ SDK credentials come back masked

[TeddyCr]

Describe your changes:

Fixes #33170

WorkflowConfigBuilder.with_table() fetches the database service with get_by_id, which the API masks for every caller that is not a bot (ServiceEntityResource.decryptOrNullify → Authorizer.shouldMaskPasswords → !subjectContext.isBot()). The masked value was stored on service_connection and passed straight into Source(...), so a run started with a personal access token failed much later as an authentication error from the source — pointing at the source rather than at the token that actually caused it.

This detects PasswordEntityMasker.PASSWORD_MASK right after the service is fetched and raises with the offending field paths and the remedy.

The check walks the dumped connection rather than the model, which matters: secret: references are valid credentials resolved at connection time by get_connection_url_common, and reading one through get_secret_value() would fire a live secrets-manager call from what should be a cheap validation. model_dump() leaves them intact, so references pass through untouched and only the literal mask trips the guard.

Type of change:

• Bug fix

High-level design:

N/A — small change.

Tests:

Use cases covered

• SDK driven with a personal access token gets an immediate, actionable error naming the masked field instead of a downstream source authentication failure
• SDK driven with a bot token against a server using an external secrets manager still builds — secret: references are not treated as masks
• SDK driven with a bot token against a server storing credentials inline still builds

Unit tests

• I added unit tests for the new logic.
• Files updated: ingestion/tests/unit/sdk/data_quality/test_workflow_config_builder.py
• Added 4 tests: masked credentials raise; secret: reference passes; real credential passes; _find_masked_fields walks nested dicts and lists
• Both behaviour changes were RED-checked — reverting the guard fails exactly test_with_table_raises_when_credentials_are_masked, and reverting the model_str() call fails the same test on service 'root='MySQL''

Backend integration tests

• Not applicable (no backend API changes).

Ingestion integration tests

• Not applicable — the change is a validation guard in the SDK builder, covered by unit tests.

Playwright (UI) tests

• Not applicable (no UI changes).

Manual testing performed

1. ruff format --check and ruff check clean on both changed files
2. pytest ingestion/tests/unit/sdk/ — 454 passed, 1 skipped
3. Reproduced the original behaviour against a live instance: with a PAT the connection came back *********; with the ingestion-bot token it came back as a secret: reference that resolved to a real password at engine-build time

UI screen recording / screenshots:

Not applicable.

Checklist:

• I have read the CONTRIBUTING document.

• My PR title is Fixes <issue-number>: <short explanation>

• My PR is linked to a GitHub issue via Fixes #<issue-number> above.

• I have commented on my code, particularly in hard-to-understand areas.

• For JSON Schema changes: I updated the migration scripts or explained why it is not needed. (No schema changes.)

• For UI changes: I attached a screen recording and/or screenshots above. (No UI changes.)

• I have added tests (unit / integration / Playwright as applicable) and listed them above.

• I have added a test that covers the exact scenario we are fixing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CLaYpRGBYYQbd4hZKhHpgT

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

• Linked issue Data quality SDK silently builds a workflow from masked credentials #33170 is not added to the Shipping project.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

[gitar-bot]

Code Review ✅ Approved

Adds a fail-fast check when DQ SDK credentials come back masked from the API, raising an immediate error with the offending field paths instead of letting the masked value cause a downstream authentication failure. The implementation walks the dumped connection model to preserve secret: references (which resolve at engine-build time) while catching literal mask values. Unit tests cover masked credentials, secret: references, real credentials, and nested field detection. No issues found.

Options

Display: compact → Counting what did not apply, without listing it.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source}

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
92.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud