Skip to content

Fixes #32606: Backport column-name classification fix to 2.0 - #33169

Open
stout-yeoman wants to merge 1 commit into
2.0from
backport/2.0/pr-32607
Open

Fixes #32606: Backport column-name classification fix to 2.0#33169
stout-yeoman wants to merge 1 commit into
2.0from
backport/2.0/pr-32607

Conversation

@stout-yeoman

Copy link
Copy Markdown
Contributor

Describe your changes:

Replaces #33125 with the same verified patch on an upstream-hosted branch so the branch-local 2.0 CI workflow runs.

Fixes #32606

Backports #32607 (commit 807dfa38fd1d97927dbdd1cf5076164db5952fdb) to 2.0. When sampling supplies no fields, auto-classification now traverses metadata leaf columns with empty sample values so column-name recognizers still run. When sampled fields exist, those fields and their values remain authoritative. The traversal guards against cycles and excessive nesting.

Resolved the import conflict while retaining the release branch’s typing conventions. Table, container, and topic classification remain supported.

Type of change:

  • Bug fix

High-level design:

Not applicable — backport of a focused processor fix. No schema or API changes.

Tests:

Use cases covered

Missing/empty sample data, nested metadata leaves, supported entity types, cycle/depth guards, sampled field selection and values, dotted paths, unmatched fields, disabled classification, and entities without columns.

Unit tests

Backported ingestion/tests/unit/metadata/pii/test_base_processor.py.

From ingestion/, using Python 3.11 and models generated from this release branch:

PYTHONPATH=src python -m pytest tests/unit/metadata/pii tests/unit/pii/test_processor.py --cov=metadata.pii.base_processor --cov-report=term-missing -q

128 passed; 95% combined line/branch coverage for metadata.pii.base_processor. Repository formatting hooks and git diff --check passed.

Backend integration tests

Not applicable — no backend API changes.

Ingestion integration tests

Backported the regression in ingestion/tests/integration/auto_classification/databases/test_global_sample_data_config.py, including fresh-table isolation. All three tests collect successfully. The 2.0 backport has not been executed against a local 2.0 database/server stack; CI integration validation remains required.

Playwright (UI) tests

Not applicable — no UI changes.

Manual testing performed

No live-stack manual test performed. Reproduction for CI/manual verification: configure a column-name recognizer matching address, disable both global sample-data reading and storage, run auto-classification, and verify its generated/suggested tag appears on address while sample data remains absent. This scenario is encoded in the integration regression above.

UI screen recording / screenshots:

Not applicable — no UI changes.

Checklist:

  • PR title references issue 32606 and the target release.
  • Linked to the original issue and PR.
  • Backported regression tests and ran the relevant unit suites.
  • Retained comments explaining traversal safeguards and test isolation.
  • Schema/migrations: not applicable.
  • UI recording: not applicable.
  • Live database integration execution: pending CI.

)

* Fixes #32606: Classify column names when sample data is disabled

* fix(ingestion): cover nested classification across entity types

* fix(ingestion): isolate auto-classification integration test state

* fix(ingestion): remove redundant sample data test cleanup

(cherry picked from commit 807dfa3)
@stout-yeoman
stout-yeoman requested a review from a team as a code owner September 10, 2026 19:36
@stout-yeoman
stout-yeoman requested review from Khairajani and removed request for a team September 10, 2026 19:36
@github-actions

github-actions Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

✅ PR checks passed

The linked issue has a description and all required Shipping project fields set. Thanks!

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions

github-actions Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

✅ Playwright Results — workflow succeeded

Validated commit f5e6546a91afb1a2162c1b091dbf4615b057cd0e in Playwright run 34521863013, attempt 1.

✅ 110 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 49m 31s

⏱️ Max setup 3m 15s · max shard execution 13m 8s · max shard-job elapsed before upload 18m 38s · reporting 7s

🌐 210.32 requests/attempt · 1.79 app boots/UI scenario · 0.00% common-shard skew

Optimization targets still in progress:

  • Browser traffic was 210.32 requests per attempt (convergence target: fewer than 200).
  • Application boot ratio was 1.79 per UI scenario (216 boots / 121 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
✅ Shard chromium-01 46 0 0 0 0 0
✅ Shard ingestion-01 21 0 0 0 0 0
✅ Shard ingestion-02 43 0 0 0 0 0

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@gitar-bot

gitar-bot Bot commented Sep 10, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Backports the column-name classification fix from #32607 to the 2.0 release branch, enabling auto-classification to run on column names even when sample data is disabled. The traversal guards against cycles and excessive nesting while preserving the authority of sampled fields when present. Unit tests achieve 95% combined line/branch coverage with 128 tests passing, and integration tests validate the regression scenario. No issues found.

Options

Display: compact → Counting what did not apply, without listing it.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@stout-yeoman stout-yeoman self-assigned this Sep 10, 2026
@stout-yeoman stout-yeoman added the safe to test Add this label to run secure Github workflows on PRs label Sep 10, 2026
@stout-yeoman stout-yeoman moved this to In Review / QA 👀 in Shipping Sep 10, 2026
@sonarqubecloud

Copy link
Copy Markdown

@stout-yeoman
stout-yeoman enabled auto-merge (squash) September 10, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

Status: In Review / QA 👀

Development

Successfully merging this pull request may close these issues.

Auto Classification skips column-name recognizers when sample data is disabled

1 participant