Context
The AF-623 discovery scanner (#623, PR #657) runs its regex/checksum detectors only over sampled cells that are scalar Strings (DiscoveryScanService.collectStringColumns). For NoSQL engines whose sampleTable returns nested structures (MongoDB documents, Elasticsearch _source maps, DynamoDB items with nested attributes, Neo4j node property maps), nested leaf values are silently skipped — an email buried in profile.contact.email is never examined.
Proposed fix
- Recursively walk
Map/List cell values in DiscoveryScanService, flattening leaves to dot-path pseudo-columns (profile.contact.email) — the same dot-path convention ColumnMasker already uses for the recursive masking in the Elasticsearch/DynamoDB/BigQuery engines.
- Findings would carry the dot-path in
column_name; confirming one creates the AF-447 tag for that dot-path column, which the engines' recursive masking already honours.
- Cap recursion depth and leaf count per row to keep the scan bounded.
Notes
- The AI pass (
DataDiscoveryAiService) should get the same flattened dot-path columns.
docs/05-backend.md → "Automated sensitive-data discovery" lists this as a v1 limitation — update it when fixed.
Context
The AF-623 discovery scanner (#623, PR #657) runs its regex/checksum detectors only over sampled cells that are scalar
Strings (DiscoveryScanService.collectStringColumns). For NoSQL engines whosesampleTablereturns nested structures (MongoDB documents, Elasticsearch_sourcemaps, DynamoDB items with nested attributes, Neo4j node property maps), nested leaf values are silently skipped — an email buried inprofile.contact.emailis never examined.Proposed fix
Map/Listcell values inDiscoveryScanService, flattening leaves to dot-path pseudo-columns (profile.contact.email) — the same dot-path conventionColumnMaskeralready uses for the recursive masking in the Elasticsearch/DynamoDB/BigQuery engines.column_name; confirming one creates the AF-447 tag for that dot-path column, which the engines' recursive masking already honours.Notes
DataDiscoveryAiService) should get the same flattened dot-path columns.docs/05-backend.md→ "Automated sensitive-data discovery" lists this as a v1 limitation — update it when fixed.