chore: clarify duplicates filter settings copy

[iabaako]

Pull Request Summary 🚀

What does this PR do? 📝

Rewords and restructures the "Records to Include" settings section on the duplicates report page. Separates the "missing values as duplicates" toggle into its own "Missing Values" box, distinct from the record-filter controls, and clarifies the help text/captions for both.

Why is this change needed? 🤔

Users didn't understand what the filter section on the duplicates settings page does — specifically, that it scopes which records are checked for duplicates (e.g. only households with complete interviews or valid consent), as opposed to defining what counts as a duplicate. Grouping the missing-value toggle in the same box as the filter compounded the confusion, since that toggle affects duplicate matching logic, not record inclusion.

How was this implemented? 🛠️

• Renamed the "Duplicates Conditions" section to "Records to Include" and rewrote its intro text with a concrete example and an explicit statement of the default (no filter = all records checked).
• Moved the "Consider missing values as duplicates" toggle out of the filter-rendering function into its own _render_missing_as_duplicates_toggle helper, displayed in a separate bordered container titled "Missing Values" with a caption clarifying it affects matching, not scope.
• Tightened help text on the Condition Column / Condition Type widgets with concrete examples.
• Renamed the "Apply Condition" button to "Apply Filter" and added a tooltip plus an explicit "no filter applied" caption when no condition column is selected.

How to test or reproduce ? 🧪

1. Open a project with data imported, go to the Duplicates report page, and open the Settings tab.
2. Confirm "Missing Values" and "Records to Include" render as two separate boxes.
3. Confirm the filter widgets, help text, and captions read as described above, and that applying/clearing a filter still works as before (no behavior change, copy only).

Screenshots (if applicable) 📷

Not included — text/UI copy change, see reproduction steps above.

Checklist ✅

• I have run and tested my changes locally
• I have limit this PR to less than 1000 lines of code change (if not, explain why)
• I have updated/added tests to cover my changes (if applicable) — copy-only change, no new logic
• I have updated/added requirements to cover my changes (if applicable)
• I have run linting and formatting on any code changes (if applicable)
• I have updated the documentation (README, etc.) accordingly — no user-facing docs reference this copy
• I have reviewed and resolved any merge conflict

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
68.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Pull request overview

This PR refactors and rewrites the duplicates report Settings UI to better communicate what the “record filter” does, and to separate the missing-values toggle into its own section.

Changes:

• Split the settings UI into two bordered sections: Missing Values and Records to Include.
• Extracted the missing-values toggle into a new _render_missing_as_duplicates_toggle() helper and threaded its value into the filter UI.
• Updated button labels, captions, and widget help text to make the record-filter behavior clearer.

Suppressed comments (2)

src/datasure/checks/duplicates.py:747

• The toggle help text currently describes “missing values matched as a duplicate pair”, but this setting is only used when building the filter expression (col_expr.is_null() | filter_expr). Update the help copy (and optionally the docstring) to reflect that it includes rows where the condition column is null when applying the filter.

    missing_as_duplicates = st.toggle(
        label="Consider missing values as duplicates",
        value=default_missing_as_duplicates,
        key="duplicates_missing_as_duplicates_key",
        help=(
            "If enabled, two records with a missing value in the same column "
            "count as a duplicate pair. If disabled, missing values are never "
            "matched as duplicates."
        ),

src/datasure/checks/duplicates.py:808

• The new UI copy/caption suggests “no filter applied” is possible, but Condition Column is a st.selectbox(options=data.columns) with no blank/None option, so condition_col is always truthy and the if not condition_col: caption + the return ... else {} branch are effectively unreachable. Add an explicit “no filter” option (e.g., empty string) so the default can truly be “all records” as described.

    co1, co2, co3 = st.columns([0.3, 0.3, 0.4])
    all_columns = data.columns
    with co1:
        condition_col = st.selectbox(
            label="Condition Column",
            options=all_columns,
            key="duplicates_condition_col_key",
            help=(
                "Column used to decide which records are included, e.g. a "
                "consent or interview-status column."
            ),
            on_change=trigger_save,
            kwargs={"state_name": TAB_NAME + "_condition_col"},
        )

        save_check_settings(settings_file, TAB_NAME, {"condition_col": condition_col})

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.