fix(mode): Ingest Mode report query metadata [Backport #32533 - 1.13] - #33147
Conversation
* Fixes #22559: Ingest Mode report query metadata * Address Mode connector review feedback * Refine Mode pagination loop guard * Harden Mode connector edge cases
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
github-actions
Bot
added
Ingestion
safe to test
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
Jest test CoverageUI tests summary
|
Report pagination terminated on `len(reports) < REPORTS_PAGE_SIZE`, which made two assumptions about Mode's API that it does not guarantee. If a page ever holds fewer than 30 records, the first page satisfies the condition and the rest of the space is dropped with no warning - reproducing the missing-reports symptom this change set exists to fix. If a space holds an exact multiple of 30 records and Mode clamps the out-of-range page back to the last one, the repeated-page guard raised and killed the whole source, dropping every dashboard of every space. Pagination now stops once a page carries no report that was not already seen in that space, which terminates on an empty page, on a clamped repeat, and on an ignored page parameter, without assuming how many records a full page holds. Reports are de-duplicated by token, falling back to the record itself when Mode omits one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Without `__init__.py`, pytest puts the test file's own directory on sys.path and imports it under its bare basename, so the new Mode test claimed the top-level module name `test_client` and collection of the pre-existing `tests/unit/source/database/burstiq/test_client.py` failed with an import file mismatch. `tests/unit/source/dashboard/qlikcloud` already carries the same marker for the same reason, and the 1.13 backport dropped the one the change carries on main. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Code Review ✅ Approved 2 resolved / 2 findingsBackport fixes Mode report ingestion by addressing duplicate final page handling that aborted the entire report and removing hardcoded pagination termination logic tied to Mode's page size of 30. No issues found. ✅ 2 resolved✅ Edge Case: Duplicate final page aborts entire report ingestion
✅ Edge Case: Pagination termination hardcodes Mode page size of 30
OptionsDisplay: compact → Counting what did not apply, without listing it. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
|
|
3 participants
backport of de358dc on 1.13