Skip to content

[To dev/1.3] [Pipe] Respect history enable with source time range (#18362) - #18385

Open
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-18362-dev-1.3
Open

[To dev/1.3] [Pipe] Respect history enable with source time range (#18362)#18385
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-18362-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Backport #18362 (154644107a82ba69431725390b8346149263bd0e) to dev/1.3.

A global source.start-time or source.end-time previously enabled historical extraction unconditionally. As a result, source.history.enable=false could still transfer data that existed before pipe creation.

This backport:

  • computes historical enablement independently from the global time range;
  • preserves the default behavior when source.history.enable is omitted;
  • preserves historical recovery for pipe restarts and newly added DataRegions;
  • keeps global time-range precedence over history-specific ranges and updates the validation warning;
  • adapts the historical implementation and regression test to the 1.3 PipeHistoricalDataRegionTsFileSource API.

Verification

  • mvn spotless:apply -pl iotdb-core/datanode
  • mvn spotless:apply -pl integration-test -P with-integration-tests
  • PipeHistoricalDataRegionTsFileSourceTest#testGlobalTimeRangeRespectsHistoryEnable: 1 test, 0 failures
  • git diff --check

The full DataNode Maven test was also attempted, but this checkout has unrelated pre-existing compilation errors in Pipe builder/sink classes; the changed sources were isolated-compiled and the targeted test passed.


This PR has:

  • been self-reviewed.
  • added or modified unit tests.
  • added an integration-test regression case.

Key changed/added classes
  • PipeHistoricalDataRegionTsFileSource
  • IoTDBDataRegionSource
  • PipeHistoricalDataRegionTsFileSourceTest
  • IoTDBPipeSourceIT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant