1185: Support incremental update of materialized samples view#7726
1185: Support incremental update of materialized samples view#7726labkey-nicka wants to merge 9 commits into
Conversation
|
|
||
| private static @Nullable Timestamp captureChangedSince() | ||
| { | ||
| return new SqlSelector(DbScope.getLabKeyScope(), "SELECT CURRENT_TIMESTAMP").getObject(Timestamp.class); |
There was a problem hiding this comment.
SQLFragment.appendNowTimestamp uses NowTimestamp (System.currentTimeMillis()). Will SELECT CURRENT_TIMESTAMP match System.currentTimeMillis() ? If not, this might result in updated data materialization skipped.
There was a problem hiding this comment.
NowTimestamp is a stand-in for CURRENT_TIMESTAMP. It is intended to be interchangeable as NowTimestamp should always be represented by the database clock when translated to SQL. I could use this instead but elected to go with the hardcoded SQL.
new SQLFragment("SELECT ").appendNowTimestamp()There was a problem hiding this comment.
I remember we had to deal with SQL Server rounding issues in the past. Are both of the usages rounded in the same way?
There was a problem hiding this comment.
For SQL Server, I added a 500ms buffer for the modified check. See buildIncrementalUpdateSql().
labkey-nicka
force-pushed
the
fb_incremental_update
branch
from
9e3ba8b to
219f73f
Compare
2 participants
Rationale
This seeks to address a portion of #1185 by introducing support for incremental update of the materialized view. The rows to update are determined by checking the modified column against a timestamp created when the mutating action is initiated. Additionally, supports incremental insert + update when it is a merge action.
Related Pull Requests
Changes
ExpMaterialTableImplchangedSincetimestamp parameterTasks
Manual testing note