Skip to content

SEA: re-fetch links in case of expiry #635

Draft
varun-edachali-dbx wants to merge 382 commits into
less-defensive-downloadfrom
sea-link-expiry
Draft

SEA: re-fetch links in case of expiry #635
varun-edachali-dbx wants to merge 382 commits into
less-defensive-downloadfrom
sea-link-expiry

Conversation

@varun-edachali-dbx

@varun-edachali-dbx varun-edachali-dbx commented Jul 14, 2025

Copy link
Copy Markdown
Contributor

WIP

What type of PR is this?

  • Refactor
  • Feature
  • Bug Fix
  • Other

Description

How is this tested?

  • Unit tests
  • E2E Tests
  • Manually
  • N/A

Related Tickets & Documents

@varun-edachali-dbx
use more fetch methods
5380c7a 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
remove redundant schema_bytes from parent constructor
27b781f 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
only call get_chunk_link with non null chunk index
238dc0a 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
align SeaResultSet structure with ThriftResultSet
b3bb07e 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
remvoe _fill_result_buffer from SeaResultSet
13e6346 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
reduce code repetition
f90b4d4 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
align SeaResultSet with ext-links-sea
23963fc 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
remove redundant methods
dd43715 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
update unit tests
34a7f66 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
remove accidental venv changes
715cc13 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
pre-fetch next chunk link on processing current
fb53dd9 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
reduce nesting
d893877 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
line break after multi line pydoc
a165f1c 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
re-introduce schema_bytes for better abstraction (likely temporary)
d68e4ea 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
Merge branch 'fetch-json-inline' into ext-links-sea
be17812
@varun-edachali-dbx
add fetchmany_arrow and fetchall_arrow
a0705bc 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
Merge branch 'metadata-sea' into fetch-json-inline
1b90c4a
@varun-edachali-dbx
remove accidental changes in sea backend tests
f7c11b9 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
Merge branch 'exec-phase-sea' into metadata-sea
349c021
@varun-edachali-dbx
remove irrelevant changes
6229848 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
remove un-necessary test changes
fd52356 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
remove un-necessary changes in thrift backend tests
64e58b0 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
remove unimplemented methods test
2903473 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
Merge branch 'metadata-sea' into fetch-json-inline
b300709
@varun-edachali-dbx
remove unimplemented method tests
021ff4c 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
modify example scripts to include fetch calls
adecd53 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
Merge branch 'fetch-json-inline' into ext-links-sea
d33e5fd
@varun-edachali-dbx
add GetChunksResponse
e3cef5c 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
remove changes to sea test
ac50669 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
re-introduce accidentally removed description extraction method
03cdc4f 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
clarify that link has already been submitted on getting row_offset
6b1b972 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
return None for out of range
2ca7c2b 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
default link_fetcher to None
00db613 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
Merge branch 'sea-decouple-link-fetch' into sea-link-expiry
47bb758
@varun-edachali-dbx
reduce repeated init
2cd802e 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
varun-edachali-dbx changed the base branch from sea-decouple-link-fetch to sea-migration July 21, 2025 03:05
@varun-edachali-dbx
add_link -> add_links for less heavy link additions
b114599 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@databricks databricks deleted a comment from github-actions Bot Jul 21, 2025
@databricks databricks deleted a comment from github-actions Bot Jul 21, 2025
@databricks databricks deleted a comment from github-actions Bot Jul 21, 2025
@varun-edachali-dbx
Merge branch 'less-defensive-download' into sea-link-expiry
38c9343
@varun-edachali-dbx
move param order
92821e7 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@varun-edachali-dbx
varun-edachali-dbx changed the base branch from sea-migration to less-defensive-download July 21, 2025 07:11
@varun-edachali-dbx
type issues
63deb04 
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
@databricks databricks deleted a comment from github-actions Bot Jul 21, 2025
@databricks databricks deleted a comment from github-actions Bot Jul 21, 2025
@github-actions

github-actions Bot commented Aug 6, 2025

Copy link
Copy Markdown

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@varun-edachali-dbx