Skip to content

{182998236}: put retry to back of the queue#5962

Open
emelialei88 wants to merge 1 commit into
bloomberg:mainfrom
emelialei88:feat/verified-retry-back-to-pool
Open

{182998236}: put retry to back of the queue#5962
emelialei88 wants to merge 1 commit into
bloomberg:mainfrom
emelialei88:feat/verified-retry-back-to-pool

Conversation

@emelialei88

@emelialei88 emelialei88 commented May 22, 2026

Copy link
Copy Markdown
Contributor

Summary:

  • Changes verify-retry (optimistic conflict replay) to dispatch the replaying transaction to a new worker thread from the pool instead of replaying inline on the current thread.

What it does:

  • Splits the old srs_tran_replay_int (which looped retries on the same thread) into srs_tran_replay_prepare (dispatches to the thread pool) and srs_tran_replay_once (executes a single retry attempt).
  • When a verify error triggers replay in done_cb_evbuffer, the current worker releases the client back to the pool (RC_INTERNAL_RETRY) instead of blocking while retrying.
  • A new worker picks up the replay in sqlengine_work_appsock and can itself re-dispatch if another retry is needed — each retry gets a fresh thread from the queue.

Why:

  • Replaying inline ties up the worker thread for potentially many retries (with sleeps), starving the thread pool under contention. Putting retries back in the queue lets other work proceed and distributes retry load more fairly.

@emelialei88 emelialei88 force-pushed the feat/verified-retry-back-to-pool branch 2 times, most recently from 2d63d21 to 0882371 Compare May 22, 2026 21:37

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
logfill [db unavailable at finish] **quarantined**
consumer_non_atomic_default_consumer_generated **quarantined**
sql_logfill_autodisable [timeout]

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume
consumer_non_atomic_default_consumer_generated **quarantined**
sql_logfill_autodisable [timeout]

@emelialei88 emelialei88 force-pushed the feat/verified-retry-back-to-pool branch from 0882371 to 4b3170f Compare May 27, 2026 13:48

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_redo [failed with core dumped]
sc_resume_logicalsc_generated **quarantined**
sc_timepart_multiddl_generated
consumer_non_atomic_default_consumer_generated **quarantined**
truncatesc_offline_generated [timeout] **quarantined**

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
consumer_non_atomic_default_consumer_generated **quarantined**
remotecreate
remotecreate_twopc_generated
truncatesc_offline_generated [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**

@emelialei88 emelialei88 changed the title put retry to back of the queue {182998236}: put retry to back of the queue Jun 18, 2026

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
noresetgen
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**

@emelialei88 emelialei88 force-pushed the feat/verified-retry-back-to-pool branch from 4b3170f to 7b68850 Compare June 18, 2026 19:40

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
sp_snapshot_generated
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**

@emelialei88 emelialei88 force-pushed the feat/verified-retry-back-to-pool branch from 7b68850 to 3e87378 Compare June 26, 2026 16:14

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
truncatesc_offline_generated [timeout] **quarantined**

@emelialei88 emelialei88 force-pushed the feat/verified-retry-back-to-pool branch from 03cb28b to 38aa879 Compare June 29, 2026 19:29

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sirace [failed with core dumped]
sc_force [failed with core dumped]
rebuild_table_options [failed with core dumped]
odh_blobs [failed with core dumped]
verify_writes [failed with core dumped]
cinsert_linearizable [failed with core dumped]
maxtable
phys_rep_tiered_firstfile_generated
phys_rep_tiered_nosource_generated
phys_rep_tiered

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sirace [failed with core dumped]
sc_force [failed with core dumped]
verify_writes [failed with core dumped]
odh_blobs [failed with core dumped]
rebuild_table_options [failed with core dumped]
sicountbug [failed with core dumped]
incremental_backup [failed with core dumped]
maxtable
phys_rep_tiered_firstfile_generated
phys_rep_tiered_nosource_generated

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sirace [failed with core dumped]
sc_force [failed with core dumped]
rebuild_table_options [failed with core dumped]
odh_blobs [failed with core dumped]
verify_writes [failed with core dumped]
serial_randtest2 [failed with core dumped]
sicountbug [failed with core dumped]
incremental_backup [failed with core dumped]
maxtable
phys_rep_tiered

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**

Signed-off-by: Emelia Lei <wlei29@bloomberg.net>
@emelialei88 emelialei88 force-pushed the feat/verified-retry-back-to-pool branch from 38aa879 to 69601ad Compare July 2, 2026 18:56
@emelialei88 emelialei88 marked this pull request as ready for review July 2, 2026 18:56

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**

@roborivers roborivers left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_truncate_multiddl_generated [db unavailable at finish] **quarantined**
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
truncatesc_offline_generated [timeout] **quarantined**

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants