Skip to content

Postgres backend: NewTasks multi-row INSERT uses a 3-stride placeholder for a 2-column table → any orchestration scheduling ≥2 activities fails #138

Description

@erma07

Summary
In the Postgres backend, when an orchestration step schedules more than one activity (parallel fan-out), CompleteOrchestrationWorkItem builds a multi-row INSERT INTO NewTasks whose $n placeholders stride by 3, but the table has 2 columns and exactly 2 arguments are bound per row. The generated SQL therefore leaves a gap ($3) that is bound but never referenced, and Postgres rejects it:

failed to insert into the NewTasks table: ERROR: could not determine data type of parameter $3 (SQLSTATE 42P18)
The orchestration work item can never be completed, so it is abandoned and retried indefinitely — the orchestration never finishes.

Offending code
backend/postgres/postgres.go, CompleteOrchestrationWorkItem, “Save outbound activity tasks” (~line 311):

builder.WriteString("INSERT INTO NewTasks (InstanceID, EventPayload) VALUES ") // 2 columns
for i := 0; i < newActivityCount; i++ {
builder.WriteString(fmt.Sprintf("($%d, $%d)", 3i+1, 3i+2)) // stride 3 ← BUG
if i < newActivityCount-1 { builder.WriteString(", ") }
}
// ...
sqlInsertArgs := make([]interface{}, 0, newActivityCount*2) // 2 args/row
for _, e := range wi.State.PendingTasks() {
sqlInsertArgs = append(sqlInsertArgs, string(wi.InstanceID), eventPayload) // 2 appended
}
_, err = tx.Exec(ctx, insertSql, sqlInsertArgs...)
Root cause
The stride-3 pattern was copied from the sibling inserts in the same function, which genuinely have 3 columns:

INSERT INTO History (InstanceID, SequenceNumber, EventPayload) → ($%d,$%d,$%d) 3i+1..3 ✅ correct
INSERT INTO NewEvents (InstanceID, EventPayload, VisibleTime) → ($%d,$%d,$%d) 3
i+1..3 ✅ correct
INSERT INTO NewTasks (InstanceID, EventPayload) → ($%d,$%d) 3i+1, 3i+2 ❌ wrong (2 columns, needs stride 2)
For 2 tasks it emits VALUES ($1, $2), ($4, $5) while binding only 4 args ($1..$4): $3 is bound but never referenced → 42P18. With 1 task it emits ($1, $2) and works — which is why single-activity orchestrations pass and the bug only appears under fan-out.

Reproduction
Minimal program against a Postgres backend (v0.6.0, no patches):

r := task.NewTaskRegistry()
r.AddOrchestratorN("Repro", func(c *task.OrchestrationContext) (any, error) {
t1 := c.CallActivity("Noop") // two activities scheduled
t2 := c.CallActivity("Noop") // in the same turn
_ = t1.Await(nil); _ = t2.Await(nil)
return nil, nil
})
r.AddActivityN("Noop", func(c task.ActivityContext) (any, error) { return "ok", nil })
// start a postgres-backed TaskHubWorker + client, ScheduleNewOrchestration("Repro")
Observed (worker log, repeating):

orchestration-processor: failed to complete work item: failed to insert into the NewTasks table: ERROR: could not determine data type of parameter $3 (SQLSTATE 42P18)
WaitForOrchestrationCompletion never returns (context deadline exceeded).

Impact
Any workflow that fans out to ≥2 activities in a single orchestration step is unusable on the Postgres backend. Single-activity steps mask it.

Fix

--- a/backend/postgres/postgres.go
+++ b/backend/postgres/postgres.go
@@ // Save outbound activity tasks
for i := 0; i < newActivityCount; i++ {

  •   builder.WriteString(fmt.Sprintf("($%d, $%d)", 3*i+1, 3*i+2))
    
  •   builder.WriteString(fmt.Sprintf("($%d, $%d)", 2*i+1, 2*i+2))
      if i < newActivityCount-1 {
      	builder.WriteString(", ")
      }
    
    }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions