Skip to content

Data loss — same table name in two schemas maps to identical Iceberg table; second archive silently destroys first schema's cold data #53

Description

@imtiazqa

Steps to Reproduce:

  1. Create public.events2 and analytics.events2 — identical structure, different payloads (public-N vs analytics-N)
  2. archiver register --schema public --table events2 ...
  3. archiver register --schema analytics --table events2 ...
  4. ./bin/archiver --config cf.yaml

Actual Result:

  • Both schemas map to "ice"."default"."events2" — same Iceberg table
  • Second archive (analytics) runs phase 0 wipe on the shared Iceberg table, destroying public cold data
  • SELECT * FROM public.events2 returns analytics-* rows — wrong schema's data
  • public-* rows are permanently lost from cold storage

Root Cause:

// cmd/archiver/main.go:415
iceTable: pgx.Identifier{"ice", cfg.Iceberg.Namespace, t.SourceTable}.Sanitize()
// schema (t.SourceSchema) is never included
Fix: Include schema in Iceberg table name:

iceTable: pgx.Identifier{"ice", cfg.Iceberg.Namespace, t.SourceSchema + "__" + t.SourceTable}.Sanitize()
// public.events2 → ice.default.public__events2
// analytics.events2 → ice.default.analytics__events2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions