Skip to content

feat(sidebar): cross-engine copy, per-table row filters and server-side copies - #2639

Merged
datlechin merged 3 commits into
mainfrom
feat/cross-engine-object-copy
Sep 5, 2026
Merged

feat(sidebar): cross-engine copy, per-table row filters and server-side copies#2639
datlechin merged 3 commits into
mainfrom
feat/cross-engine-object-copy

Conversation

@datlechin

Copy link
Copy Markdown
Member

Closes the last of #1491. Copy To already carried tables, schemas and whole databases across
connections; what it could not do was cross an engine, narrow a table, or let the server do the work
when both sides were the same connection. All three land here.

Why it was refused

A column's dataType is the source driver's own string, and SchemaSyncScriptBuilder hands it
straight to the target driver's generateCreateTableSQL. So ObjectCopyEligibility.engineRefusal
turned every cross-engine copy away before it read anything: there was no vocabulary in the app that
could say a MySQL LONGTEXT in PostgreSQL's words. PluginColumnKind has five cases and
ColumnType is documented display-only; neither carries a length, a precision or a time zone, and
DDL needs all three.

The fix

A canonical type layer in TablePro/Core/CrossEngine/, all pure and unit-tested, sitting between the
source snapshot and the target driver:

  • CanonicalColumnType and CanonicalTypeKind, the third type vocabulary and the only one that
    carries what DDL needs.

  • SQLTypeParser reads a native spelling per family, and the family matters: Oracle's DATE has a
    time and PostgreSQL's does not, MySQL's TINYINT(1) is a boolean and SQL Server's TINYINT is an
    unsigned byte, REAL is 32 bits on PostgreSQL and 64 on SQLite.

  • SQLTypeRenderer writes it back in the target's own words. Every family answers every kind: a type
    with no equivalent becomes the target's widest text column rather than the column going missing.

  • CrossEngineStructureTranslator rewrites one table's columns, defaults and indexes and returns a
    note per change. CrossEngineIndexTranslator drops the indexes the target cannot build, because a
    GIN index reaching MySQL as a plain INDEX over a now-JSON column takes the whole
    CREATE TABLE down with it.

  • CrossEngineValueCoercer reshapes the values that would otherwise arrive wrong: a PostgreSQL
    t bound into a MySQL TINYINT(1) is 0 outside strict mode, so every true in the table would
    silently become false; a MySQL 0000-00-00 is accepted by no other engine; and a TIMESTAMPTZ
    carries an offset a target with no time zone rejects.

    It takes both sides of each column, because neither answers alone: t is a boolean because the
    source column was one, while a time zone is dropped because the target has none. The target
    kind is read back out of the spelling the renderer produced rather than carried over from the
    source, which is the only way timestamptz → DATETIME knows it lost its zone.

Nothing in the target driver changed. Quoting, the primary key clause, index and foreign key syntax,
SERIAL versus AUTO_INCREMENT versus IDENTITY were all already correct on the target side.

Two guarantees hold the change down. A pair within one type family returns the snapshot it was given,
byte for byte, so MySQL to MariaDB and PostgreSQL to PGlite run the path they always ran. And every
approximation is listed per column in the review step before Copy is available, because a
conversion the user finds out about afterwards is indistinguishable from a bug.

Also in this PR

Per-table WHERE and row limit. A funnel beside each table in the object list opens the same
ExportRowScopeEditor the export tree uses, so the rule that a filter is one expression is written
once. That rule is what keeps a second statement out of the SELECT the copy runs. A filtered table
reports no row estimate, because the only count a driver has is of the whole table.

Server-side INSERT … SELECT. When both sides are one connection the server does the copy
itself, and the statement is shown in the review step in place of the query it would otherwise have
walked. The gate is narrow: PostgreSQL cannot name a second database in one statement under any
spelling, so it takes this path only between two schemas of one database, while MySQL, MariaDB, SQL
Server and ClickHouse get the fully qualified name. The plan warns that there is no row-by-row
progress and that Stop cannot interrupt a statement the server has already started.

Views, routines and triggers do not cross. Their definitions are the source engine's own SQL and
nothing here parses it, so each is listed under Left out with the reason and the tables beside it
copy anyway, rather than the whole run being refused.

Verified

  • verify.sh build: PASS
  • verify.sh test over the fifteen suites that own the changed types: PASS, 201 cases
  • verify.sh uitest CopyObjectsUITests: PASS, 10 cases
  • verify.sh lint TablePro TableProTests TableProUITests: SwiftLint 0 violations. The wrapper's
    agent docs sub-check reports one stale symbol, AXCell in CLAUDE.md:220, which is red on
    origin/main too and is untouched here.
  • docs/scripts/check-writing-style.sh and check-docs-against-source.py: PASS

New unit coverage: SQLTypeParserTests, SQLTypeRendererTests,
CrossEngineStructureTranslatorTests, CrossEngineValueCoercerTests,
ObjectCopyServerSideInsertTests, plus the row-scope and coercion cases added to
ObjectCopySelectQueryTests, ObjectCopyRowCopierTests and ObjectCopySessionTests.

Two new CopyObjectsUITests cases cover the filter flow end to end, against the SQLite sample
database. A cross-engine copy cannot get deterministic UI automation: it needs two live connections
on two different engines, which the runner does not have. Its contract is asserted without one in the
five new unit suites.

No screenshots: the only visual additions are a funnel button in an existing list row and a section
in the existing review pane, both of which need a second live connection to photograph in a state
worth looking at.

Review

Codex could not read this diff: its usage limit is exhausted until Sep 7, which status reports as
failed and the job log spells out. The fallback reviewer read it instead and found seven issues,
all fixed here with a test pinning each:

  • The one that mattered. Result.targetKinds carried the source kind, so the coercer was
    deciding from the wrong side of the crossing. A PostgreSQL timestamptz rendered as MySQL
    DATETIME still claimed a time zone, so the offset was left on every value, and an int[]
    rendered as JSON was not recognised as needing conversion at all. The unit tests passed because
    they hand-built the kinds; the wiring between translator and coercer was never exercised. Fixed by
    carrying both sides and re-parsing the target kind from what the renderer wrote.
  • A clamped decimal precision reported .exact, so DECIMAL(65,30)DECIMAL(38,30) was silent.
  • A unique index over unbounded text became a 255-byte prefix index on MySQL with no note, which is
    a weaker constraint than the source had.
  • The gate keyed on editor language alone, and DynamoDB declares .sql for PartiQL, so MySQL to
    DynamoDB passed. It now also requires a type family the translator knows on both sides.
  • A server-side copy fell back to the row estimate when the driver reported zero, claiming rows that
    were never written.
  • A doc comment attached to the wrong function; accessibility identifiers keyed on a name that is
    not unique across schemas.

Known limits

Redshift takes PostgreSQL's type names and not all of PostgreSQL's types. A copy from PostgreSQL into
it is treated as one family and passes its types through, so a jsonb, uuid or array column is
refused by the server with its own error rather than being converted. Giving Redshift its own family
would render it as ANSI, whose TEXT Redshift silently aliases to VARCHAR(256), which is worse.
The docs page says so and names the workaround.

Fixes #1491

https://claude.ai/code/session_011EqgjCjCAU6tiiVmnMpF86

@mintlify

mintlify Bot commented Sep 4, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
TablePro 🟢 Ready View Preview Sep 4, 2026, 1:52 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

Signed-off-by: Ngô Quốc Đạt <datlechin@gmail.com>
Signed-off-by: Ngô Quốc Đạt <datlechin@gmail.com>
@datlechin
datlechin merged commit 35de39e into main Sep 5, 2026
14 checks passed
@datlechin
datlechin deleted the feat/cross-engine-object-copy branch September 5, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Data transfer between connections: copy tables, schemas, and databases across servers (and engines)

1 participant