feat(sidebar): cross-engine copy, per-table row filters and server-side copies - #2639
Merged
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
Signed-off-by: Ngô Quốc Đạt <datlechin@gmail.com>
Signed-off-by: Ngô Quốc Đạt <datlechin@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the last of #1491. Copy To already carried tables, schemas and whole databases across
connections; what it could not do was cross an engine, narrow a table, or let the server do the work
when both sides were the same connection. All three land here.
Why it was refused
A column's
dataTypeis the source driver's own string, andSchemaSyncScriptBuilderhands itstraight to the target driver's
generateCreateTableSQL. SoObjectCopyEligibility.engineRefusalturned every cross-engine copy away before it read anything: there was no vocabulary in the app that
could say a MySQL
LONGTEXTin PostgreSQL's words.PluginColumnKindhas five cases andColumnTypeis documented display-only; neither carries a length, a precision or a time zone, andDDL needs all three.
The fix
A canonical type layer in
TablePro/Core/CrossEngine/, all pure and unit-tested, sitting between thesource snapshot and the target driver:
CanonicalColumnTypeandCanonicalTypeKind, the third type vocabulary and the only one thatcarries what DDL needs.
SQLTypeParserreads a native spelling per family, and the family matters: Oracle'sDATEhas atime and PostgreSQL's does not, MySQL's
TINYINT(1)is a boolean and SQL Server'sTINYINTis anunsigned byte,
REALis 32 bits on PostgreSQL and 64 on SQLite.SQLTypeRendererwrites it back in the target's own words. Every family answers every kind: a typewith no equivalent becomes the target's widest text column rather than the column going missing.
CrossEngineStructureTranslatorrewrites one table's columns, defaults and indexes and returns anote per change.
CrossEngineIndexTranslatordrops the indexes the target cannot build, because aGINindex reaching MySQL as a plainINDEXover a now-JSONcolumn takes the wholeCREATE TABLEdown with it.CrossEngineValueCoercerreshapes the values that would otherwise arrive wrong: a PostgreSQLtbound into a MySQLTINYINT(1)is0outside strict mode, so everytruein the table wouldsilently become
false; a MySQL0000-00-00is accepted by no other engine; and aTIMESTAMPTZcarries an offset a target with no time zone rejects.
It takes both sides of each column, because neither answers alone:
tis a boolean because thesource column was one, while a time zone is dropped because the target has none. The target
kind is read back out of the spelling the renderer produced rather than carried over from the
source, which is the only way
timestamptz → DATETIMEknows it lost its zone.Nothing in the target driver changed. Quoting, the primary key clause, index and foreign key syntax,
SERIALversusAUTO_INCREMENTversusIDENTITYwere all already correct on the target side.Two guarantees hold the change down. A pair within one type family returns the snapshot it was given,
byte for byte, so MySQL to MariaDB and PostgreSQL to PGlite run the path they always ran. And every
approximation is listed per column in the review step before Copy is available, because a
conversion the user finds out about afterwards is indistinguishable from a bug.
Also in this PR
Per-table
WHEREand row limit. A funnel beside each table in the object list opens the sameExportRowScopeEditorthe export tree uses, so the rule that a filter is one expression is writtenonce. That rule is what keeps a second statement out of the
SELECTthe copy runs. A filtered tablereports no row estimate, because the only count a driver has is of the whole table.
Server-side
INSERT … SELECT. When both sides are one connection the server does the copyitself, and the statement is shown in the review step in place of the query it would otherwise have
walked. The gate is narrow: PostgreSQL cannot name a second database in one statement under any
spelling, so it takes this path only between two schemas of one database, while MySQL, MariaDB, SQL
Server and ClickHouse get the fully qualified name. The plan warns that there is no row-by-row
progress and that Stop cannot interrupt a statement the server has already started.
Views, routines and triggers do not cross. Their definitions are the source engine's own SQL and
nothing here parses it, so each is listed under Left out with the reason and the tables beside it
copy anyway, rather than the whole run being refused.
Verified
verify.sh build: PASSverify.sh testover the fifteen suites that own the changed types: PASS, 201 casesverify.sh uitest CopyObjectsUITests: PASS, 10 casesverify.sh lint TablePro TableProTests TableProUITests: SwiftLint 0 violations. The wrapper'sagent docssub-check reports one stale symbol,AXCellinCLAUDE.md:220, which is red onorigin/maintoo and is untouched here.docs/scripts/check-writing-style.shandcheck-docs-against-source.py: PASSNew unit coverage:
SQLTypeParserTests,SQLTypeRendererTests,CrossEngineStructureTranslatorTests,CrossEngineValueCoercerTests,ObjectCopyServerSideInsertTests, plus the row-scope and coercion cases added toObjectCopySelectQueryTests,ObjectCopyRowCopierTestsandObjectCopySessionTests.Two new
CopyObjectsUITestscases cover the filter flow end to end, against the SQLite sampledatabase. A cross-engine copy cannot get deterministic UI automation: it needs two live connections
on two different engines, which the runner does not have. Its contract is asserted without one in the
five new unit suites.
No screenshots: the only visual additions are a funnel button in an existing list row and a section
in the existing review pane, both of which need a second live connection to photograph in a state
worth looking at.
Review
Codex could not read this diff: its usage limit is exhausted until Sep 7, which
statusreports asfailedand the job log spells out. The fallback reviewer read it instead and found seven issues,all fixed here with a test pinning each:
Result.targetKindscarried the source kind, so the coercer wasdeciding from the wrong side of the crossing. A PostgreSQL
timestamptzrendered as MySQLDATETIMEstill claimed a time zone, so the offset was left on every value, and anint[]rendered as
JSONwas not recognised as needing conversion at all. The unit tests passed becausethey hand-built the kinds; the wiring between translator and coercer was never exercised. Fixed by
carrying both sides and re-parsing the target kind from what the renderer wrote.
.exact, soDECIMAL(65,30)→DECIMAL(38,30)was silent.a weaker constraint than the source had.
.sqlfor PartiQL, so MySQL toDynamoDB passed. It now also requires a type family the translator knows on both sides.
were never written.
not unique across schemas.
Known limits
Redshift takes PostgreSQL's type names and not all of PostgreSQL's types. A copy from PostgreSQL into
it is treated as one family and passes its types through, so a
jsonb,uuidor array column isrefused by the server with its own error rather than being converted. Giving Redshift its own family
would render it as ANSI, whose
TEXTRedshift silently aliases toVARCHAR(256), which is worse.The docs page says so and names the workaround.
Fixes #1491
https://claude.ai/code/session_011EqgjCjCAU6tiiVmnMpF86