Skip to content

fix(datagrid): filters on PostgreSQL non-text columns and MongoDB collections named like db methods - #2625

Merged
datlechin merged 1 commit into
TableProApp:mainfrom
MinhQuang28:fix/query-filter-postgres-mongo
Sep 3, 2026
Merged

fix(datagrid): filters on PostgreSQL non-text columns and MongoDB collections named like db methods#2625
datlechin merged 1 commit into
TableProApp:mainfrom
MinhQuang28:fix/query-filter-postgres-mongo

Conversation

@MinhQuang28

Copy link
Copy Markdown
Contributor

Summary

Four defects in the way the filter panel and the browse query are built for PostgreSQL and MongoDB, found while auditing both paths side by side.

PostgreSQL: pattern filters on a non-text column fail on the server

Root cause. FilterSQLGenerator emits "col" ILIKE '%x%', "col" ~ 'x' and LOWER("col") on whatever column the row names. MySQL and SQLite coerce the operand; PostgreSQL does not, and answers operator does not exist: uuid ~~* unknown (likewise for integer, timestamptz, an enum, jsonb, text[]). ColumnTypeClassifier files uuid and every unknown type under .text, so the generator could not even tell a uuid from a varchar, and allowsCaseFolding happily wrapped it in LOWER(). A Contains filter on a uuid primary key, the default operator with its default ignore-case setting, hits this every time.

Fix. SQLDialectDescriptor gains textCastTypeName, nil by default. A dialect that names one (PostgreSQL sets TEXT) has every column that is not character data cast with CAST(col AS TEXT) before LIKE, ILIKE, a regex, a case fold, and the = '' half of Is empty. A plain = keeps the column's own type and its index. ColumnTypeSQLQuoting.isCharacterType reads the raw type name behind a .text column, which is the only place uuid and varchar still differ. Cockroach, Redshift and PGlite inherit the descriptor.

The field is added through a new SQLDialectDescriptor initializer overload; the previous full initializer keeps its signature and is now @_disfavoredOverload, so an already-built plugin keeps loading. withCaseSensitivityStyle carries the field, which is how the Redshift variant gets it.

PostgreSQL: .array was treated as text

ColumnTypeSQLQuoting.isKnownTextLike listed .array, so Is empty produced "tags" = '' (malformed array literal) and ignore-case produced LOWER("tags"). Arrays are now outside the text-like set: Is empty is IS NULL, and Contains searches the array's text form through the cast above.

MongoDB: a collection named like a db method cannot be opened

Root cause. MongoDBQueryBuilder and MongoDBStatementGenerator spelled the collection as db.stats or db["stats"]. Both go through the DB proxy in MongoScriptPrelude, whose get trap returns a prototype member before it falls back to a collection, so stats, version, getName, toString, constructor and the rest come back as functions and .find() throws db.stats.find is not a function. mongosh resolves db.stats the same way, so the MQL export had the same bug in its output.

Fix. MongoCollectionAccessor in PluginKit is the one place the accessor is spelled: db.<name> for a plain identifier that no db method shadows, db.getCollection("<name>") for everything else. The query builder, the statement generator and the MQL export use it. QuerySqlParser.extractTableName learns the getCollection form so the edited grid still resolves its collection. A test scans MongoScriptPrelude.source for every DB.prototype.<name> and asserts the accessor refuses each, so a method added to the shell later cannot reopen this.

MongoDB: a raw filter in shell syntax loses the row count

Root cause. A raw filter row such as {status: "active", _id: ObjectId("…")} is spliced into find(...), which evaluates it as JavaScript and works. countDocuments hands the same text to bson_new_from_json, which rejects it, and the coordinator swallows the error, so the count disappears while the rows show.

Fix. MongoDBRawFilterNormalizer evaluates __ejson((<raw>)) once in a JSContext loaded with the shell's own prelude, with the host stubbed so only the load-time currentDatabase call is answered. Both buildFilteredQuery and documentCount on the driver build through it, so find and countDocuments receive the same canonical Extended JSON. Text the shell cannot evaluate is kept as typed, which leaves find reporting the syntax error as before. The text is JavaScript and JavaScriptCore cannot interrupt it, so the engine runs on its own queue with a 2 second deadline and is abandoned when it misses it, the same shape MongoScriptRuntime uses.

Tests

  • FilterSQLGeneratorColumnTypeTests: casting on uuid, integer, enum, timestamp, array, unknown type, a dialect without a cast, Is empty, In list on uuid and on integers.
  • ColumnTypeSQLQuotingTests: isCharacterType over Postgres, MySQL, Oracle and ClickHouse spellings; arrays outside the text-like set.
  • SQLDialectDescriptorTests: default and carry-through of textCastTypeName.
  • MongoDBQueryBuilderTests: shadowed names through getCollection, parser round trip, prelude scan, raw filter normalization and its fallback.
  • MongoDBStatementGeneratorTests, MQLExportHelpersTests, QueryExecutorTests: accessor and parser expectations.

Verification

All through .claude/skills/fix-issue/scripts/verify.sh on Xcode 27.0 beta (27A5252f), signing disabled locally the way CI does.

  • generate: PASS.
  • build TablePro: PASS.
  • test over the 24 suites that own a changed type (FilterSQLGenerator*, ColumnTypeSQLQuoting, SQLDialectDescriptor, TableQueryBuilderFilter, QueryExecutor, ExtractTableName, MongoShellParser, MongoDB QueryBuilder / NestedFilter / StatementGenerator / WriteBackType, MQLExportHelpers, ClickHouseDialect, PluginMetadataRegistryVariant, ForeignKey*Query, FilterMove, FilterCaseSensitivityPersistence, DocumentStoreCaseSensitivity, SQLRowToStatementConverter, VisibleColumnProjection): PASS, 650 cases executed, 650 passed.
  • build MongoDBDriver (registry-only, so the app scheme never compiles it): PASS. plugins (AllPlugins) stops on DamengDriver for a missing Native/DamengBridge/lib/libdameng_bridge.a, an environment gap unrelated to this change; every other target compiled.
  • abi main: additive. MongoCollectionAccessor is new, textCastTypeName is a new field on a non-frozen struct, the previous full SQLDialectDescriptor initializer keeps its symbol and the new one is a separate overload. No PluginKit version bump.
  • lint on every changed file: the source files are clean. The five findings are on main already, in the import order and opening-brace whitespace of ColumnTypeSQLQuotingTests and FilterSQLGeneratorColumnTypeTests, which .swiftlint.yml never runs over.
  • The normalizer deadline was exercised once with {a: (function(){for(;;){}})()} in a scratch SwiftPM harness: nil after 2.0 s, and the next document serialized normally. That probe is not committed, because it leaves a spinning thread in the test host.

Not in this change

  • The iOS FilterSQLGenerator in Packages/TableProCore/Sources/TableProQuery has no column types, so it cannot decide when to cast and keeps the bare ILIKE. It needs the grid's types plumbed through first.
  • Redshift shares the PostgreSQL descriptor and casts to TEXT, which there is VARCHAR(256). A pattern filter on a value longer than 256 characters is truncated before matching, and a SUPER column still refuses the cast. Both already failed before this change.
  • MongoDBQueryBuilder accepts a raw filter on { and } alone, so {a:1}), evil(), ({b:2} runs statements in the normalizer. The same text already runs in the shell runtime with a live host bridge on the find path, so nothing is widened; the normalizer's context has no host.
  • A URL's condition parameter becomes a raw filter row through applyURLFilter, which on MongoDB reaches the shell runtime without SQLBoundaryValidator. Pre-existing and unrelated to the four fixes, flagged by the security pass.

https://claude.ai/code/session_019aSCJ4CVmua6eqveqtSRYx

@datlechin
datlechin merged commit 9184811 into TableProApp:main Sep 3, 2026
10 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants