Skip to content

Support common application usage: forked workers, shared connections, DATABASE_URL - #64

Merged
doron050 merged 6 commits into
mainfrom
devin/1793961247-real-world-usage
Nov 6, 2026
Merged

doron050 merged 6 commits into
mainfrom
devin/1793961247-real-world-usage

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

E2E tests (and, in follow-up commits on this PR, fixes) for ways Ruby applications commonly use the generated code that were not covered yet.

New scenarios (PostgreSQL / MySQL / SQLite where applicable):

  • forked workers (Puma / Sidekiq / Resque) after the parent ran queries, pooled and single connections
  • a raw connection shared with another library (ActiveRecord raw_connection, Sequel): PG type maps / field_name_type, mysql2 query_options and Mysql2::Client.default_query_options, SQLite results_as_hash
  • QuerySql.connect(DATABASE_URL) on PostgreSQL
  • #transaction inside a transaction opened by another library
  • schema-per-tenant search_path switching with prepared statements
  • primary + read-replica query objects
  • memoized frozen query objects, subclasses / prepended wrappers forwarding ... / **, SimpleDelegator
  • rows through Marshal (Rails.cache) and JSON

Failing on main (a44f282):

  • pg test_a_forked_child_leaves_the_parent_connections_usable (child's connection_pool after_fork closes the inherited connections and terminates the parent's sessions)
  • pg test_a_connection_shared_with_another_library_keeps_its_settings (generated code overwrites the connection's type maps; field_name_type = :symbol raises UndefinedEncoder)
  • pg test_connect_accepts_a_connection_url (TypeError)
  • mysql2 test_a_forked_child_leaves_the_parent_connections_usable (same as pg)
  • mysql2 test_a_client_with_other_query_options_still_returns_typed_rows (cast: false returns Strings, cast_booleans: true returns true/false for Integer columns)

Fixes (generated code, lib/sqlc_gen_ruby/drivers/{pg,mysql2}.rb):

  • pg: generated queries pass their own type maps (exec_prepared / exec_params with client.query_types, results get client.result_types and field_name_type = :string), so the connection's type_map_for_results / _for_queries / field_name_type are never changed. QuerySql.connect also takes a URL string.
  • pg / mysql2: ConnectionState remembers the pid that opened the connection; when connection_pool's after_fork closes it in a child, its socket is first pointed at /dev/null, so the parent's session survives. A pg connection used directly in a child reconnects; a direct mysql2 client raises Mysql2::Error::ConnectionError.
  • pg / mysql2: a Process._fork hook releases, in the child, every inherited connection the generated code has used (ObjectSpace.each_object(ConnectionState)), also ones another thread had checked out at fork time and ones the child never uses, so the child's exit does not end the parent's sessions. Closing an already closed pg connection is a no-op (connection_pool's after_fork closes those too).
  • mysql2: rows are read with as: :array, cast: true, cast_booleans: false whatever the client's / default query options.
  • Documented (README, Limitations): MySQL transaction inside a raw BEGIN commits it; SQLite children must reopen the database. docs/Coverage.md lists every new test.

Checklist

  • examples/ is regenerated (rake sqlc_generate_process) and committed
  • Tests added or updated (rake test, and rake end2end_* when the generated code changes)
  • The generated code stays RuboCop-clean (rake rubocop) and Steep-clean (rake steep)
  • Labelled (bug, enhancement, documentation, or breaking) so the release notes are grouped correctly

Requested by: @doron050

Forked workers (Puma/Sidekiq), connections shared with another library,
DATABASE_URL, transactions opened by another library, schema-per-tenant
search_path, read replicas, memoized/subclassed/wrapped query objects and
Marshal/JSON of rows.

Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot added the bug Something isn't working label Nov 6, 2026
devin-ai-integration Bot and others added 5 commits November 6, 2026 10:51
pg: queries pass their own type maps and read rows with string field names,
so a connection shared with another library keeps type_map_for_results /
_for_queries and field_name_type. QuerySql.connect takes a URL.
pg / mysql2: a connection inherited through fork is detached from the
parent's socket before connection_pool's after_fork closes it; a direct pg
connection reconnects in the child, a direct mysql2 client raises.
mysql2: rows are read with cast: true, cast_booleans: false whatever the
client's query options. Document the MySQL outer transaction limitation.

Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
connection_pool's after_fork closes every pooled connection in a forked
child, also ones the application closed before; PG::Connection#finish
raises for those, which broke the fork in the child.

Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
…r fork on macOS)

Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
connection_pool's after_fork only closes the idle pooled connections; one
another thread had checked out at fork time, or a direct connection the
child never used, was still the parent's socket in the child, and the
child's exit (PQfinish / COM_QUIT on finalization) ended the parent's
session. A Process._fork hook points all of them at /dev/null in the child.

Co-Authored-By: Doron Eli Rachman <doron050@gmail.com>
@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review November 6, 2026 11:47
@devin-ai-integration devin-ai-integration Bot added the enhancement New feature or request label Nov 6, 2026
@doron050
doron050 merged commit f8b9969 into main Nov 6, 2026
42 checks passed
@doron050
doron050 deleted the devin/1793961247-real-world-usage branch November 6, 2026 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant