diff --git a/README.md b/README.md index 0989d55..d8273de 100644 --- a/README.md +++ b/README.md @@ -156,6 +156,24 @@ end client that mysql2 closed because a running query was interrupted (an `Async` task stopped, `Thread#raise`) is replaced the same way, without failing the next call. * SQLite: threads and fibers sharing one `SQLite3::Database` take turns; a transaction holds the database until it ends. +* `pg`: `QuerySql.connect` also takes a connection URL: `QuerySql.connect(ENV.fetch('DATABASE_URL'), size: 5)`. +* A connection shared with another library (ActiveRecord's `raw_connection`, Sequel, your own code) keeps its + settings: the generated queries pass their own type maps (`pg`; the connection's `type_map_for_results`, + `type_map_for_queries` and `field_name_type` are left as they are) or query options (`mysql2`: rows are read with + `as: :array, cast: true, cast_booleans: false`, whatever the client's or `Mysql2::Client.default_query_options`). + On SQLite `results_as_hash` and the other settings of the `SQLite3::Database` are kept too. +* Forked processes (Puma / Sidekiq / Resque workers, `Process.fork`): connections opened before the fork stay the + parent's. connection_pool's fork handling replaces the pooled ones in the child without closing the parent's + sessions, and the child lets go of every inherited connection the generated code has used (also ones another + thread had checked out at fork time, and ones the child never uses), so its exit does not end them either. A `pg` + connection used directly (not from a `ConnectionPool`) reconnects in the child; a `mysql2` client + used that way raises `Mysql2::Error::ConnectionError` in the child (open clients in the process that uses them). + SQLite: the sqlite3 gem closes writable databases a child inherits, so open the database again in the child. + On macOS, connecting from a forked child can crash in libpq's GSSAPI probe; set `gssencmode: 'disable'` (or + `PGGSSENCMODE=disable`), as usual for forking `pg` apps there. +* A `transaction` inside a transaction that another library opened on the same connection (`BEGIN`) is a savepoint + on PostgreSQL and SQLite. MySQL has no nested transactions: its `BEGIN` commits the open transaction first (see + [Limitations](#limitations)). ## Supported features ### Query annotations @@ -247,6 +265,9 @@ server; duplicate keys and invalid values raise `Mysql2::Error` instead of being elements raises `SQLite3::SQLException: too many SQL variables`; on MySQL the limit is the server's `max_allowed_packet`. Load larger sets into a table (`:copyfrom`) and join it instead. * `:batchexec` / `:batchone` / `:batchmany` are not atomic on their own: wrap them in `transaction` for all-or-nothing. +* MySQL: `transaction` on a client inside a transaction that was started with raw SQL or by another library commits + that transaction first (MySQL's `BEGIN` does), so a later `ROLLBACK` no longer undoes it. Run the outer work in + `transaction` too (inner calls become savepoints), or finish the other transaction first. * SQLite: sqlc does not see parameters in `ON CONFLICT ... DO UPDATE SET` (it leaves `sqlc.arg(...)` in the SQL or reports fewer parameters than there are `?`), so such queries are generation errors. Use `excluded.` (`DO UPDATE SET n = counters.n + excluded.n`) or move the value into `VALUES`. diff --git a/docs/Coverage.md b/docs/Coverage.md index 775d5b1..397ebb7 100644 --- a/docs/Coverage.md +++ b/docs/Coverage.md @@ -35,6 +35,15 @@ only when the behavior is rejected or decided during generation and generated co | Top-level Float JSON parameters, Symbol enum values | [`test_top_level_float_json_parameters_round_trip_exactly`](../tests/end2end_pg.rb#test_top_level_float_json_parameters_round_trip_exactly) | [`test_symbol_enum_value_is_bound_as_its_name`](../tests/end2end_mysql2.rb#test_symbol_enum_value_is_bound_as_its_name) | not SQLite (no enums / JSON encoding) | | | Modules sharing a connection keep separate prepared statements | [`test_generator_options_preserve_runtime_values_and_generate_expected_layouts`](../tests/end2end_options.rb#test_generator_options_preserve_runtime_values_and_generate_expected_layouts) | not MySQL (no server-side prepared statements) | not SQLite (statements are per call) | [`test_statements_are_named_after_the_module_and_query_class`](../test/features_test.rb#test_statements_are_named_after_the_module_and_query_class) | | SQLite parameters sqlc does not report (`ON CONFLICT ... DO UPDATE SET`) are generation errors | not PostgreSQL | not MySQL | unit: rejected at generation | [`test_sqlite_parameters_sqlc_does_not_report_are_generation_errors`](../test/features_test.rb#test_sqlite_parameters_sqlc_does_not_report_are_generation_errors), [`test_sqlite_placeholders_in_literals_and_comments_are_not_parameters`](../test/features_test.rb#test_sqlite_placeholders_in_literals_and_comments_are_not_parameters) | +| Forked processes (Puma / Sidekiq workers): pooled connections are replaced in the child without ending the parent's sessions; `pg` direct connections reconnect, `mysql2` direct clients raise, SQLite children open their own database | [`test_a_forked_child_leaves_the_parent_connections_usable`](../tests/end2end_pg.rb#test_a_forked_child_leaves_the_parent_connections_usable) | [`test_a_forked_child_leaves_the_parent_connections_usable`](../tests/end2end_mysql2.rb#test_a_forked_child_leaves_the_parent_connections_usable) | [`test_a_forked_child_opens_its_own_database_and_leaves_the_parent_usable`](../tests/end2end_sqlite3.rb#test_a_forked_child_opens_its_own_database_and_leaves_the_parent_usable) | | +| A forked child's plain `exit` leaves the parent's sessions open: a pooled connection another thread has checked out at fork time, a direct connection the child never uses | [pg](../tests/end2end_pg.rb) `test_a_child_exit_leaves_connections_checked_out_or_unused_at_fork_open` · [mysql2](../tests/end2end_mysql2.rb) `test_a_child_exit_leaves_connections_checked_out_or_unused_at_fork_open` | +| A connection shared with another library keeps its settings (pg type maps / `field_name_type`, mysql2 query options and `default_query_options`, SQLite `results_as_hash`) | [`test_a_connection_shared_with_another_library_keeps_its_settings`](../tests/end2end_pg.rb#test_a_connection_shared_with_another_library_keeps_its_settings) | [`test_a_client_with_other_query_options_still_returns_typed_rows`](../tests/end2end_mysql2.rb#test_a_client_with_other_query_options_still_returns_typed_rows) | [`test_a_database_shared_with_another_library_keeps_its_settings`](../tests/end2end_sqlite3.rb#test_a_database_shared_with_another_library_keeps_its_settings) | | +| `pg`: `QuerySql.connect` takes a connection URL (`DATABASE_URL`) | [`test_connect_accepts_a_connection_url`](../tests/end2end_pg.rb#test_connect_accepts_a_connection_url) | not MySQL (mysql2 takes no URL) | not SQLite (no `connect`) | | +| `transaction` inside a transaction opened by another library: a savepoint (PostgreSQL, SQLite); MySQL limitation: commits it | [`test_transaction_inside_a_transaction_opened_by_another_library_is_a_savepoint`](../tests/end2end_pg.rb#test_transaction_inside_a_transaction_opened_by_another_library_is_a_savepoint) | [`test_transaction_inside_a_transaction_opened_by_another_library_commits_it`](../tests/end2end_mysql2.rb#test_transaction_inside_a_transaction_opened_by_another_library_commits_it) | [`test_transaction_inside_a_transaction_opened_by_another_library_is_a_savepoint`](../tests/end2end_sqlite3.rb#test_transaction_inside_a_transaction_opened_by_another_library_is_a_savepoint) | | +| Schema-per-tenant `search_path` switching with prepared statements | [`test_queries_follow_the_search_path_of_each_tenant`](../tests/end2end_pg.rb#test_queries_follow_the_search_path_of_each_tenant) | not MySQL (databases, no search path) | not SQLite (no schemas) | | +| Primary + read-replica query objects; a write to the read-only replica raises and leaves it usable | [`test_primary_and_read_replica_query_objects`](../tests/end2end_pg.rb#test_primary_and_read_replica_query_objects) | [`test_primary_and_read_replica_query_objects`](../tests/end2end_mysql2.rb#test_primary_and_read_replica_query_objects) | [`test_primary_and_read_replica_query_objects`](../tests/end2end_sqlite3.rb#test_primary_and_read_replica_query_objects) | | +| Memoized frozen query objects, subclasses and prepended wrappers forwarding `...` / `**`, `SimpleDelegator`, shared across threads | [`test_memoized_frozen_subclassed_and_wrapped_query_objects`](../tests/end2end_pg.rb#test_memoized_frozen_subclassed_and_wrapped_query_objects) | [`test_memoized_frozen_subclassed_and_wrapped_query_objects`](../tests/end2end_mysql2.rb#test_memoized_frozen_subclassed_and_wrapped_query_objects) | [`test_memoized_frozen_subclassed_and_wrapped_query_objects`](../tests/end2end_sqlite3.rb#test_memoized_frozen_subclassed_and_wrapped_query_objects) | | +| Rows through `Marshal` (Rails.cache) and JSON (`to_h.to_json`) | [`test_rows_round_trip_through_marshal_and_json`](../tests/end2end_pg.rb#test_rows_round_trip_through_marshal_and_json) | [`test_rows_round_trip_through_marshal_and_json`](../tests/end2end_mysql2.rb#test_rows_round_trip_through_marshal_and_json) | [`test_rows_round_trip_through_marshal_and_json`](../tests/end2end_sqlite3.rb#test_rows_round_trip_through_marshal_and_json) | | CI runs `test/coverage_map_test.rb`, which checks every linked test method exists in its referenced `tests/` or `test/` file. | Models named like Ruby / driver constants (`Exception`, `Data`, `Thread`, `Set`, `Monitor`, ...) do not shadow them | [`test_models_named_like_core_constants_do_not_shadow_them`](../tests/end2end_pg.rb#test_models_named_like_core_constants_do_not_shadow_them) | [`test_models_named_like_core_constants_do_not_shadow_them`](../tests/end2end_mysql2.rb#test_models_named_like_core_constants_do_not_shadow_them) | [`test_models_named_like_core_constants_do_not_shadow_them`](../tests/end2end_sqlite3.rb#test_models_named_like_core_constants_do_not_shadow_them) | | diff --git a/examples/mysql2/connection.rb b/examples/mysql2/connection.rb index 4d4b1ec..180e61e 100644 --- a/examples/mysql2/connection.rb +++ b/examples/mysql2/connection.rb @@ -3,8 +3,42 @@ # auto-generated by sqlc - do not edit module Mysql2Codegen - # State kept on every Mysql2::Client the first time it is used. + # State kept on every Mysql2::Client the first time it is used. Rows are read with their own query options + # (as: :array, cast: true, cast_booleans: false), whatever the client's or Mysql2::Client.default_query_options. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # connection_pool closes the connections a forked child inherits: point this process's copy of the socket + # at /dev/null first, so the QUIT sent by #close does not end the parent's session. + def close + release_parent_socket if inherited? + super + end + + def release_parent_socket + self.automatic_close = false + ::IO.for_fd(socket, autoclose: false).reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + attr_accessor :in_transaction # The error with which the server rolled back the whole transaction (a deadlock, or a lock wait timeout with # innodb_rollback_on_timeout): the transaction's later queries raise instead of running in autocommit mode. @@ -42,7 +76,7 @@ def ends_transaction?(error) if @rollback_on_timeout.nil? @rollback_on_timeout = begin - query('SELECT @@innodb_rollback_on_timeout', as: :array).first.first == 1 + query('SELECT @@innodb_rollback_on_timeout', as: :array, cast: true).first.first == 1 rescue ::Mysql2::Error false end diff --git a/examples/mysql2/connection.rbs b/examples/mysql2/connection.rbs index e720c06..05a11d7 100644 --- a/examples/mysql2/connection.rbs +++ b/examples/mysql2/connection.rbs @@ -7,6 +7,16 @@ module Mysql2Codegen def query_running?: () -> bool def rolled_back_error: () -> untyped def ends_transaction?: (untyped error) -> bool + def self.extended: (untyped client) -> void + def inherited?: () -> bool + def close: () -> void + def release_parent_socket: () -> void + def self.after_fork: () -> void + module ForkHook + def _fork: () -> ::Integer + end + def socket: () -> ::Integer + def automatic_close=: (bool) -> bool end class TransactionEnded < ::StandardError diff --git a/examples/mysql2/query_sql.rb b/examples/mysql2/query_sql.rb index 120ecdd..40f2bbf 100644 --- a/examples/mysql2/query_sql.rb +++ b/examples/mysql2/query_sql.rb @@ -636,7 +636,7 @@ class ExecResult < ::Data.define(:rows_affected, :last_insert_id) def get_author(id:) with_client('get_author') do |client| query_params = [id] - result = client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetAuthorRow.new(row[0], row[1], row[2]) end @@ -649,7 +649,7 @@ def get_author!(...) def list_authors with_client('list_authors') do |client| - result = client.query(bind_params(client, LIST_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsRow.new(row[0], row[1], row[2]) end @@ -718,7 +718,7 @@ def insert_mysql_numeric_types(id:, c_bool:, c_boolean:, c_tinyint:, c_smallint: def get_mysql_numeric_types(id:) with_client('get_mysql_numeric_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_NUMERIC_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_NUMERIC_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlNumericTypesRow.new( row[0], @@ -771,7 +771,7 @@ def insert_mysql_string_types(id:, c_char:, c_nchar:, c_national_char:, c_varcha def get_mysql_string_types(id:) with_client('get_mysql_string_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_STRING_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_STRING_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlStringTypesRow.new( row[0], @@ -812,7 +812,7 @@ def insert_mysql_datetime_types(id:, c_date:, c_time:, c_datetime:, c_datetime_6 def get_mysql_datetime_types(id:) with_client('get_mysql_datetime_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_DATETIME_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_DATETIME_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlDatetimeTypesRow.new( row[0], @@ -865,7 +865,7 @@ def insert_mysql_binary_types(id:, c_bit:, c_bit_16:, c_binary:, c_varbinary:, c def get_mysql_binary_types(id:) with_client('get_mysql_binary_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_BINARY_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_BINARY_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlBinaryTypesRow.new( row[0], @@ -889,7 +889,7 @@ def get_mysql_binary_types!(...) def list_authors_paged(limit:, offset:) with_client('list_authors_paged') do |client| query_params = [limit, offset] - result = client.query(bind_params(client, LIST_AUTHORS_PAGED_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_PAGED_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsPagedRow.new(row[0], row[1], row[2]) end @@ -899,7 +899,7 @@ def list_authors_paged(limit:, offset:) def get_author_by_name_pattern(name_pattern: nil) with_client('get_author_by_name_pattern') do |client| query_params = [name_pattern] - result = client.query(bind_params(client, GET_AUTHOR_BY_NAME_PATTERN_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_BY_NAME_PATTERN_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorByNamePatternRow.new(row[0], row[1], row[2]) end @@ -909,7 +909,7 @@ def get_author_by_name_pattern(name_pattern: nil) def get_authors_by_optional_name(limit:, name_filter: nil) with_client('get_authors_by_optional_name') do |client| query_params = [name_filter, name_filter, limit] - result = client.query(bind_params(client, GET_AUTHORS_BY_OPTIONAL_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHORS_BY_OPTIONAL_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByOptionalNameRow.new(row[0], row[1], row[2]) end @@ -919,7 +919,7 @@ def get_authors_by_optional_name(limit:, name_filter: nil) def pause_query(seconds:) with_client('pause_query') do |client| query_params = [seconds] - client.query(bind_params(client, PAUSE_QUERY_SQL, query_params), as: :array) + client.query(bind_params(client, PAUSE_QUERY_SQL, query_params), as: :array, cast: true, cast_booleans: false) nil end end @@ -937,7 +937,7 @@ def get_authors_by_ids(ids:) with_client('get_authors_by_ids') do |client| query_params = [*ids] sql = GET_AUTHORS_BY_IDS_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByIdsRow.new(row[0], row[1], row[2]) end @@ -949,7 +949,7 @@ def get_authors_by_ids_and_names(ids:, names:) query_params = [*ids, *names] sql = GET_AUTHORS_BY_IDS_AND_NAMES_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) sql = sql.gsub('/*SLICE:names*/?', slice_placeholders(names)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByIdsAndNamesRow.new(row[0], row[1], row[2]) end @@ -987,7 +987,7 @@ def create_book(name:, author_id:) def list_all_authors_books with_client('list_all_authors_books') do |client| - result = client.query(bind_params(client, LIST_ALL_AUTHORS_BOOKS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_ALL_AUTHORS_BOOKS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAllAuthorsBooksRow.new( Author.new(row[0], row[1], row[2]), @@ -999,7 +999,7 @@ def list_all_authors_books def get_duplicate_authors with_client('get_duplicate_authors') do |client| - result = client.query(bind_params(client, GET_DUPLICATE_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_DUPLICATE_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| GetDuplicateAuthorsRow.new(Author.new(row[0], row[1], row[2]), Author.new(row[3], row[4], row[5])) end @@ -1009,7 +1009,7 @@ def get_duplicate_authors def get_authors_by_book_name(name:) with_client('get_authors_by_book_name') do |client| query_params = [name] - result = client.query(bind_params(client, GET_AUTHORS_BY_BOOK_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHORS_BY_BOOK_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByBookNameRow.new(row[0], row[1], row[2], Book.new(row[3], row[4], row[5], row[6])) end @@ -1027,7 +1027,7 @@ def create_extended_bio(author_name:, name:, bio_type:, author_type:) def get_first_extended_bio_by_type(bio_type:) with_client('get_first_extended_bio_by_type') do |client| query_params = [bio_type] - result = client.query(bind_params(client, GET_FIRST_EXTENDED_BIO_BY_TYPE_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_FIRST_EXTENDED_BIO_BY_TYPE_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetFirstExtendedBioByTypeRow.new(row[0], row[1], row[2], row[3]) end @@ -1067,7 +1067,7 @@ def get_author_batch(rows) with_client('get_author_batch') do |client| rows.map do |args| query_params = [args.id] - result = client.query(bind_params(client, GET_AUTHOR_BATCH_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_BATCH_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetAuthorBatchRow.new(row[0], row[1], row[2]) end @@ -1078,7 +1078,7 @@ def list_authors_by_name_batch(rows) with_client('list_authors_by_name_batch') do |client| rows.map do |args| query_params = [args.name] - result = client.query(bind_params(client, LIST_AUTHORS_BY_NAME_BATCH_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_BY_NAME_BATCH_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsByNameBatchRow.new(row[0], row[1], row[2]) end @@ -1089,7 +1089,7 @@ def list_authors_by_name_batch(rows) def count_regressions_by_text(term:) with_client('count_regressions_by_text') do |client| query_params = [term, term] - result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_TEXT_SQL, query_params), as: :array) + result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_TEXT_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && CountRegressionsByTextRow.new(row[0]) end @@ -1103,7 +1103,7 @@ def count_regressions_by_text!(...) def list_regressions_by_ids(id:, id_2:, id_3:) with_client('list_regressions_by_ids') do |client| query_params = [id, id_2, id_3] - result = client.query(bind_params(client, LIST_REGRESSIONS_BY_IDS_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_REGRESSIONS_BY_IDS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListRegressionsByIdsRow.new(row[0], row[1], row[2]) end @@ -1147,7 +1147,7 @@ def set_regression_step(step:, id:) def get_regression_step(id:) with_client('get_regression_step') do |client| query_params = [id] - result = client.query(bind_params(client, GET_REGRESSION_STEP_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_REGRESSION_STEP_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetRegressionStepRow.new(row[0]) end @@ -1160,7 +1160,7 @@ def get_regression_step!(...) def list_regression_rows with_client('list_regression_rows') do |client| - result = client.query(bind_params(client, LIST_REGRESSION_ROWS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_REGRESSION_ROWS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListRegressionRowsRow.new( row[0], @@ -1178,7 +1178,7 @@ def list_regression_rows def list_authors_with_optional_book with_client('list_authors_with_optional_book') do |client| - result = client.query(bind_params(client, LIST_AUTHORS_WITH_OPTIONAL_BOOK_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_WITH_OPTIONAL_BOOK_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsWithOptionalBookRow.new( row[0].nil? && row[1].nil? ? nil : Author.new(row[0], row[1], row[2]), @@ -1248,7 +1248,7 @@ def count_regressions_by_codes(rows) with_client('count_regressions_by_codes') do |client| rows.map do |args| query_params = [args.code, args.code_2] - result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_CODES_SQL, query_params), as: :array) + result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_CODES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && CountRegressionsByCodesRow.new(row[0]) end @@ -1258,7 +1258,7 @@ def count_regressions_by_codes(rows) def find_authors_by_exact_name(name:) with_client('find_authors_by_exact_name') do |client| query_params = [name] - result = client.query(bind_params(client, FIND_AUTHORS_BY_EXACT_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, FIND_AUTHORS_BY_EXACT_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| FindAuthorsByExactNameRow.new(row[0]) end @@ -1269,7 +1269,7 @@ def find_authors_by_exact_name(name:) def get_author_with_question_marks(name:) with_client('get_author_with_question_marks') do |client| query_params = [name] - result = client.query(bind_params(client, GET_AUTHOR_WITH_QUESTION_MARKS_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_WITH_QUESTION_MARKS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorWithQuestionMarksRow.new(row[0]) end @@ -1307,7 +1307,7 @@ def get_name_case(id:, class_:, end_:, hash:, method:, count:, http_server:, cam camel_case, initialize ] - result = client.query(bind_params(client, GET_NAME_CASE_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_NAME_CASE_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetNameCaseRow.new( row[0], @@ -1331,7 +1331,7 @@ def get_name_case!(...) def get_http_server(id:) with_client('get_http_server') do |client| query_params = [id] - result = client.query(bind_params(client, GET_HTTP_SERVER_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_HTTP_SERVER_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetHTTPServerRow.new( row[0], @@ -1388,7 +1388,7 @@ def copy_mixed_case_copies_unquoted(rows) def list_mixed_case_copies with_client('list_mixed_case_copies') do |client| - result = client.query(bind_params(client, LIST_MIXED_CASE_COPIES_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_MIXED_CASE_COPIES_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListMixedCaseCopiesRow.new(row[0], row[1]) end @@ -1404,7 +1404,7 @@ def truncate_mixed_case_copies def get_unnamed_columns with_client('get_unnamed_columns') do |client| - result = client.query(bind_params(client, GET_UNNAMED_COLUMNS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_UNNAMED_COLUMNS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && GetUnnamedColumnsRow.new(row[0], row[1]) end @@ -1417,7 +1417,7 @@ def get_unnamed_columns!(...) def get_quoted_alias with_client('get_quoted_alias') do |client| - result = client.query(bind_params(client, GET_QUOTED_ALIAS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_QUOTED_ALIAS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && GetQuotedAliasRow.new(row[0]) end @@ -1432,7 +1432,7 @@ def get_authors_not_in_ids(ids:) with_client('get_authors_not_in_ids') do |client| query_params = [*ids] sql = GET_AUTHORS_NOT_IN_IDS_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsNotInIdsRow.new(row[0], row[1], row[2]) end @@ -1448,7 +1448,7 @@ def create_exception def count_exceptions with_client('count_exceptions') do |client| - result = client.query(bind_params(client, COUNT_EXCEPTIONS_SQL, []), as: :array) + result = client.query(bind_params(client, COUNT_EXCEPTIONS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && CountExceptionsRow.new(row[0]) end @@ -1469,7 +1469,7 @@ def truncate_exceptions # rubocop:disable Naming/PredicatePrefix def has_authors with_client('has_authors') do |client| - result = client.query(bind_params(client, HAS_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, HAS_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && HasAuthorsRow.new(row[0]) end @@ -1501,6 +1501,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/examples/pg/connection.rb b/examples/pg/connection.rb index 3a2a08e..dd35aa8 100644 --- a/examples/pg/connection.rb +++ b/examples/pg/connection.rb @@ -3,8 +3,65 @@ # auto-generated by sqlc - do not edit module PgCodegen - # State kept on every PG::Connection the first time it is used. + # State kept on every PG::Connection the first time it is used. The connection's own settings + # (type_map_for_results / _for_queries, field_name_type) are left to the application and other libraries + # sharing it: the generated queries pass their type maps with every call. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + attr_accessor :query_types, :result_types + + def typed(result) + result.type_map = result_types + result.field_name_type = :string + result + end + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # Points the inherited socket at /dev/null, so closing or resetting it here does not end the parent's session. + def detach_from_parent + release_parent_socket + ensure + @owner_pid = ::Process.pid + end + + def release_parent_socket + socket_io.reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + + # connection_pool closes the connections a forked child inherits, also ones closed before (which would raise). + def close + return if finished? + + detach_from_parent if inherited? + super + end + + def finish + return if finished? + + detach_from_parent if inherited? + super + end + # Names of the statements prepared on this connection; forgotten when the server session changes # (PG::Connection#reset after a failover / terminated backend). def prepared_statements @@ -104,6 +161,7 @@ def decoder(oid) @decoders.fetch(oid) do row = @client.exec("SELECT typcategory, typdelim FROM pg_type WHERE oid = #{Integer(oid)}").tap do |types| types.type_map = ::PG::TypeMapAllStrings.new + types.field_name_type = :string end.first decoder = row && NewTypes.decoder(oid, row['typcategory'], row['typdelim']) @known.add_coder(decoder) if decoder # later results no longer come here diff --git a/examples/pg/connection.rbs b/examples/pg/connection.rbs index f96bbee..03c7965 100644 --- a/examples/pg/connection.rbs +++ b/examples/pg/connection.rbs @@ -2,9 +2,20 @@ module PgCodegen interface _PgConnection def backend_pid: () -> ::Integer + + def socket_io: () -> ::IO + + def reset: () -> void + + def finished?: () -> bool + + def close: () -> void + + def finish: () -> void end module ConnectionState : _PgConnection + @owner_pid: ::Integer @prepared_statements: ::Set[::String] @prepared_statements_pid: ::Integer @stale_statements: ::Set[::String] @@ -14,6 +25,30 @@ module PgCodegen def stale_statements: () -> ::Set[::String] attr_accessor in_transaction: bool? + + def self.extended: (untyped client) -> void + + attr_accessor query_types: untyped + + attr_accessor result_types: untyped + + def typed: (untyped result) -> untyped + + def inherited?: () -> bool + + def detach_from_parent: () -> void + + def release_parent_socket: () -> void + + def self.after_fork: () -> void + + module ForkHook + def _fork: () -> ::Integer + end + + def close: () -> void + + def finish: () -> void end class TransactionEnded < ::StandardError diff --git a/examples/pg/query_sql.rb b/examples/pg/query_sql.rb index 495cf9a..27abd77 100644 --- a/examples/pg/query_sql.rb +++ b/examples/pg/query_sql.rb @@ -576,9 +576,11 @@ class QuerySql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -924,7 +926,7 @@ def insert_postgres_array_types_batch(rows) def copy_postgres_array_types(rows) with_client('copy_postgres_array_types') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "postgres_array_types" ("id", "c_boolean", "c_double_precision", "c_bytea_array", "c_jsonb_array", "c_integer_array", "c_numeric_array", "c_timestamptz_array") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.c_boolean, encode_float(row.c_double_precision), encode_array(row.c_bytea_array, :bytea), encode_array(row.c_jsonb_array, :json), encode_array(row.c_integer_array), encode_array(row.c_numeric_array), encode_array(row.c_timestamptz_array)]) @@ -1077,7 +1079,7 @@ def get_authors_by_ids_and_names(param1:, param2:) # Bulk insert: pass an Array of CopyAuthorsArgs. def copy_authors(rows) with_client('copy_authors') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "authors" ("name", "bio") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.name, row.bio]) @@ -1267,7 +1269,7 @@ def create_regression_if_new(id:, order:) def copy_regressions(rows) with_client('copy_regressions') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "regressions" ("id", "order", "payload", "doc") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.order, encode_bytea(row.payload), encode_json(row.doc)]) @@ -1397,7 +1399,7 @@ def get_http_server!(...) def copy_mixed_case_copies(rows) with_client('copy_mixed_case_copies') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "MixedCaseCopies" ("Id", "SomeValue") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.some_value]) @@ -1525,6 +1527,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -1556,14 +1564,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -1584,14 +1599,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/examples/pg/query_sql.rbs b/examples/pg/query_sql.rbs index 5ad0445..462ff48 100644 --- a/examples/pg/query_sql.rbs +++ b/examples/pg/query_sql.rbs @@ -595,7 +595,7 @@ module PgCodegen class NotFound < ::StandardError end - def self.connect: (::Hash[untyped, untyped] pg_params, ?logger: untyped, **untyped pool_options) -> instance + def self.connect: (::Hash[untyped, untyped] | ::String pg_params, ?logger: untyped, **untyped pool_options) -> instance def initialize: (untyped db, ?logger: untyped) -> void diff --git a/lib/sqlc_gen_ruby/drivers/mysql2.rb b/lib/sqlc_gen_ruby/drivers/mysql2.rb index a1c9c24..f61854f 100644 --- a/lib/sqlc_gen_ruby/drivers/mysql2.rb +++ b/lib/sqlc_gen_ruby/drivers/mysql2.rb @@ -13,7 +13,12 @@ def module_members [CONNECTION_STATE, "module ConnectionState\n attr_accessor in_transaction: bool\n" \ " attr_accessor rolled_back_by: untyped\n" \ " def query: (*untyped) -> untyped\n def query_running?: () -> bool\n" \ - " def rolled_back_error: () -> untyped\n def ends_transaction?: (untyped error) -> bool\nend"], + " def rolled_back_error: () -> untyped\n def ends_transaction?: (untyped error) -> bool\n" \ + " def self.extended: (untyped client) -> void\n def inherited?: () -> bool\n" \ + " def close: () -> void\n def release_parent_socket: () -> void\n" \ + " def self.after_fork: () -> void\n" \ + " module ForkHook\n def _fork: () -> ::Integer\n end\n" \ + " def socket: () -> ::Integer\n def automatic_close=: (bool) -> bool\nend"], [TRANSACTION_CONNECTION, TRANSACTION_CONNECTION_RBS], [ISOLATION_LEVELS, ISOLATION_RBS] ] @@ -47,7 +52,7 @@ def prepare_stmt(_stmt_name, _const) = nil def execute_stmt(_stmt_name, const, has_params, array_rows: false) sql = "bind_params(client, #{const}, #{has_params ? 'query_params' : '[]'})" sql = "client.query(#{sql}" - sql += ', as: :array' if array_rows + sql += ', as: :array, cast: true, cast_booleans: false' if array_rows "#{sql})" end @@ -135,8 +140,42 @@ def helper_methods(queries) end CONNECTION_STATE = <<~'RUBY'.chomp - # State kept on every Mysql2::Client the first time it is used. + # State kept on every Mysql2::Client the first time it is used. Rows are read with their own query options + # (as: :array, cast: true, cast_booleans: false), whatever the client's or Mysql2::Client.default_query_options. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # connection_pool closes the connections a forked child inherits: point this process's copy of the socket + # at /dev/null first, so the QUIT sent by #close does not end the parent's session. + def close + release_parent_socket if inherited? + super + end + + def release_parent_socket + self.automatic_close = false + ::IO.for_fd(socket, autoclose: false).reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + attr_accessor :in_transaction # The error with which the server rolled back the whole transaction (a deadlock, or a lock wait timeout with # innodb_rollback_on_timeout): the transaction's later queries raise instead of running in autocommit mode. @@ -174,7 +213,7 @@ def ends_transaction?(error) if @rollback_on_timeout.nil? @rollback_on_timeout = begin - query('SELECT @@innodb_rollback_on_timeout', as: :array).first.first == 1 + query('SELECT @@innodb_rollback_on_timeout', as: :array, cast: true).first.first == 1 rescue ::Mysql2::Error false end @@ -290,6 +329,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/lib/sqlc_gen_ruby/drivers/pg.rb b/lib/sqlc_gen_ruby/drivers/pg.rb index a575e3a..d6df7ee 100644 --- a/lib/sqlc_gen_ruby/drivers/pg.rb +++ b/lib/sqlc_gen_ruby/drivers/pg.rb @@ -36,7 +36,7 @@ def class_methods(ruby_version) end [ [NOT_FOUND, "class NotFound < ::StandardError\nend"], - [CONNECT, 'def self.connect: (::Hash[untyped, untyped] pg_params, ?logger: untyped, **untyped pool_options) -> instance'], + [CONNECT, 'def self.connect: (::Hash[untyped, untyped] | ::String pg_params, ?logger: untyped, **untyped pool_options) -> instance'], [INITIALIZE, 'def initialize: (untyped db, ?logger: untyped) -> void'], [transaction, TRANSACTION_RBS] ] @@ -65,7 +65,7 @@ def execute_stmt(stmt_name, const, has_params, array_rows: false) call = if @prepared "run_prepared(client, '#{stmt_name}', #{const}#{args})" else - "client.exec_params(#{const}#{has_params ? ', query_params' : ', []'})" + "client.typed(client.exec_params(#{const}, #{has_params ? 'query_params' : '[]'}, 0, client.query_types))" end array_rows ? "#{call}.values" : call end @@ -93,7 +93,7 @@ def copy_from_lines(rows_var, table, columns, values, _label, query_name:, sourc put = Code.block("#{rows_var}.each do |row|", "client.put_copy_data([#{values.join(', ')}])") [ - 'encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries)', + 'encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types)', Code.block("client.copy_data('COPY #{table} (#{columns.join(', ')}) FROM STDIN', encoder) do", put), 'nil' ] @@ -178,8 +178,65 @@ def array_elements: (::Array[untyped] values, ::Symbol kind) -> ::Array[untyped] }.freeze CONNECTION_STATE = <<~'RUBY'.chomp - # State kept on every PG::Connection the first time it is used. + # State kept on every PG::Connection the first time it is used. The connection's own settings + # (type_map_for_results / _for_queries, field_name_type) are left to the application and other libraries + # sharing it: the generated queries pass their type maps with every call. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + attr_accessor :query_types, :result_types + + def typed(result) + result.type_map = result_types + result.field_name_type = :string + result + end + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # Points the inherited socket at /dev/null, so closing or resetting it here does not end the parent's session. + def detach_from_parent + release_parent_socket + ensure + @owner_pid = ::Process.pid + end + + def release_parent_socket + socket_io.reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + + # connection_pool closes the connections a forked child inherits, also ones closed before (which would raise). + def close + return if finished? + + detach_from_parent if inherited? + super + end + + def finish + return if finished? + + detach_from_parent if inherited? + super + end + # Names of the statements prepared on this connection; forgotten when the server session changes # (PG::Connection#reset after a failover / terminated backend). def prepared_statements @@ -206,9 +263,20 @@ def stale_statements CONNECTION_STATE_RBS = <<~'RBS'.chomp interface _PgConnection def backend_pid: () -> ::Integer + + def socket_io: () -> ::IO + + def reset: () -> void + + def finished?: () -> bool + + def close: () -> void + + def finish: () -> void end module ConnectionState : _PgConnection + @owner_pid: ::Integer @prepared_statements: ::Set[::String] @prepared_statements_pid: ::Integer @stale_statements: ::Set[::String] @@ -218,6 +286,30 @@ def prepared_statements: () -> ::Set[::String] def stale_statements: () -> ::Set[::String] attr_accessor in_transaction: bool? + + def self.extended: (untyped client) -> void + + attr_accessor query_types: untyped + + attr_accessor result_types: untyped + + def typed: (untyped result) -> untyped + + def inherited?: () -> bool + + def detach_from_parent: () -> void + + def release_parent_socket: () -> void + + def self.after_fork: () -> void + + module ForkHook + def _fork: () -> ::Integer + end + + def close: () -> void + + def finish: () -> void end RBS @@ -255,6 +347,7 @@ def decoder(oid) @decoders.fetch(oid) do row = @client.exec("SELECT typcategory, typdelim FROM pg_type WHERE oid = #{Integer(oid)}").tap do |types| types.type_map = ::PG::TypeMapAllStrings.new + types.field_name_type = :string end.first decoder = row && NewTypes.decoder(oid, row['typcategory'], row['typdelim']) @known.add_coder(decoder) if decoder # later results no longer come here @@ -299,9 +392,11 @@ def initialize(db, logger: nil) CONNECT = <<~'RUBY'.chomp # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end RUBY @@ -386,14 +481,14 @@ def transaction(isolation: nil) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end @@ -425,6 +520,12 @@ def prepare(client, name, sql) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -455,14 +556,21 @@ def with_client(name) CONFIGURE = <<~'RUBY'.chomp # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. diff --git a/steep_expectations.yml b/steep_expectations.yml index 2415cca..38d42f5 100644 --- a/steep_expectations.yml +++ b/steep_expectations.yml @@ -3,20 +3,20 @@ diagnostics: - range: start: - line: 21 + line: 55 character: 13 end: - line: 21 + line: 55 character: 19 severity: WARNING message: 'Cannot find the declaration of constant: `Mysql2`' code: Ruby::UnknownConstant - range: start: - line: 46 + line: 80 character: 17 end: - line: 46 + line: 80 character: 23 severity: WARNING message: 'Cannot find the declaration of constant: `Mysql2`' @@ -105,10 +105,20 @@ code: Ruby::UnknownConstant - range: start: - line: 1587 + line: 1507 character: 20 end: - line: 1587 + line: 1507 + character: 26 + severity: WARNING + message: 'Cannot find the declaration of constant: `Mysql2`' + code: Ruby::UnknownConstant + - range: + start: + line: 1593 + character: 20 + end: + line: 1593 character: 38 severity: ERROR message: |- @@ -121,20 +131,20 @@ code: Ruby::ArgumentTypeMismatch - range: start: - line: 1588 + line: 1594 character: 78 end: - line: 1588 + line: 1594 character: 84 severity: ERROR message: Type `(::String | nil)` does not have method `match?` code: Ruby::NoMethod - range: start: - line: 1590 + line: 1596 character: 20 end: - line: 1590 + line: 1596 character: 39 severity: ERROR message: |- @@ -147,10 +157,10 @@ code: Ruby::ArgumentTypeMismatch - range: start: - line: 1594 + line: 1600 character: 20 end: - line: 1594 + line: 1600 character: 39 severity: ERROR message: |- @@ -163,10 +173,10 @@ code: Ruby::ArgumentTypeMismatch - range: start: - line: 1599 + line: 1605 character: 20 end: - line: 1599 + line: 1605 character: 39 severity: ERROR message: |- @@ -179,10 +189,10 @@ code: Ruby::ArgumentTypeMismatch - range: start: - line: 1608 + line: 1614 character: 20 end: - line: 1608 + line: 1614 character: 24 severity: ERROR message: |- @@ -195,10 +205,10 @@ code: Ruby::ArgumentTypeMismatch - range: start: - line: 1641 + line: 1647 character: 18 end: - line: 1641 + line: 1647 character: 24 severity: WARNING message: 'Cannot find the declaration of constant: `Mysql2`' @@ -207,60 +217,60 @@ diagnostics: - range: start: - line: 78 + line: 135 character: 58 end: - line: 78 + line: 135 character: 60 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 81 + line: 138 character: 15 end: - line: 81 + line: 138 character: 17 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 83 + line: 140 character: 8 end: - line: 83 + line: 140 character: 10 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 83 + line: 140 character: 84 end: - line: 83 + line: 140 character: 86 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 106 + line: 163 character: 29 end: - line: 106 + line: 163 character: 31 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 112 + line: 170 character: 13 end: - line: 112 + line: 170 character: 15 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' @@ -269,470 +279,500 @@ diagnostics: - range: start: - line: 581 - character: 12 + line: 582 + character: 51 end: - line: 581 - character: 26 + line: 582 + character: 53 severity: WARNING - message: 'Cannot find the declaration of constant: `ConnectionPool`' + message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 581 - character: 51 + line: 582 + character: 84 end: - line: 581 - character: 53 + line: 582 + character: 86 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 581 - character: 62 + line: 582 + character: 95 end: - line: 581 - character: 73 + line: 582 + character: 106 severity: WARNING message: Empty kwargs doesn't have type annotation code: Ruby::UnannotatedEmptyCollection - range: start: - line: 599 + line: 583 + character: 12 + end: + line: 583 + character: 26 + severity: WARNING + message: 'Cannot find the declaration of constant: `ConnectionPool`' + code: Ruby::UnknownConstant + - range: + start: + line: 601 character: 51 end: - line: 599 + line: 601 character: 53 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 612 + line: 614 character: 54 end: - line: 656 + line: 658 character: 11 severity: WARNING message: The method cannot be called with a block code: Ruby::UnexpectedBlockGiven - range: start: - line: 620 + line: 622 character: 62 end: - line: 622 + line: 624 character: 15 severity: WARNING message: The method cannot be called with a block code: Ruby::UnexpectedBlockGiven - range: start: - line: 626 + line: 628 character: 60 end: - line: 626 + line: 628 character: 62 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 628 + line: 630 character: 76 end: - line: 628 + line: 630 character: 78 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 629 + line: 631 character: 65 end: - line: 629 + line: 631 character: 67 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 636 + line: 638 character: 37 end: - line: 636 + line: 638 character: 39 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 638 + line: 640 character: 22 end: - line: 638 + line: 640 character: 24 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 641 + line: 643 character: 55 end: - line: 641 + line: 643 character: 57 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 647 + line: 649 character: 64 end: - line: 647 + line: 649 character: 66 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 648 + line: 650 character: 42 end: - line: 648 + line: 650 character: 44 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 648 + line: 650 character: 95 end: - line: 648 + line: 650 character: 97 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 653 + line: 655 character: 22 end: - line: 653 + line: 655 character: 24 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 927 + line: 929 + character: 20 + end: + line: 929 + character: 22 + severity: WARNING + message: 'Cannot find the declaration of constant: `PG`' + code: Ruby::UnknownConstant + - range: + start: + line: 1082 character: 20 end: - line: 927 + line: 1082 character: 22 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1080 + line: 1272 character: 20 end: - line: 1080 + line: 1272 character: 22 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1270 + line: 1402 character: 20 end: - line: 1270 + line: 1402 character: 22 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1400 + line: 1531 character: 20 end: - line: 1400 + line: 1531 character: 22 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1528 + line: 1536 character: 32 end: - line: 1528 + line: 1536 character: 34 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1530 + line: 1538 character: 20 end: - line: 1530 + line: 1538 character: 22 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1543 + line: 1551 character: 55 end: - line: 1543 + line: 1551 character: 57 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1544 + line: 1552 character: 60 end: - line: 1547 + line: 1555 character: 17 severity: WARNING message: The method cannot be called with a block code: Ruby::UnexpectedBlockGiven - range: start: - line: 1559 + line: 1570 + character: 38 + end: + line: 1570 + character: 40 + severity: WARNING + message: 'Cannot find the declaration of constant: `PG`' + code: Ruby::UnknownConstant + - range: + start: + line: 1571 character: 19 end: - line: 1559 + line: 1571 character: 21 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1562 + line: 1573 character: 18 end: - line: 1562 + line: 1573 character: 20 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1564 + line: 1575 character: 55 end: - line: 1564 + line: 1575 character: 57 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1571 + line: 1586 character: 18 end: - line: 1571 + line: 1586 character: 20 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1572 + line: 1587 character: 16 end: - line: 1572 + line: 1587 character: 18 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1573 + line: 1588 character: 68 end: - line: 1573 + line: 1588 character: 70 severity: WARNING message: Empty array doesn't have type annotation code: Ruby::UnannotatedEmptyCollection - range: start: - line: 1588 + line: 1603 character: 15 end: - line: 1588 + line: 1603 character: 17 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1588 + line: 1603 character: 46 end: - line: 1588 + line: 1603 character: 48 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1590 + line: 1605 character: 55 end: - line: 1590 + line: 1605 character: 57 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1591 + line: 1606 character: 52 end: - line: 1591 + line: 1606 character: 54 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1602 + line: 1617 character: 52 end: - line: 1615 + line: 1630 character: 9 severity: WARNING message: The method cannot be called with a block code: Ruby::UnexpectedBlockGiven - range: start: - line: 1606 + line: 1621 character: 19 end: - line: 1606 + line: 1621 character: 21 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1607 + line: 1622 character: 56 end: - line: 1607 + line: 1622 character: 58 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1612 + line: 1627 character: 15 end: - line: 1612 + line: 1627 character: 17 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1614 + line: 1629 character: 52 end: - line: 1614 + line: 1629 character: 54 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1620 + line: 1635 character: 17 end: - line: 1620 + line: 1635 character: 19 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1627 + line: 1642 character: 53 end: - line: 1627 + line: 1642 character: 55 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1654 + line: 1669 character: 8 end: - line: 1654 + line: 1669 character: 10 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1654 + line: 1669 character: 52 end: - line: 1654 + line: 1669 character: 54 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1661 + line: 1676 character: 44 end: - line: 1661 + line: 1676 character: 46 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1662 + line: 1677 character: 27 end: - line: 1662 + line: 1677 character: 29 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1663 + line: 1678 character: 46 end: - line: 1663 + line: 1678 character: 48 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' code: Ruby::UnknownConstant - range: start: - line: 1664 + line: 1679 character: 27 end: - line: 1664 + line: 1679 character: 29 severity: WARNING message: 'Cannot find the declaration of constant: `PG`' diff --git a/test/features_test.rb b/test/features_test.rb index b97a52b..900eec7 100644 --- a/test/features_test.rb +++ b/test/features_test.rb @@ -205,7 +205,7 @@ def test_mysql_copyfrom_writes_its_own_csv def test_mysql_uses_client_side_binding_and_sqlite_closes_prepared_statements mysql = generate(load_request('mysql2')).fetch('query_sql.rb') - assert_includes mysql, 'client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array)' + assert_includes mysql, 'client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array, cast: true, cast_booleans: false)' assert_includes mysql, 'def quote(client, value)' assert_includes mysql, 'def bind_params(client, sql, params)' refute_match(/\bstmt(?:\.| =)/, mysql) @@ -299,7 +299,7 @@ def test_columns_named_like_row_methods_still_build_and_read_rows def test_pg_can_skip_prepared_statements rb = generate(load_request('pg', preparedStatements: false)).fetch('query_sql.rb') - assert_includes rb, 'client.exec_params(GET_AUTHOR_SQL, query_params)' + assert_includes rb, 'client.typed(client.exec_params(GET_AUTHOR_SQL, query_params, 0, client.query_types))' refute_includes rb, 'client.prepare(' end diff --git a/test/golden/mysql2/default/connection.rb b/test/golden/mysql2/default/connection.rb index 4d4b1ec..180e61e 100644 --- a/test/golden/mysql2/default/connection.rb +++ b/test/golden/mysql2/default/connection.rb @@ -3,8 +3,42 @@ # auto-generated by sqlc - do not edit module Mysql2Codegen - # State kept on every Mysql2::Client the first time it is used. + # State kept on every Mysql2::Client the first time it is used. Rows are read with their own query options + # (as: :array, cast: true, cast_booleans: false), whatever the client's or Mysql2::Client.default_query_options. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # connection_pool closes the connections a forked child inherits: point this process's copy of the socket + # at /dev/null first, so the QUIT sent by #close does not end the parent's session. + def close + release_parent_socket if inherited? + super + end + + def release_parent_socket + self.automatic_close = false + ::IO.for_fd(socket, autoclose: false).reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + attr_accessor :in_transaction # The error with which the server rolled back the whole transaction (a deadlock, or a lock wait timeout with # innodb_rollback_on_timeout): the transaction's later queries raise instead of running in autocommit mode. @@ -42,7 +76,7 @@ def ends_transaction?(error) if @rollback_on_timeout.nil? @rollback_on_timeout = begin - query('SELECT @@innodb_rollback_on_timeout', as: :array).first.first == 1 + query('SELECT @@innodb_rollback_on_timeout', as: :array, cast: true).first.first == 1 rescue ::Mysql2::Error false end diff --git a/test/golden/mysql2/default/connection.rbs b/test/golden/mysql2/default/connection.rbs index e720c06..05a11d7 100644 --- a/test/golden/mysql2/default/connection.rbs +++ b/test/golden/mysql2/default/connection.rbs @@ -7,6 +7,16 @@ module Mysql2Codegen def query_running?: () -> bool def rolled_back_error: () -> untyped def ends_transaction?: (untyped error) -> bool + def self.extended: (untyped client) -> void + def inherited?: () -> bool + def close: () -> void + def release_parent_socket: () -> void + def self.after_fork: () -> void + module ForkHook + def _fork: () -> ::Integer + end + def socket: () -> ::Integer + def automatic_close=: (bool) -> bool end class TransactionEnded < ::StandardError diff --git a/test/golden/mysql2/default/my_queries_sql.rb b/test/golden/mysql2/default/my_queries_sql.rb index f7a6548..cfa778b 100644 --- a/test/golden/mysql2/default/my_queries_sql.rb +++ b/test/golden/mysql2/default/my_queries_sql.rb @@ -139,6 +139,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/mysql2/default/other_sql.rb b/test/golden/mysql2/default/other_sql.rb index acdff09..564f501 100644 --- a/test/golden/mysql2/default/other_sql.rb +++ b/test/golden/mysql2/default/other_sql.rb @@ -142,7 +142,7 @@ def no_cols(c0:, c1:, c2:, c3:, c4:) def other_one with_client('other_one') do |client| - result = client.query(bind_params(client, OTHER_ONE_SQL, []), as: :array) + result = client.query(bind_params(client, OTHER_ONE_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && OtherOneRow.new(row[0]) end @@ -179,6 +179,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/mysql2/default/query_sql.rb b/test/golden/mysql2/default/query_sql.rb index 1a260a2..7447e03 100644 --- a/test/golden/mysql2/default/query_sql.rb +++ b/test/golden/mysql2/default/query_sql.rb @@ -666,7 +666,7 @@ class ExecResult < ::Data.define(:rows_affected, :last_insert_id) def get_author(id:) with_client('get_author') do |client| query_params = [id] - result = client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetAuthorRow.new(row[0], row[1], row[2]) end @@ -679,7 +679,7 @@ def get_author!(...) def list_authors with_client('list_authors') do |client| - result = client.query(bind_params(client, LIST_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsRow.new(row[0], row[1], row[2]) end @@ -748,7 +748,7 @@ def insert_mysql_numeric_types(id:, c_bool:, c_boolean:, c_tinyint:, c_smallint: def get_mysql_numeric_types(id:) with_client('get_mysql_numeric_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_NUMERIC_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_NUMERIC_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlNumericTypesRow.new( row[0], @@ -801,7 +801,7 @@ def insert_mysql_string_types(id:, c_char:, c_nchar:, c_national_char:, c_varcha def get_mysql_string_types(id:) with_client('get_mysql_string_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_STRING_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_STRING_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlStringTypesRow.new( row[0], @@ -842,7 +842,7 @@ def insert_mysql_datetime_types(id:, c_date:, c_time:, c_datetime:, c_datetime_6 def get_mysql_datetime_types(id:) with_client('get_mysql_datetime_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_DATETIME_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_DATETIME_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlDatetimeTypesRow.new( row[0], @@ -895,7 +895,7 @@ def insert_mysql_binary_types(id:, c_bit:, c_bit_16:, c_binary:, c_varbinary:, c def get_mysql_binary_types(id:) with_client('get_mysql_binary_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_BINARY_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_BINARY_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlBinaryTypesRow.new( row[0], @@ -919,7 +919,7 @@ def get_mysql_binary_types!(...) def list_authors_paged(limit:, offset:) with_client('list_authors_paged') do |client| query_params = [limit, offset] - result = client.query(bind_params(client, LIST_AUTHORS_PAGED_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_PAGED_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsPagedRow.new(row[0], row[1], row[2]) end @@ -929,7 +929,7 @@ def list_authors_paged(limit:, offset:) def get_author_by_name_pattern(name_pattern: nil) with_client('get_author_by_name_pattern') do |client| query_params = [name_pattern] - result = client.query(bind_params(client, GET_AUTHOR_BY_NAME_PATTERN_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_BY_NAME_PATTERN_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorByNamePatternRow.new(row[0], row[1], row[2]) end @@ -939,7 +939,7 @@ def get_author_by_name_pattern(name_pattern: nil) def get_authors_by_optional_name(limit:, name_filter: nil) with_client('get_authors_by_optional_name') do |client| query_params = [name_filter, name_filter, limit] - result = client.query(bind_params(client, GET_AUTHORS_BY_OPTIONAL_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHORS_BY_OPTIONAL_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByOptionalNameRow.new(row[0], row[1], row[2]) end @@ -949,7 +949,7 @@ def get_authors_by_optional_name(limit:, name_filter: nil) def pause_query(seconds:) with_client('pause_query') do |client| query_params = [seconds] - client.query(bind_params(client, PAUSE_QUERY_SQL, query_params), as: :array) + client.query(bind_params(client, PAUSE_QUERY_SQL, query_params), as: :array, cast: true, cast_booleans: false) nil end end @@ -967,7 +967,7 @@ def get_authors_by_ids(ids:) with_client('get_authors_by_ids') do |client| query_params = [*ids] sql = GET_AUTHORS_BY_IDS_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByIdsRow.new(row[0], row[1], row[2]) end @@ -979,7 +979,7 @@ def get_authors_by_ids_and_names(ids:, names:) query_params = [*ids, *names] sql = GET_AUTHORS_BY_IDS_AND_NAMES_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) sql = sql.gsub('/*SLICE:names*/?', slice_placeholders(names)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByIdsAndNamesRow.new(row[0], row[1], row[2]) end @@ -1017,7 +1017,7 @@ def create_book(name:, author_id:) def list_all_authors_books with_client('list_all_authors_books') do |client| - result = client.query(bind_params(client, LIST_ALL_AUTHORS_BOOKS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_ALL_AUTHORS_BOOKS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAllAuthorsBooksRow.new( Author.new(row[0], row[1], row[2]), @@ -1029,7 +1029,7 @@ def list_all_authors_books def get_duplicate_authors with_client('get_duplicate_authors') do |client| - result = client.query(bind_params(client, GET_DUPLICATE_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_DUPLICATE_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| GetDuplicateAuthorsRow.new(Author.new(row[0], row[1], row[2]), Author.new(row[3], row[4], row[5])) end @@ -1039,7 +1039,7 @@ def get_duplicate_authors def get_authors_by_book_name(name:) with_client('get_authors_by_book_name') do |client| query_params = [name] - result = client.query(bind_params(client, GET_AUTHORS_BY_BOOK_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHORS_BY_BOOK_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByBookNameRow.new(row[0], row[1], row[2], Book.new(row[3], row[4], row[5], row[6])) end @@ -1057,7 +1057,7 @@ def create_extended_bio(author_name:, name:, bio_type:, author_type:) def get_first_extended_bio_by_type(bio_type:) with_client('get_first_extended_bio_by_type') do |client| query_params = [bio_type] - result = client.query(bind_params(client, GET_FIRST_EXTENDED_BIO_BY_TYPE_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_FIRST_EXTENDED_BIO_BY_TYPE_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetFirstExtendedBioByTypeRow.new(row[0], row[1], row[2], row[3]) end @@ -1097,7 +1097,7 @@ def get_author_batch(rows) with_client('get_author_batch') do |client| rows.map do |args| query_params = [args.id] - result = client.query(bind_params(client, GET_AUTHOR_BATCH_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_BATCH_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetAuthorBatchRow.new(row[0], row[1], row[2]) end @@ -1108,7 +1108,7 @@ def list_authors_by_name_batch(rows) with_client('list_authors_by_name_batch') do |client| rows.map do |args| query_params = [args.name] - result = client.query(bind_params(client, LIST_AUTHORS_BY_NAME_BATCH_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_BY_NAME_BATCH_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsByNameBatchRow.new(row[0], row[1], row[2]) end @@ -1119,7 +1119,7 @@ def list_authors_by_name_batch(rows) def count_regressions_by_text(term:) with_client('count_regressions_by_text') do |client| query_params = [term, term] - result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_TEXT_SQL, query_params), as: :array) + result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_TEXT_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && CountRegressionsByTextRow.new(row[0]) end @@ -1133,7 +1133,7 @@ def count_regressions_by_text!(...) def list_regressions_by_ids(id:, id_2:, id_3:) with_client('list_regressions_by_ids') do |client| query_params = [id, id_2, id_3] - result = client.query(bind_params(client, LIST_REGRESSIONS_BY_IDS_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_REGRESSIONS_BY_IDS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListRegressionsByIdsRow.new(row[0], row[1], row[2]) end @@ -1177,7 +1177,7 @@ def set_regression_step(step:, id:) def get_regression_step(id:) with_client('get_regression_step') do |client| query_params = [id] - result = client.query(bind_params(client, GET_REGRESSION_STEP_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_REGRESSION_STEP_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetRegressionStepRow.new(row[0]) end @@ -1190,7 +1190,7 @@ def get_regression_step!(...) def list_regression_rows with_client('list_regression_rows') do |client| - result = client.query(bind_params(client, LIST_REGRESSION_ROWS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_REGRESSION_ROWS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListRegressionRowsRow.new( row[0], @@ -1208,7 +1208,7 @@ def list_regression_rows def list_authors_with_optional_book with_client('list_authors_with_optional_book') do |client| - result = client.query(bind_params(client, LIST_AUTHORS_WITH_OPTIONAL_BOOK_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_WITH_OPTIONAL_BOOK_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsWithOptionalBookRow.new( row[0].nil? && row[1].nil? ? nil : Author.new(row[0], row[1], row[2]), @@ -1278,7 +1278,7 @@ def count_regressions_by_codes(rows) with_client('count_regressions_by_codes') do |client| rows.map do |args| query_params = [args.code, args.code_2] - result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_CODES_SQL, query_params), as: :array) + result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_CODES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && CountRegressionsByCodesRow.new(row[0]) end @@ -1288,7 +1288,7 @@ def count_regressions_by_codes(rows) def find_authors_by_exact_name(name:) with_client('find_authors_by_exact_name') do |client| query_params = [name] - result = client.query(bind_params(client, FIND_AUTHORS_BY_EXACT_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, FIND_AUTHORS_BY_EXACT_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| FindAuthorsByExactNameRow.new(row[0]) end @@ -1299,7 +1299,7 @@ def find_authors_by_exact_name(name:) def get_author_with_question_marks(name:) with_client('get_author_with_question_marks') do |client| query_params = [name] - result = client.query(bind_params(client, GET_AUTHOR_WITH_QUESTION_MARKS_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_WITH_QUESTION_MARKS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorWithQuestionMarksRow.new(row[0]) end @@ -1337,7 +1337,7 @@ def get_name_case(id:, class_:, end_:, hash:, method:, count:, http_server:, cam camel_case, initialize ] - result = client.query(bind_params(client, GET_NAME_CASE_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_NAME_CASE_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetNameCaseRow.new( row[0], @@ -1361,7 +1361,7 @@ def get_name_case!(...) def get_http_server(id:) with_client('get_http_server') do |client| query_params = [id] - result = client.query(bind_params(client, GET_HTTP_SERVER_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_HTTP_SERVER_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetHTTPServerRow.new( row[0], @@ -1418,7 +1418,7 @@ def copy_mixed_case_copies_unquoted(rows) def list_mixed_case_copies with_client('list_mixed_case_copies') do |client| - result = client.query(bind_params(client, LIST_MIXED_CASE_COPIES_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_MIXED_CASE_COPIES_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListMixedCaseCopiesRow.new(row[0], row[1]) end @@ -1434,7 +1434,7 @@ def truncate_mixed_case_copies def get_unnamed_columns with_client('get_unnamed_columns') do |client| - result = client.query(bind_params(client, GET_UNNAMED_COLUMNS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_UNNAMED_COLUMNS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && GetUnnamedColumnsRow.new(row[0], row[1]) end @@ -1447,7 +1447,7 @@ def get_unnamed_columns!(...) def get_quoted_alias with_client('get_quoted_alias') do |client| - result = client.query(bind_params(client, GET_QUOTED_ALIAS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_QUOTED_ALIAS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && GetQuotedAliasRow.new(row[0]) end @@ -1462,7 +1462,7 @@ def get_authors_not_in_ids(ids:) with_client('get_authors_not_in_ids') do |client| query_params = [*ids] sql = GET_AUTHORS_NOT_IN_IDS_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsNotInIdsRow.new(row[0], row[1], row[2]) end @@ -1478,7 +1478,7 @@ def create_exception def count_exceptions with_client('count_exceptions') do |client| - result = client.query(bind_params(client, COUNT_EXCEPTIONS_SQL, []), as: :array) + result = client.query(bind_params(client, COUNT_EXCEPTIONS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && CountExceptionsRow.new(row[0]) end @@ -1499,7 +1499,7 @@ def truncate_exceptions # rubocop:disable Naming/PredicatePrefix def has_authors with_client('has_authors') do |client| - result = client.query(bind_params(client, HAS_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, HAS_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && HasAuthorsRow.new(row[0]) end @@ -1522,7 +1522,7 @@ def many_params(c0:, c1:, c2:, c3:, c4:) c3, c4 ] - result = client.query(bind_params(client, MANY_PARAMS_SQL, query_params), as: :array) + result = client.query(bind_params(client, MANY_PARAMS_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && ManyParamsRow.new(row[0], row[1]) end @@ -1536,7 +1536,7 @@ def many_params!(...) def many_rows(c0:) with_client('many_rows') do |client| query_params = [c0] - result = client.query(bind_params(client, MANY_ROWS_SQL, query_params), as: :array) + result = client.query(bind_params(client, MANY_ROWS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ManyRowsRow.new( row[0], @@ -1551,7 +1551,7 @@ def many_rows(c0:) def four_cols with_client('four_cols') do |client| - result = client.query(bind_params(client, FOUR_COLS_SQL, []), as: :array) + result = client.query(bind_params(client, FOUR_COLS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| FourColsRow.new(row[0], row[1], row[2], row[3]) end @@ -1576,6 +1576,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/mysql2/ruby31_gemfile/connection.rb b/test/golden/mysql2/ruby31_gemfile/connection.rb index 4d4b1ec..180e61e 100644 --- a/test/golden/mysql2/ruby31_gemfile/connection.rb +++ b/test/golden/mysql2/ruby31_gemfile/connection.rb @@ -3,8 +3,42 @@ # auto-generated by sqlc - do not edit module Mysql2Codegen - # State kept on every Mysql2::Client the first time it is used. + # State kept on every Mysql2::Client the first time it is used. Rows are read with their own query options + # (as: :array, cast: true, cast_booleans: false), whatever the client's or Mysql2::Client.default_query_options. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # connection_pool closes the connections a forked child inherits: point this process's copy of the socket + # at /dev/null first, so the QUIT sent by #close does not end the parent's session. + def close + release_parent_socket if inherited? + super + end + + def release_parent_socket + self.automatic_close = false + ::IO.for_fd(socket, autoclose: false).reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + attr_accessor :in_transaction # The error with which the server rolled back the whole transaction (a deadlock, or a lock wait timeout with # innodb_rollback_on_timeout): the transaction's later queries raise instead of running in autocommit mode. @@ -42,7 +76,7 @@ def ends_transaction?(error) if @rollback_on_timeout.nil? @rollback_on_timeout = begin - query('SELECT @@innodb_rollback_on_timeout', as: :array).first.first == 1 + query('SELECT @@innodb_rollback_on_timeout', as: :array, cast: true).first.first == 1 rescue ::Mysql2::Error false end diff --git a/test/golden/mysql2/ruby31_gemfile/connection.rbs b/test/golden/mysql2/ruby31_gemfile/connection.rbs index e720c06..05a11d7 100644 --- a/test/golden/mysql2/ruby31_gemfile/connection.rbs +++ b/test/golden/mysql2/ruby31_gemfile/connection.rbs @@ -7,6 +7,16 @@ module Mysql2Codegen def query_running?: () -> bool def rolled_back_error: () -> untyped def ends_transaction?: (untyped error) -> bool + def self.extended: (untyped client) -> void + def inherited?: () -> bool + def close: () -> void + def release_parent_socket: () -> void + def self.after_fork: () -> void + module ForkHook + def _fork: () -> ::Integer + end + def socket: () -> ::Integer + def automatic_close=: (bool) -> bool end class TransactionEnded < ::StandardError diff --git a/test/golden/mysql2/ruby31_gemfile/my_queries_sql.rb b/test/golden/mysql2/ruby31_gemfile/my_queries_sql.rb index 3053a1b..0bac704 100644 --- a/test/golden/mysql2/ruby31_gemfile/my_queries_sql.rb +++ b/test/golden/mysql2/ruby31_gemfile/my_queries_sql.rb @@ -142,6 +142,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/mysql2/ruby31_gemfile/other_sql.rb b/test/golden/mysql2/ruby31_gemfile/other_sql.rb index c79b4f5..d3da397 100644 --- a/test/golden/mysql2/ruby31_gemfile/other_sql.rb +++ b/test/golden/mysql2/ruby31_gemfile/other_sql.rb @@ -144,7 +144,7 @@ def no_cols(c0:, c1:, c2:, c3:, c4:) def other_one with_client('other_one') do |client| - result = client.query(bind_params(client, OTHER_ONE_SQL, []), as: :array) + result = client.query(bind_params(client, OTHER_ONE_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && OtherOneRow.new(c0: row[0]) end @@ -181,6 +181,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/mysql2/ruby31_gemfile/query_sql.rb b/test/golden/mysql2/ruby31_gemfile/query_sql.rb index 1b68c09..9e99d6d 100644 --- a/test/golden/mysql2/ruby31_gemfile/query_sql.rb +++ b/test/golden/mysql2/ruby31_gemfile/query_sql.rb @@ -656,7 +656,7 @@ def transaction(isolation: nil) def get_author(id:) with_client('get_author') do |client| query_params = [id] - result = client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetAuthorRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -669,7 +669,7 @@ def get_author!(...) def list_authors with_client('list_authors') do |client| - result = client.query(bind_params(client, LIST_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -738,7 +738,7 @@ def insert_mysql_numeric_types(id:, c_bool:, c_boolean:, c_tinyint:, c_smallint: def get_mysql_numeric_types(id:) with_client('get_mysql_numeric_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_NUMERIC_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_NUMERIC_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlNumericTypesRow.new( id: row[0], @@ -791,7 +791,7 @@ def insert_mysql_string_types(id:, c_char:, c_nchar:, c_national_char:, c_varcha def get_mysql_string_types(id:) with_client('get_mysql_string_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_STRING_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_STRING_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlStringTypesRow.new( id: row[0], @@ -833,7 +833,7 @@ def insert_mysql_datetime_types(id:, c_date:, c_time:, c_datetime:, c_datetime_6 def get_mysql_datetime_types(id:) with_client('get_mysql_datetime_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_DATETIME_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_DATETIME_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlDatetimeTypesRow.new( id: row[0], @@ -888,7 +888,7 @@ def insert_mysql_binary_types(id:, c_bit:, c_bit_16:, c_binary:, c_varbinary:, c def get_mysql_binary_types(id:) with_client('get_mysql_binary_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_BINARY_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_BINARY_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlBinaryTypesRow.new( id: row[0], @@ -913,7 +913,7 @@ def get_mysql_binary_types!(...) def list_authors_paged(limit:, offset:) with_client('list_authors_paged') do |client| query_params = [limit, offset] - result = client.query(bind_params(client, LIST_AUTHORS_PAGED_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_PAGED_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsPagedRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -923,7 +923,7 @@ def list_authors_paged(limit:, offset:) def get_author_by_name_pattern(name_pattern: nil) with_client('get_author_by_name_pattern') do |client| query_params = [name_pattern] - result = client.query(bind_params(client, GET_AUTHOR_BY_NAME_PATTERN_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_BY_NAME_PATTERN_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorByNamePatternRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -933,7 +933,7 @@ def get_author_by_name_pattern(name_pattern: nil) def get_authors_by_optional_name(limit:, name_filter: nil) with_client('get_authors_by_optional_name') do |client| query_params = [name_filter, name_filter, limit] - result = client.query(bind_params(client, GET_AUTHORS_BY_OPTIONAL_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHORS_BY_OPTIONAL_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByOptionalNameRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -943,7 +943,7 @@ def get_authors_by_optional_name(limit:, name_filter: nil) def pause_query(seconds:) with_client('pause_query') do |client| query_params = [seconds] - client.query(bind_params(client, PAUSE_QUERY_SQL, query_params), as: :array) + client.query(bind_params(client, PAUSE_QUERY_SQL, query_params), as: :array, cast: true, cast_booleans: false) nil end end @@ -961,7 +961,7 @@ def get_authors_by_ids(ids:) with_client('get_authors_by_ids') do |client| query_params = [*ids] sql = GET_AUTHORS_BY_IDS_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByIdsRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -973,7 +973,7 @@ def get_authors_by_ids_and_names(ids:, names:) query_params = [*ids, *names] sql = GET_AUTHORS_BY_IDS_AND_NAMES_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) sql = sql.gsub('/*SLICE:names*/?', slice_placeholders(names)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByIdsAndNamesRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -1011,7 +1011,7 @@ def create_book(name:, author_id:) def list_all_authors_books with_client('list_all_authors_books') do |client| - result = client.query(bind_params(client, LIST_ALL_AUTHORS_BOOKS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_ALL_AUTHORS_BOOKS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAllAuthorsBooksRow.new( author: Author.new(id: row[0], name: row[1], bio: row[2]), @@ -1023,7 +1023,7 @@ def list_all_authors_books def get_duplicate_authors with_client('get_duplicate_authors') do |client| - result = client.query(bind_params(client, GET_DUPLICATE_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_DUPLICATE_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| GetDuplicateAuthorsRow.new( author: Author.new(id: row[0], name: row[1], bio: row[2]), @@ -1036,7 +1036,7 @@ def get_duplicate_authors def get_authors_by_book_name(name:) with_client('get_authors_by_book_name') do |client| query_params = [name] - result = client.query(bind_params(client, GET_AUTHORS_BY_BOOK_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHORS_BY_BOOK_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByBookNameRow.new( id: row[0], @@ -1059,7 +1059,7 @@ def create_extended_bio(author_name:, name:, bio_type:, author_type:) def get_first_extended_bio_by_type(bio_type:) with_client('get_first_extended_bio_by_type') do |client| query_params = [bio_type] - result = client.query(bind_params(client, GET_FIRST_EXTENDED_BIO_BY_TYPE_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_FIRST_EXTENDED_BIO_BY_TYPE_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetFirstExtendedBioByTypeRow.new( author_name: row[0], @@ -1104,7 +1104,7 @@ def get_author_batch(rows) with_client('get_author_batch') do |client| rows.map do |args| query_params = [args.id] - result = client.query(bind_params(client, GET_AUTHOR_BATCH_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_BATCH_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetAuthorBatchRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -1115,7 +1115,7 @@ def list_authors_by_name_batch(rows) with_client('list_authors_by_name_batch') do |client| rows.map do |args| query_params = [args.name] - result = client.query(bind_params(client, LIST_AUTHORS_BY_NAME_BATCH_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_BY_NAME_BATCH_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsByNameBatchRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -1126,7 +1126,7 @@ def list_authors_by_name_batch(rows) def count_regressions_by_text(term:) with_client('count_regressions_by_text') do |client| query_params = [term, term] - result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_TEXT_SQL, query_params), as: :array) + result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_TEXT_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && CountRegressionsByTextRow.new(count_: row[0]) end @@ -1140,7 +1140,7 @@ def count_regressions_by_text!(...) def list_regressions_by_ids(id:, id_2:, id_3:) with_client('list_regressions_by_ids') do |client| query_params = [id, id_2, id_3] - result = client.query(bind_params(client, LIST_REGRESSIONS_BY_IDS_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_REGRESSIONS_BY_IDS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListRegressionsByIdsRow.new(id: row[0], order: row[1], note: row[2]) end @@ -1184,7 +1184,7 @@ def set_regression_step(step:, id:) def get_regression_step(id:) with_client('get_regression_step') do |client| query_params = [id] - result = client.query(bind_params(client, GET_REGRESSION_STEP_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_REGRESSION_STEP_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetRegressionStepRow.new(step: row[0]) end @@ -1197,7 +1197,7 @@ def get_regression_step!(...) def list_regression_rows with_client('list_regression_rows') do |client| - result = client.query(bind_params(client, LIST_REGRESSION_ROWS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_REGRESSION_ROWS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListRegressionRowsRow.new( id: row[0], @@ -1215,7 +1215,7 @@ def list_regression_rows def list_authors_with_optional_book with_client('list_authors_with_optional_book') do |client| - result = client.query(bind_params(client, LIST_AUTHORS_WITH_OPTIONAL_BOOK_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_WITH_OPTIONAL_BOOK_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsWithOptionalBookRow.new( author: row[0].nil? && row[1].nil? ? nil : Author.new(id: row[0], name: row[1], bio: row[2]), @@ -1285,7 +1285,7 @@ def count_regressions_by_codes(rows) with_client('count_regressions_by_codes') do |client| rows.map do |args| query_params = [args.code, args.code_2] - result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_CODES_SQL, query_params), as: :array) + result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_CODES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && CountRegressionsByCodesRow.new(matches: row[0]) end @@ -1295,7 +1295,7 @@ def count_regressions_by_codes(rows) def find_authors_by_exact_name(name:) with_client('find_authors_by_exact_name') do |client| query_params = [name] - result = client.query(bind_params(client, FIND_AUTHORS_BY_EXACT_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, FIND_AUTHORS_BY_EXACT_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| FindAuthorsByExactNameRow.new(id: row[0]) end @@ -1306,7 +1306,7 @@ def find_authors_by_exact_name(name:) def get_author_with_question_marks(name:) with_client('get_author_with_question_marks') do |client| query_params = [name] - result = client.query(bind_params(client, GET_AUTHOR_WITH_QUESTION_MARKS_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_WITH_QUESTION_MARKS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorWithQuestionMarksRow.new(id: row[0]) end @@ -1344,7 +1344,7 @@ def get_name_case(id:, class_:, end_:, hash:, method:, count:, http_server:, cam camel_case, initialize ] - result = client.query(bind_params(client, GET_NAME_CASE_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_NAME_CASE_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetNameCaseRow.new( id: row[0], @@ -1368,7 +1368,7 @@ def get_name_case!(...) def get_http_server(id:) with_client('get_http_server') do |client| query_params = [id] - result = client.query(bind_params(client, GET_HTTP_SERVER_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_HTTP_SERVER_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetHTTPServerRow.new( id: row[0], @@ -1425,7 +1425,7 @@ def copy_mixed_case_copies_unquoted(rows) def list_mixed_case_copies with_client('list_mixed_case_copies') do |client| - result = client.query(bind_params(client, LIST_MIXED_CASE_COPIES_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_MIXED_CASE_COPIES_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListMixedCaseCopiesRow.new(id: row[0], somevalue: row[1]) end @@ -1442,7 +1442,7 @@ def truncate_mixed_case_copies # rubocop:disable Naming/VariableNumber def get_unnamed_columns with_client('get_unnamed_columns') do |client| - result = client.query(bind_params(client, GET_UNNAMED_COLUMNS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_UNNAMED_COLUMNS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && GetUnnamedColumnsRow.new(column_1: row[0], column_2: row[1]) end @@ -1456,7 +1456,7 @@ def get_unnamed_columns!(...) def get_quoted_alias with_client('get_quoted_alias') do |client| - result = client.query(bind_params(client, GET_QUOTED_ALIAS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_QUOTED_ALIAS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && GetQuotedAliasRow.new(it_s: row[0]) end @@ -1471,7 +1471,7 @@ def get_authors_not_in_ids(ids:) with_client('get_authors_not_in_ids') do |client| query_params = [*ids] sql = GET_AUTHORS_NOT_IN_IDS_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsNotInIdsRow.new(id: row[0], name: row[1], bio: row[2]) end @@ -1487,7 +1487,7 @@ def create_exception def count_exceptions with_client('count_exceptions') do |client| - result = client.query(bind_params(client, COUNT_EXCEPTIONS_SQL, []), as: :array) + result = client.query(bind_params(client, COUNT_EXCEPTIONS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && CountExceptionsRow.new(n: row[0]) end @@ -1508,7 +1508,7 @@ def truncate_exceptions # rubocop:disable Naming/PredicatePrefix def has_authors with_client('has_authors') do |client| - result = client.query(bind_params(client, HAS_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, HAS_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && HasAuthorsRow.new(has_any: row[0]) end @@ -1531,7 +1531,7 @@ def many_params(c0:, c1:, c2:, c3:, c4:) c3, c4 ] - result = client.query(bind_params(client, MANY_PARAMS_SQL, query_params), as: :array) + result = client.query(bind_params(client, MANY_PARAMS_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && ManyParamsRow.new(c0: row[0], c1: row[1]) end @@ -1545,7 +1545,7 @@ def many_params!(...) def many_rows(c0:) with_client('many_rows') do |client| query_params = [c0] - result = client.query(bind_params(client, MANY_ROWS_SQL, query_params), as: :array) + result = client.query(bind_params(client, MANY_ROWS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ManyRowsRow.new( c0: row[0], @@ -1560,7 +1560,7 @@ def many_rows(c0:) def four_cols with_client('four_cols') do |client| - result = client.query(bind_params(client, FOUR_COLS_SQL, []), as: :array) + result = client.query(bind_params(client, FOUR_COLS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| FourColsRow.new(c0: row[0], c1: row[1], c2: row[2], c3: row[3]) end @@ -1585,6 +1585,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/mysql2/ruby32_notypes/connection.rb b/test/golden/mysql2/ruby32_notypes/connection.rb index 4d4b1ec..180e61e 100644 --- a/test/golden/mysql2/ruby32_notypes/connection.rb +++ b/test/golden/mysql2/ruby32_notypes/connection.rb @@ -3,8 +3,42 @@ # auto-generated by sqlc - do not edit module Mysql2Codegen - # State kept on every Mysql2::Client the first time it is used. + # State kept on every Mysql2::Client the first time it is used. Rows are read with their own query options + # (as: :array, cast: true, cast_booleans: false), whatever the client's or Mysql2::Client.default_query_options. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # connection_pool closes the connections a forked child inherits: point this process's copy of the socket + # at /dev/null first, so the QUIT sent by #close does not end the parent's session. + def close + release_parent_socket if inherited? + super + end + + def release_parent_socket + self.automatic_close = false + ::IO.for_fd(socket, autoclose: false).reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + attr_accessor :in_transaction # The error with which the server rolled back the whole transaction (a deadlock, or a lock wait timeout with # innodb_rollback_on_timeout): the transaction's later queries raise instead of running in autocommit mode. @@ -42,7 +76,7 @@ def ends_transaction?(error) if @rollback_on_timeout.nil? @rollback_on_timeout = begin - query('SELECT @@innodb_rollback_on_timeout', as: :array).first.first == 1 + query('SELECT @@innodb_rollback_on_timeout', as: :array, cast: true).first.first == 1 rescue ::Mysql2::Error false end diff --git a/test/golden/mysql2/ruby32_notypes/my_queries_sql.rb b/test/golden/mysql2/ruby32_notypes/my_queries_sql.rb index 3053a1b..0bac704 100644 --- a/test/golden/mysql2/ruby32_notypes/my_queries_sql.rb +++ b/test/golden/mysql2/ruby32_notypes/my_queries_sql.rb @@ -142,6 +142,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/mysql2/ruby32_notypes/other_sql.rb b/test/golden/mysql2/ruby32_notypes/other_sql.rb index dbfbec3..3f625fb 100644 --- a/test/golden/mysql2/ruby32_notypes/other_sql.rb +++ b/test/golden/mysql2/ruby32_notypes/other_sql.rb @@ -145,7 +145,7 @@ def no_cols(c0:, c1:, c2:, c3:, c4:) def other_one with_client('other_one') do |client| - result = client.query(bind_params(client, OTHER_ONE_SQL, []), as: :array) + result = client.query(bind_params(client, OTHER_ONE_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && OtherOneRow.new(row[0]) end @@ -182,6 +182,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/mysql2/ruby32_notypes/query_sql.rb b/test/golden/mysql2/ruby32_notypes/query_sql.rb index 20e66f1..7fa09cf 100644 --- a/test/golden/mysql2/ruby32_notypes/query_sql.rb +++ b/test/golden/mysql2/ruby32_notypes/query_sql.rb @@ -669,7 +669,7 @@ class ExecResult < ::Data.define(:rows_affected, :last_insert_id) def get_author(id:) with_client('get_author') do |client| query_params = [id] - result = client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetAuthorRow.new(row[0], row[1], row[2]) end @@ -682,7 +682,7 @@ def get_author!(...) def list_authors with_client('list_authors') do |client| - result = client.query(bind_params(client, LIST_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsRow.new(row[0], row[1], row[2]) end @@ -751,7 +751,7 @@ def insert_mysql_numeric_types(id:, c_bool:, c_boolean:, c_tinyint:, c_smallint: def get_mysql_numeric_types(id:) with_client('get_mysql_numeric_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_NUMERIC_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_NUMERIC_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlNumericTypesRow.new( row[0], @@ -804,7 +804,7 @@ def insert_mysql_string_types(id:, c_char:, c_nchar:, c_national_char:, c_varcha def get_mysql_string_types(id:) with_client('get_mysql_string_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_STRING_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_STRING_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlStringTypesRow.new( row[0], @@ -845,7 +845,7 @@ def insert_mysql_datetime_types(id:, c_date:, c_time:, c_datetime:, c_datetime_6 def get_mysql_datetime_types(id:) with_client('get_mysql_datetime_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_DATETIME_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_DATETIME_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlDatetimeTypesRow.new( row[0], @@ -898,7 +898,7 @@ def insert_mysql_binary_types(id:, c_bit:, c_bit_16:, c_binary:, c_varbinary:, c def get_mysql_binary_types(id:) with_client('get_mysql_binary_types') do |client| query_params = [id] - result = client.query(bind_params(client, GET_MYSQL_BINARY_TYPES_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_MYSQL_BINARY_TYPES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetMysqlBinaryTypesRow.new( row[0], @@ -922,7 +922,7 @@ def get_mysql_binary_types!(...) def list_authors_paged(limit:, offset:) with_client('list_authors_paged') do |client| query_params = [limit, offset] - result = client.query(bind_params(client, LIST_AUTHORS_PAGED_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_PAGED_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsPagedRow.new(row[0], row[1], row[2]) end @@ -932,7 +932,7 @@ def list_authors_paged(limit:, offset:) def get_author_by_name_pattern(name_pattern: nil) with_client('get_author_by_name_pattern') do |client| query_params = [name_pattern] - result = client.query(bind_params(client, GET_AUTHOR_BY_NAME_PATTERN_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_BY_NAME_PATTERN_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorByNamePatternRow.new(row[0], row[1], row[2]) end @@ -942,7 +942,7 @@ def get_author_by_name_pattern(name_pattern: nil) def get_authors_by_optional_name(limit:, name_filter: nil) with_client('get_authors_by_optional_name') do |client| query_params = [name_filter, name_filter, limit] - result = client.query(bind_params(client, GET_AUTHORS_BY_OPTIONAL_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHORS_BY_OPTIONAL_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByOptionalNameRow.new(row[0], row[1], row[2]) end @@ -952,7 +952,7 @@ def get_authors_by_optional_name(limit:, name_filter: nil) def pause_query(seconds:) with_client('pause_query') do |client| query_params = [seconds] - client.query(bind_params(client, PAUSE_QUERY_SQL, query_params), as: :array) + client.query(bind_params(client, PAUSE_QUERY_SQL, query_params), as: :array, cast: true, cast_booleans: false) nil end end @@ -970,7 +970,7 @@ def get_authors_by_ids(ids:) with_client('get_authors_by_ids') do |client| query_params = [*ids] sql = GET_AUTHORS_BY_IDS_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByIdsRow.new(row[0], row[1], row[2]) end @@ -982,7 +982,7 @@ def get_authors_by_ids_and_names(ids:, names:) query_params = [*ids, *names] sql = GET_AUTHORS_BY_IDS_AND_NAMES_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) sql = sql.gsub('/*SLICE:names*/?', slice_placeholders(names)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByIdsAndNamesRow.new(row[0], row[1], row[2]) end @@ -1020,7 +1020,7 @@ def create_book(name:, author_id:) def list_all_authors_books with_client('list_all_authors_books') do |client| - result = client.query(bind_params(client, LIST_ALL_AUTHORS_BOOKS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_ALL_AUTHORS_BOOKS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAllAuthorsBooksRow.new( Author.new(row[0], row[1], row[2]), @@ -1032,7 +1032,7 @@ def list_all_authors_books def get_duplicate_authors with_client('get_duplicate_authors') do |client| - result = client.query(bind_params(client, GET_DUPLICATE_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_DUPLICATE_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| GetDuplicateAuthorsRow.new(Author.new(row[0], row[1], row[2]), Author.new(row[3], row[4], row[5])) end @@ -1042,7 +1042,7 @@ def get_duplicate_authors def get_authors_by_book_name(name:) with_client('get_authors_by_book_name') do |client| query_params = [name] - result = client.query(bind_params(client, GET_AUTHORS_BY_BOOK_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHORS_BY_BOOK_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsByBookNameRow.new(row[0], row[1], row[2], Book.new(row[3], row[4], row[5], row[6])) end @@ -1060,7 +1060,7 @@ def create_extended_bio(author_name:, name:, bio_type:, author_type:) def get_first_extended_bio_by_type(bio_type:) with_client('get_first_extended_bio_by_type') do |client| query_params = [bio_type] - result = client.query(bind_params(client, GET_FIRST_EXTENDED_BIO_BY_TYPE_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_FIRST_EXTENDED_BIO_BY_TYPE_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetFirstExtendedBioByTypeRow.new(row[0], row[1], row[2], row[3]) end @@ -1100,7 +1100,7 @@ def get_author_batch(rows) with_client('get_author_batch') do |client| rows.map do |args| query_params = [args.id] - result = client.query(bind_params(client, GET_AUTHOR_BATCH_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_BATCH_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetAuthorBatchRow.new(row[0], row[1], row[2]) end @@ -1111,7 +1111,7 @@ def list_authors_by_name_batch(rows) with_client('list_authors_by_name_batch') do |client| rows.map do |args| query_params = [args.name] - result = client.query(bind_params(client, LIST_AUTHORS_BY_NAME_BATCH_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_BY_NAME_BATCH_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsByNameBatchRow.new(row[0], row[1], row[2]) end @@ -1122,7 +1122,7 @@ def list_authors_by_name_batch(rows) def count_regressions_by_text(term:) with_client('count_regressions_by_text') do |client| query_params = [term, term] - result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_TEXT_SQL, query_params), as: :array) + result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_TEXT_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && CountRegressionsByTextRow.new(row[0]) end @@ -1136,7 +1136,7 @@ def count_regressions_by_text!(...) def list_regressions_by_ids(id:, id_2:, id_3:) with_client('list_regressions_by_ids') do |client| query_params = [id, id_2, id_3] - result = client.query(bind_params(client, LIST_REGRESSIONS_BY_IDS_SQL, query_params), as: :array) + result = client.query(bind_params(client, LIST_REGRESSIONS_BY_IDS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ListRegressionsByIdsRow.new(row[0], row[1], row[2]) end @@ -1180,7 +1180,7 @@ def set_regression_step(step:, id:) def get_regression_step(id:) with_client('get_regression_step') do |client| query_params = [id] - result = client.query(bind_params(client, GET_REGRESSION_STEP_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_REGRESSION_STEP_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetRegressionStepRow.new(row[0]) end @@ -1193,7 +1193,7 @@ def get_regression_step!(...) def list_regression_rows with_client('list_regression_rows') do |client| - result = client.query(bind_params(client, LIST_REGRESSION_ROWS_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_REGRESSION_ROWS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListRegressionRowsRow.new( row[0], @@ -1211,7 +1211,7 @@ def list_regression_rows def list_authors_with_optional_book with_client('list_authors_with_optional_book') do |client| - result = client.query(bind_params(client, LIST_AUTHORS_WITH_OPTIONAL_BOOK_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_AUTHORS_WITH_OPTIONAL_BOOK_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListAuthorsWithOptionalBookRow.new( row[0].nil? && row[1].nil? ? nil : Author.new(row[0], row[1], row[2]), @@ -1281,7 +1281,7 @@ def count_regressions_by_codes(rows) with_client('count_regressions_by_codes') do |client| rows.map do |args| query_params = [args.code, args.code_2] - result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_CODES_SQL, query_params), as: :array) + result = client.query(bind_params(client, COUNT_REGRESSIONS_BY_CODES_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && CountRegressionsByCodesRow.new(row[0]) end @@ -1291,7 +1291,7 @@ def count_regressions_by_codes(rows) def find_authors_by_exact_name(name:) with_client('find_authors_by_exact_name') do |client| query_params = [name] - result = client.query(bind_params(client, FIND_AUTHORS_BY_EXACT_NAME_SQL, query_params), as: :array) + result = client.query(bind_params(client, FIND_AUTHORS_BY_EXACT_NAME_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| FindAuthorsByExactNameRow.new(row[0]) end @@ -1302,7 +1302,7 @@ def find_authors_by_exact_name(name:) def get_author_with_question_marks(name:) with_client('get_author_with_question_marks') do |client| query_params = [name] - result = client.query(bind_params(client, GET_AUTHOR_WITH_QUESTION_MARKS_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_AUTHOR_WITH_QUESTION_MARKS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorWithQuestionMarksRow.new(row[0]) end @@ -1340,7 +1340,7 @@ def get_name_case(id:, class_:, end_:, hash:, method:, count:, http_server:, cam camel_case, initialize ] - result = client.query(bind_params(client, GET_NAME_CASE_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_NAME_CASE_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetNameCaseRow.new( row[0], @@ -1364,7 +1364,7 @@ def get_name_case!(...) def get_http_server(id:) with_client('get_http_server') do |client| query_params = [id] - result = client.query(bind_params(client, GET_HTTP_SERVER_SQL, query_params), as: :array) + result = client.query(bind_params(client, GET_HTTP_SERVER_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && GetHTTPServerRow.new( row[0], @@ -1421,7 +1421,7 @@ def copy_mixed_case_copies_unquoted(rows) def list_mixed_case_copies with_client('list_mixed_case_copies') do |client| - result = client.query(bind_params(client, LIST_MIXED_CASE_COPIES_SQL, []), as: :array) + result = client.query(bind_params(client, LIST_MIXED_CASE_COPIES_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| ListMixedCaseCopiesRow.new(row[0], row[1]) end @@ -1437,7 +1437,7 @@ def truncate_mixed_case_copies def get_unnamed_columns with_client('get_unnamed_columns') do |client| - result = client.query(bind_params(client, GET_UNNAMED_COLUMNS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_UNNAMED_COLUMNS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && GetUnnamedColumnsRow.new(row[0], row[1]) end @@ -1450,7 +1450,7 @@ def get_unnamed_columns!(...) def get_quoted_alias with_client('get_quoted_alias') do |client| - result = client.query(bind_params(client, GET_QUOTED_ALIAS_SQL, []), as: :array) + result = client.query(bind_params(client, GET_QUOTED_ALIAS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && GetQuotedAliasRow.new(row[0]) end @@ -1465,7 +1465,7 @@ def get_authors_not_in_ids(ids:) with_client('get_authors_not_in_ids') do |client| query_params = [*ids] sql = GET_AUTHORS_NOT_IN_IDS_SQL.gsub('/*SLICE:ids*/?', slice_placeholders(ids)) - result = client.query(bind_params(client, sql, query_params), as: :array) + result = client.query(bind_params(client, sql, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| GetAuthorsNotInIdsRow.new(row[0], row[1], row[2]) end @@ -1481,7 +1481,7 @@ def create_exception def count_exceptions with_client('count_exceptions') do |client| - result = client.query(bind_params(client, COUNT_EXCEPTIONS_SQL, []), as: :array) + result = client.query(bind_params(client, COUNT_EXCEPTIONS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && CountExceptionsRow.new(row[0]) end @@ -1502,7 +1502,7 @@ def truncate_exceptions # rubocop:disable Naming/PredicatePrefix def has_authors with_client('has_authors') do |client| - result = client.query(bind_params(client, HAS_AUTHORS_SQL, []), as: :array) + result = client.query(bind_params(client, HAS_AUTHORS_SQL, []), as: :array, cast: true, cast_booleans: false) row = result.first row && HasAuthorsRow.new(row[0]) end @@ -1525,7 +1525,7 @@ def many_params(c0:, c1:, c2:, c3:, c4:) c3, c4 ] - result = client.query(bind_params(client, MANY_PARAMS_SQL, query_params), as: :array) + result = client.query(bind_params(client, MANY_PARAMS_SQL, query_params), as: :array, cast: true, cast_booleans: false) row = result.first row && ManyParamsRow.new(row[0], row[1]) end @@ -1539,7 +1539,7 @@ def many_params!(...) def many_rows(c0:) with_client('many_rows') do |client| query_params = [c0] - result = client.query(bind_params(client, MANY_ROWS_SQL, query_params), as: :array) + result = client.query(bind_params(client, MANY_ROWS_SQL, query_params), as: :array, cast: true, cast_booleans: false) result.map do |row| ManyRowsRow.new( row[0], @@ -1554,7 +1554,7 @@ def many_rows(c0:) def four_cols with_client('four_cols') do |client| - result = client.query(bind_params(client, FOUR_COLS_SQL, []), as: :array) + result = client.query(bind_params(client, FOUR_COLS_SQL, []), as: :array, cast: true, cast_booleans: false) result.map do |row| FourColsRow.new(row[0], row[1], row[2], row[3]) end @@ -1579,6 +1579,12 @@ def with_client(name) instrument(name) do @db.with do |client| client.extend(ConnectionState) unless client.is_a?(ConnectionState) + if client.inherited? + client.close + @db.discard_current_connection if @db.respond_to?(:discard_current_connection) + raise ::Mysql2::Error::ConnectionError, 'this Mysql2::Client was opened by another process (inherited through fork): ' \ + 'open clients in the process that uses them, or use a ConnectionPool' + end begin yield client rescue ::Exception # rubocop:disable Lint/RescueException diff --git a/test/golden/pg/default/connection.rb b/test/golden/pg/default/connection.rb index 3a2a08e..dd35aa8 100644 --- a/test/golden/pg/default/connection.rb +++ b/test/golden/pg/default/connection.rb @@ -3,8 +3,65 @@ # auto-generated by sqlc - do not edit module PgCodegen - # State kept on every PG::Connection the first time it is used. + # State kept on every PG::Connection the first time it is used. The connection's own settings + # (type_map_for_results / _for_queries, field_name_type) are left to the application and other libraries + # sharing it: the generated queries pass their type maps with every call. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + attr_accessor :query_types, :result_types + + def typed(result) + result.type_map = result_types + result.field_name_type = :string + result + end + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # Points the inherited socket at /dev/null, so closing or resetting it here does not end the parent's session. + def detach_from_parent + release_parent_socket + ensure + @owner_pid = ::Process.pid + end + + def release_parent_socket + socket_io.reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + + # connection_pool closes the connections a forked child inherits, also ones closed before (which would raise). + def close + return if finished? + + detach_from_parent if inherited? + super + end + + def finish + return if finished? + + detach_from_parent if inherited? + super + end + # Names of the statements prepared on this connection; forgotten when the server session changes # (PG::Connection#reset after a failover / terminated backend). def prepared_statements @@ -104,6 +161,7 @@ def decoder(oid) @decoders.fetch(oid) do row = @client.exec("SELECT typcategory, typdelim FROM pg_type WHERE oid = #{Integer(oid)}").tap do |types| types.type_map = ::PG::TypeMapAllStrings.new + types.field_name_type = :string end.first decoder = row && NewTypes.decoder(oid, row['typcategory'], row['typdelim']) @known.add_coder(decoder) if decoder # later results no longer come here diff --git a/test/golden/pg/default/connection.rbs b/test/golden/pg/default/connection.rbs index f96bbee..03c7965 100644 --- a/test/golden/pg/default/connection.rbs +++ b/test/golden/pg/default/connection.rbs @@ -2,9 +2,20 @@ module PgCodegen interface _PgConnection def backend_pid: () -> ::Integer + + def socket_io: () -> ::IO + + def reset: () -> void + + def finished?: () -> bool + + def close: () -> void + + def finish: () -> void end module ConnectionState : _PgConnection + @owner_pid: ::Integer @prepared_statements: ::Set[::String] @prepared_statements_pid: ::Integer @stale_statements: ::Set[::String] @@ -14,6 +25,30 @@ module PgCodegen def stale_statements: () -> ::Set[::String] attr_accessor in_transaction: bool? + + def self.extended: (untyped client) -> void + + attr_accessor query_types: untyped + + attr_accessor result_types: untyped + + def typed: (untyped result) -> untyped + + def inherited?: () -> bool + + def detach_from_parent: () -> void + + def release_parent_socket: () -> void + + def self.after_fork: () -> void + + module ForkHook + def _fork: () -> ::Integer + end + + def close: () -> void + + def finish: () -> void end class TransactionEnded < ::StandardError diff --git a/test/golden/pg/default/my_queries_sql.rb b/test/golden/pg/default/my_queries_sql.rb index b5f9c11..9826f1e 100644 --- a/test/golden/pg/default/my_queries_sql.rb +++ b/test/golden/pg/default/my_queries_sql.rb @@ -17,9 +17,11 @@ class MyQueriesSql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -123,6 +125,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -154,14 +162,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -182,14 +197,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/test/golden/pg/default/my_queries_sql.rbs b/test/golden/pg/default/my_queries_sql.rbs index d6bdda8..5ae62ca 100644 --- a/test/golden/pg/default/my_queries_sql.rbs +++ b/test/golden/pg/default/my_queries_sql.rbs @@ -6,7 +6,7 @@ module PgCodegen class NotFound < ::StandardError end - def self.connect: (::Hash[untyped, untyped] pg_params, ?logger: untyped, **untyped pool_options) -> instance + def self.connect: (::Hash[untyped, untyped] | ::String pg_params, ?logger: untyped, **untyped pool_options) -> instance def initialize: (untyped db, ?logger: untyped) -> void diff --git a/test/golden/pg/default/other_sql.rb b/test/golden/pg/default/other_sql.rb index c70eaa2..4a233b8 100644 --- a/test/golden/pg/default/other_sql.rb +++ b/test/golden/pg/default/other_sql.rb @@ -30,9 +30,11 @@ class OtherSql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -163,6 +165,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -194,14 +202,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -222,14 +237,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/test/golden/pg/default/other_sql.rbs b/test/golden/pg/default/other_sql.rbs index c50ee4b..c704c2f 100644 --- a/test/golden/pg/default/other_sql.rbs +++ b/test/golden/pg/default/other_sql.rbs @@ -15,7 +15,7 @@ module PgCodegen class NotFound < ::StandardError end - def self.connect: (::Hash[untyped, untyped] pg_params, ?logger: untyped, **untyped pool_options) -> instance + def self.connect: (::Hash[untyped, untyped] | ::String pg_params, ?logger: untyped, **untyped pool_options) -> instance def initialize: (untyped db, ?logger: untyped) -> void diff --git a/test/golden/pg/default/query_sql.rb b/test/golden/pg/default/query_sql.rb index df9da72..faca56c 100644 --- a/test/golden/pg/default/query_sql.rb +++ b/test/golden/pg/default/query_sql.rb @@ -606,9 +606,11 @@ class QuerySql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -954,7 +956,7 @@ def insert_postgres_array_types_batch(rows) def copy_postgres_array_types(rows) with_client('copy_postgres_array_types') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "postgres_array_types" ("id", "c_boolean", "c_double_precision", "c_bytea_array", "c_jsonb_array", "c_integer_array", "c_numeric_array", "c_timestamptz_array") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.c_boolean, encode_float(row.c_double_precision), encode_array(row.c_bytea_array, :bytea), encode_array(row.c_jsonb_array, :json), encode_array(row.c_integer_array), encode_array(row.c_numeric_array), encode_array(row.c_timestamptz_array)]) @@ -1107,7 +1109,7 @@ def get_authors_by_ids_and_names(param1:, param2:) # Bulk insert: pass an Array of CopyAuthorsArgs. def copy_authors(rows) with_client('copy_authors') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "authors" ("name", "bio") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.name, row.bio]) @@ -1297,7 +1299,7 @@ def create_regression_if_new(id:, order:) def copy_regressions(rows) with_client('copy_regressions') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "regressions" ("id", "order", "payload", "doc") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.order, encode_bytea(row.payload), encode_json(row.doc)]) @@ -1427,7 +1429,7 @@ def get_http_server!(...) def copy_mixed_case_copies(rows) with_client('copy_mixed_case_copies') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "MixedCaseCopies" ("Id", "SomeValue") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.some_value]) @@ -1600,6 +1602,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -1631,14 +1639,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -1659,14 +1674,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/test/golden/pg/default/query_sql.rbs b/test/golden/pg/default/query_sql.rbs index b222086..66d8ed2 100644 --- a/test/golden/pg/default/query_sql.rbs +++ b/test/golden/pg/default/query_sql.rbs @@ -630,7 +630,7 @@ module PgCodegen class NotFound < ::StandardError end - def self.connect: (::Hash[untyped, untyped] pg_params, ?logger: untyped, **untyped pool_options) -> instance + def self.connect: (::Hash[untyped, untyped] | ::String pg_params, ?logger: untyped, **untyped pool_options) -> instance def initialize: (untyped db, ?logger: untyped) -> void diff --git a/test/golden/pg/ruby31_gemfile/connection.rb b/test/golden/pg/ruby31_gemfile/connection.rb index 3a2a08e..dd35aa8 100644 --- a/test/golden/pg/ruby31_gemfile/connection.rb +++ b/test/golden/pg/ruby31_gemfile/connection.rb @@ -3,8 +3,65 @@ # auto-generated by sqlc - do not edit module PgCodegen - # State kept on every PG::Connection the first time it is used. + # State kept on every PG::Connection the first time it is used. The connection's own settings + # (type_map_for_results / _for_queries, field_name_type) are left to the application and other libraries + # sharing it: the generated queries pass their type maps with every call. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + attr_accessor :query_types, :result_types + + def typed(result) + result.type_map = result_types + result.field_name_type = :string + result + end + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # Points the inherited socket at /dev/null, so closing or resetting it here does not end the parent's session. + def detach_from_parent + release_parent_socket + ensure + @owner_pid = ::Process.pid + end + + def release_parent_socket + socket_io.reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + + # connection_pool closes the connections a forked child inherits, also ones closed before (which would raise). + def close + return if finished? + + detach_from_parent if inherited? + super + end + + def finish + return if finished? + + detach_from_parent if inherited? + super + end + # Names of the statements prepared on this connection; forgotten when the server session changes # (PG::Connection#reset after a failover / terminated backend). def prepared_statements @@ -104,6 +161,7 @@ def decoder(oid) @decoders.fetch(oid) do row = @client.exec("SELECT typcategory, typdelim FROM pg_type WHERE oid = #{Integer(oid)}").tap do |types| types.type_map = ::PG::TypeMapAllStrings.new + types.field_name_type = :string end.first decoder = row && NewTypes.decoder(oid, row['typcategory'], row['typdelim']) @known.add_coder(decoder) if decoder # later results no longer come here diff --git a/test/golden/pg/ruby31_gemfile/connection.rbs b/test/golden/pg/ruby31_gemfile/connection.rbs index f96bbee..03c7965 100644 --- a/test/golden/pg/ruby31_gemfile/connection.rbs +++ b/test/golden/pg/ruby31_gemfile/connection.rbs @@ -2,9 +2,20 @@ module PgCodegen interface _PgConnection def backend_pid: () -> ::Integer + + def socket_io: () -> ::IO + + def reset: () -> void + + def finished?: () -> bool + + def close: () -> void + + def finish: () -> void end module ConnectionState : _PgConnection + @owner_pid: ::Integer @prepared_statements: ::Set[::String] @prepared_statements_pid: ::Integer @stale_statements: ::Set[::String] @@ -14,6 +25,30 @@ module PgCodegen def stale_statements: () -> ::Set[::String] attr_accessor in_transaction: bool? + + def self.extended: (untyped client) -> void + + attr_accessor query_types: untyped + + attr_accessor result_types: untyped + + def typed: (untyped result) -> untyped + + def inherited?: () -> bool + + def detach_from_parent: () -> void + + def release_parent_socket: () -> void + + def self.after_fork: () -> void + + module ForkHook + def _fork: () -> ::Integer + end + + def close: () -> void + + def finish: () -> void end class TransactionEnded < ::StandardError diff --git a/test/golden/pg/ruby31_gemfile/my_queries_sql.rb b/test/golden/pg/ruby31_gemfile/my_queries_sql.rb index 097c98b..3a89361 100644 --- a/test/golden/pg/ruby31_gemfile/my_queries_sql.rb +++ b/test/golden/pg/ruby31_gemfile/my_queries_sql.rb @@ -18,9 +18,11 @@ class MyQueriesSql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -127,6 +129,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -158,14 +166,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -186,14 +201,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/test/golden/pg/ruby31_gemfile/my_queries_sql.rbs b/test/golden/pg/ruby31_gemfile/my_queries_sql.rbs index d7dd1af..2b97101 100644 --- a/test/golden/pg/ruby31_gemfile/my_queries_sql.rbs +++ b/test/golden/pg/ruby31_gemfile/my_queries_sql.rbs @@ -6,7 +6,7 @@ module PgCodegen class NotFound < ::StandardError end - def self.connect: (::Hash[untyped, untyped] pg_params, ?logger: untyped, **untyped pool_options) -> instance + def self.connect: (::Hash[untyped, untyped] | ::String pg_params, ?logger: untyped, **untyped pool_options) -> instance def initialize: (untyped db, ?logger: untyped) -> void diff --git a/test/golden/pg/ruby31_gemfile/other_sql.rb b/test/golden/pg/ruby31_gemfile/other_sql.rb index 1ae3eaf..724d27c 100644 --- a/test/golden/pg/ruby31_gemfile/other_sql.rb +++ b/test/golden/pg/ruby31_gemfile/other_sql.rb @@ -30,9 +30,11 @@ class OtherSql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -166,6 +168,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -197,14 +205,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -225,14 +240,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/test/golden/pg/ruby31_gemfile/other_sql.rbs b/test/golden/pg/ruby31_gemfile/other_sql.rbs index 6f04f1c..42ed8a3 100644 --- a/test/golden/pg/ruby31_gemfile/other_sql.rbs +++ b/test/golden/pg/ruby31_gemfile/other_sql.rbs @@ -16,7 +16,7 @@ module PgCodegen class NotFound < ::StandardError end - def self.connect: (::Hash[untyped, untyped] pg_params, ?logger: untyped, **untyped pool_options) -> instance + def self.connect: (::Hash[untyped, untyped] | ::String pg_params, ?logger: untyped, **untyped pool_options) -> instance def initialize: (untyped db, ?logger: untyped) -> void diff --git a/test/golden/pg/ruby31_gemfile/query_sql.rb b/test/golden/pg/ruby31_gemfile/query_sql.rb index 4c60454..216f025 100644 --- a/test/golden/pg/ruby31_gemfile/query_sql.rb +++ b/test/golden/pg/ruby31_gemfile/query_sql.rb @@ -593,9 +593,11 @@ class QuerySql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -943,7 +945,7 @@ def insert_postgres_array_types_batch(rows) def copy_postgres_array_types(rows) with_client('copy_postgres_array_types') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "postgres_array_types" ("id", "c_boolean", "c_double_precision", "c_bytea_array", "c_jsonb_array", "c_integer_array", "c_numeric_array", "c_timestamptz_array") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.c_boolean, encode_float(row.c_double_precision), encode_array(row.c_bytea_array, :bytea), encode_array(row.c_jsonb_array, :json), encode_array(row.c_integer_array), encode_array(row.c_numeric_array), encode_array(row.c_timestamptz_array)]) @@ -1096,7 +1098,7 @@ def get_authors_by_ids_and_names(param1:, param2:) # Bulk insert: pass an Array of CopyAuthorsArgs. def copy_authors(rows) with_client('copy_authors') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "authors" ("name", "bio") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.name, row.bio]) @@ -1298,7 +1300,7 @@ def create_regression_if_new(id:, order:) def copy_regressions(rows) with_client('copy_regressions') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "regressions" ("id", "order", "payload", "doc") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.order, encode_bytea(row.payload), encode_json(row.doc)]) @@ -1428,7 +1430,7 @@ def get_http_server!(...) def copy_mixed_case_copies(rows) with_client('copy_mixed_case_copies') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "MixedCaseCopies" ("Id", "SomeValue") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.some_value]) @@ -1603,6 +1605,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -1634,14 +1642,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -1662,14 +1677,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/test/golden/pg/ruby31_gemfile/query_sql.rbs b/test/golden/pg/ruby31_gemfile/query_sql.rbs index 9f920bf..ce2a359 100644 --- a/test/golden/pg/ruby31_gemfile/query_sql.rbs +++ b/test/golden/pg/ruby31_gemfile/query_sql.rbs @@ -795,7 +795,7 @@ module PgCodegen class NotFound < ::StandardError end - def self.connect: (::Hash[untyped, untyped] pg_params, ?logger: untyped, **untyped pool_options) -> instance + def self.connect: (::Hash[untyped, untyped] | ::String pg_params, ?logger: untyped, **untyped pool_options) -> instance def initialize: (untyped db, ?logger: untyped) -> void diff --git a/test/golden/pg/ruby32_notypes/connection.rb b/test/golden/pg/ruby32_notypes/connection.rb index 3a2a08e..dd35aa8 100644 --- a/test/golden/pg/ruby32_notypes/connection.rb +++ b/test/golden/pg/ruby32_notypes/connection.rb @@ -3,8 +3,65 @@ # auto-generated by sqlc - do not edit module PgCodegen - # State kept on every PG::Connection the first time it is used. + # State kept on every PG::Connection the first time it is used. The connection's own settings + # (type_map_for_results / _for_queries, field_name_type) are left to the application and other libraries + # sharing it: the generated queries pass their type maps with every call. module ConnectionState + def self.extended(client) = client.instance_variable_set(:@owner_pid, ::Process.pid) + + # A forked child (Puma / Sidekiq / Resque worker) lets go of every inherited session the generated code has + # used, also ones another thread had checked out at fork time and ones it never uses: its exit would end + # them for the parent. + def self.after_fork = ::ObjectSpace.each_object(self) { |client| client.release_parent_socket if client.inherited? } + + module ForkHook + def _fork + pid = super + ConnectionState.after_fork if pid.zero? + pid + end + end + ::Process.singleton_class.prepend(ForkHook) + + attr_accessor :query_types, :result_types + + def typed(result) + result.type_map = result_types + result.field_name_type = :string + result + end + + # Opened by the parent of a forked process (Puma / Sidekiq workers): its socket is the parent's session. + def inherited? = @owner_pid != ::Process.pid + + # Points the inherited socket at /dev/null, so closing or resetting it here does not end the parent's session. + def detach_from_parent + release_parent_socket + ensure + @owner_pid = ::Process.pid + end + + def release_parent_socket + socket_io.reopen(::IO::NULL) + rescue ::StandardError # already closed + nil + end + + # connection_pool closes the connections a forked child inherits, also ones closed before (which would raise). + def close + return if finished? + + detach_from_parent if inherited? + super + end + + def finish + return if finished? + + detach_from_parent if inherited? + super + end + # Names of the statements prepared on this connection; forgotten when the server session changes # (PG::Connection#reset after a failover / terminated backend). def prepared_statements @@ -104,6 +161,7 @@ def decoder(oid) @decoders.fetch(oid) do row = @client.exec("SELECT typcategory, typdelim FROM pg_type WHERE oid = #{Integer(oid)}").tap do |types| types.type_map = ::PG::TypeMapAllStrings.new + types.field_name_type = :string end.first decoder = row && NewTypes.decoder(oid, row['typcategory'], row['typdelim']) @known.add_coder(decoder) if decoder # later results no longer come here diff --git a/test/golden/pg/ruby32_notypes/my_queries_sql.rb b/test/golden/pg/ruby32_notypes/my_queries_sql.rb index 96bcd42..3cf0175 100644 --- a/test/golden/pg/ruby32_notypes/my_queries_sql.rb +++ b/test/golden/pg/ruby32_notypes/my_queries_sql.rb @@ -17,9 +17,11 @@ class MyQueriesSql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -126,6 +128,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -157,14 +165,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -185,14 +200,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/test/golden/pg/ruby32_notypes/other_sql.rb b/test/golden/pg/ruby32_notypes/other_sql.rb index e6f9b59..e4374cc 100644 --- a/test/golden/pg/ruby32_notypes/other_sql.rb +++ b/test/golden/pg/ruby32_notypes/other_sql.rb @@ -30,9 +30,11 @@ class OtherSql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -166,6 +168,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -197,14 +205,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -225,14 +240,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/test/golden/pg/ruby32_notypes/query_sql.rb b/test/golden/pg/ruby32_notypes/query_sql.rb index 3988c38..020f70b 100644 --- a/test/golden/pg/ruby32_notypes/query_sql.rb +++ b/test/golden/pg/ruby32_notypes/query_sql.rb @@ -606,9 +606,11 @@ class QuerySql class NotFound < ::StandardError; end # Opens a pool of connections: QuerySql.connect({ dbname: 'app' }, size: 5, timeout: 5, logger: Logger.new($stdout)) - # (pool_options go to ConnectionPool.new). To share an existing pool use QuerySql.new(pool). + # or QuerySql.connect(ENV.fetch('DATABASE_URL')) (pool_options go to ConnectionPool.new). To share an existing + # pool use QuerySql.new(pool). def self.connect(pg_params, logger: nil, **pool_options) - new(::ConnectionPool.new(**pool_options) { ::PG.connect(**pg_params) }, logger: logger) + connect = pg_params.is_a?(::String) ? -> { ::PG.connect(pg_params) } : -> { ::PG.connect(**pg_params) } + new(::ConnectionPool.new(**pool_options, &connect), logger: logger) end # db: a ConnectionPool (or anything that responds to #with and yields a PG::Connection). @@ -957,7 +959,7 @@ def insert_postgres_array_types_batch(rows) def copy_postgres_array_types(rows) with_client('copy_postgres_array_types') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "postgres_array_types" ("id", "c_boolean", "c_double_precision", "c_bytea_array", "c_jsonb_array", "c_integer_array", "c_numeric_array", "c_timestamptz_array") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.c_boolean, encode_float(row.c_double_precision), encode_array(row.c_bytea_array, :bytea), encode_array(row.c_jsonb_array, :json), encode_array(row.c_integer_array), encode_array(row.c_numeric_array), encode_array(row.c_timestamptz_array)]) @@ -1110,7 +1112,7 @@ def get_authors_by_ids_and_names(param1:, param2:) # Bulk insert: pass an Array of CopyAuthorsArgs. def copy_authors(rows) with_client('copy_authors') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "authors" ("name", "bio") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.name, row.bio]) @@ -1300,7 +1302,7 @@ def create_regression_if_new(id:, order:) def copy_regressions(rows) with_client('copy_regressions') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "regressions" ("id", "order", "payload", "doc") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.order, encode_bytea(row.payload), encode_json(row.doc)]) @@ -1430,7 +1432,7 @@ def get_http_server!(...) def copy_mixed_case_copies(rows) with_client('copy_mixed_case_copies') do |client| - encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.type_map_for_queries) + encoder = ::PG::TextEncoder::CopyRow.new(type_map: client.query_types) client.copy_data('COPY "MixedCaseCopies" ("Id", "SomeValue") FROM STDIN', encoder) do rows.each do |row| client.put_copy_data([row.id, row.some_value]) @@ -1603,6 +1605,12 @@ def instrument(name) def with_client(name) instrument(name) do @db.with do |client| + if client.is_a?(ConnectionState) && client.inherited? # opened before a fork: reconnect in this process + raise ::PG::ConnectionBad, 'the connection was inherited by a forked process inside a transaction' if client.in_transaction + + client.detach_from_parent + client.reset + end if client.status == ::PG::CONNECTION_BAD # closed by the server (restart, failover, killed) # inside #transaction a new session would run the rest of the block outside the transaction raise ::PG::ConnectionBad, 'the connection was lost inside a transaction' if client.is_a?(ConnectionState) && client.in_transaction @@ -1634,14 +1642,21 @@ def bound_to(connection) = dup.tap { |tx| tx.instance_variable_set(:@db, connect # Connections of an injected pool are set up on first use: typed results and Ruby arrays as parameters. def configure(client) + names = client.field_name_type + results = client.type_map_for_results + client.field_name_type = :string # the pg gem reads pg_type by column name + client.type_map_for_results = ::PG::TypeMapAllStrings.new type_map = ::PG::BasicTypeMapForResults.new(client) type_map.default_type_map = string_arrays(client) # uuid, enums, money, ... come back as Strings - client.type_map_for_results = type_map queries = ::PG::BasicTypeMapForQueries.new(client) # true / false as text ('t' / 'f'), also valid in COPY (the default binary encoder is not) queries[::TrueClass] = queries[::FalseClass] = ::PG::TextEncoder::Boolean.new(oid: 16) - client.type_map_for_queries = queries client.extend(ConnectionState) + client.result_types = type_map + client.query_types = queries + ensure + client.field_name_type = names if names + client.type_map_for_results = results if results end # Arrays of types without a decoder (uuid[], enum[], ...) become Arrays of Strings, other types Strings. @@ -1662,14 +1677,14 @@ def string_arrays(client) def run_prepared(client, name, sql, params = []) prepare(client, name, sql) unless client.prepared_statements.include?(name) begin - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) rescue ::PG::InvalidSqlStatementName, ::PG::FeatureNotSupported => e client.prepared_statements.delete(name) client.stale_statements.add(name) if e.is_a?(::PG::FeatureNotSupported) raise unless client.transaction_status == ::PG::PQTRANS_IDLE prepare(client, name, sql) - client.exec_prepared(name, params) + client.typed(client.exec_prepared(name, params, 0, client.query_types)) end end diff --git a/tests/app_helpers.rb b/tests/app_helpers.rb new file mode 100644 index 0000000..88f41e5 --- /dev/null +++ b/tests/app_helpers.rb @@ -0,0 +1,64 @@ +# frozen_string_literal: true + +# Helpers for the tests that use the generated code the way applications do. +module AppHelpers + # The smallest pool an application hands in: one connection it also uses itself (a raw driver connection shared + # with another library, or a connection opened before forking). + class SingleConnection + def initialize(connection) + @connection = connection + end + + def with + yield @connection + end + end + + # Runs the block in a forked child (Puma / Sidekiq / Resque workers) and returns its value, or the class and + # message of what it raised. + def in_forked_child + reader, writer = ::IO.pipe + parent = ::Process.pid + pid = fork_child(parent, writer) do + reader.close + result = begin + yield + rescue ::Exception => e + "#{e.class}: #{e.message}" + end + writer.write(::Marshal.dump(result)) + writer.close + exit!(0) + end + writer.close + result = ::Marshal.load(reader.read) + ::Process.wait(pid) + result + ensure + reader&.close + end + + # A fork hook that raises in the child (connection_pool's after_fork) must not let the child run the rest of the suite. + def fork_child(parent, writer, &) + fork(&) + rescue ::Exception => e + raise if ::Process.pid == parent + + writer.write(::Marshal.dump("fork hook: #{e.class}: #{e.message} #{e.backtrace&.first(8)}")) + exit!(1) + end + + # Forks a child that ends with a plain `exit`, like a worker process: at_exit handlers and finalizers run (and + # close what the child inherited). Returns its exit status. + def exit_forked_child + pid = fork do + yield if block_given? + exit 0 + end + ::Process.wait2(pid).last.exitstatus + end + + def skip_without_fork + skip 'Process.fork is not available on this platform' unless ::Process.respond_to?(:fork) + end +end diff --git a/tests/end2end_mysql2.rb b/tests/end2end_mysql2.rb index b144b30..1c0ec72 100755 --- a/tests/end2end_mysql2.rb +++ b/tests/end2end_mysql2.rb @@ -12,6 +12,8 @@ EXAMPLES_DIR = File.expand_path(ENV.fetch('EXAMPLES_DIR', File.expand_path('../examples', __dir__))).freeze require_relative 'consts' require_relative 'rbs_types' +require_relative 'app_helpers' +require 'delegate' require File.join(EXAMPLES_DIR, 'mysql2/query_sql') RbsTypes.install(Object.const_get("#{ENV.fetch('EXAMPLES_MODULE', 'Mysql2Codegen')}::QuerySql"), File.join(EXAMPLES_DIR, 'mysql2/query_sql.rbs')) @@ -19,6 +21,7 @@ # End-to-end tests of the code generated for examples/authors/mysql, against a real MySQL / MariaDB server. class TestMysql2 < Minitest::Test include Object.const_get(ENV.fetch('EXAMPLES_MODULE', 'Mysql2Codegen')) + include AppHelpers TEXT_PARTS = ["é", '🙂', "\t", "\n", "\r", '\\', "'", '"', ',', '\N', 'NULL'].freeze @@ -1297,4 +1300,169 @@ def test_interrupts_around_begin_and_commit_never_leave_the_connection_in_a_tran client&.close end end + + # ---- usage patterns of applications ---------------------------------------------------------------------------- + + def mysql2_client(**options) + Mysql2::Client.new(**mysql2_params.transform_keys(&:to_sym), **options) + end + + # Puma / Sidekiq / Resque fork workers from a parent that already ran queries. connection_pool drops the pooled + # clients in the child (after_fork) without ending the parent's sessions; a client inherited outside a pool is not + # used by the child (it shares the parent's socket) and raises Mysql2::Error::ConnectionError there. + def test_a_forked_child_leaves_the_parent_connections_usable + skip_without_fork + pooled = QuerySql.connect(mysql2_params, size: 1) + client = mysql2_client + single = QuerySql.new(SingleConnection.new(client)) + id = pooled.create_author_return_id(name: 'before fork', bio: nil) + assert_equal 'before fork', single.get_author!(id: id).name + parent_threads = [pooled.instance_variable_get(:@db).with(&:thread_id), client.thread_id] + child = in_forked_child do + error = assert_raises(Mysql2::Error::ConnectionError) { single.list_authors } + [pooled.get_author!(id: id).name, pooled.create_author_return_id(name: 'from child', bio: nil).class, + parent_threads.include?(pooled.instance_variable_get(:@db).with(&:thread_id)), error.message] + end + assert_kind_of ::Array, child, child.inspect + assert_equal ['before fork', ::Integer, false], child.first(3) + assert_match(/another process/, child.last) + assert_equal ['before fork', 'from child'], pooled.list_authors.map(&:name).sort + assert_equal 2, single.list_authors.size + assert_equal 2, pooled.transaction { |tx| tx.list_authors.size } + assert_equal parent_threads, [pooled.instance_variable_get(:@db).with(&:thread_id), client.thread_id] + ensure + client&.close + end + + # A worker forked while another thread has a pooled connection checked out (a Puma thread mid-request, a heartbeat + # thread during a Resque fork), and a direct connection the child never uses: the child's plain exit leaves both of + # the parent's sessions open. + def test_a_child_exit_leaves_connections_checked_out_or_unused_at_fork_open + skip_without_fork + pooled = QuerySql.connect(mysql2_params, size: 2) + client = mysql2_client + direct = QuerySql.new(SingleConnection.new(client)) + direct.list_authors + ready = ::Queue.new + go = ::Queue.new + worker = ::Thread.new do + pooled.transaction do |tx| + tx.list_authors + ready << true + go.pop + tx.list_authors.size + end + end + ready.pop + assert_equal 0, exit_forked_child + go << true + assert_equal 0, worker.value + assert_equal [], direct.list_authors + assert_equal 0, pooled.list_authors.size + ensure + go&.push(true) + client&.close + end + + # A client also used by another library or configured globally (Mysql2::Client.default_query_options with + # cast_booleans / symbolize_keys, a client with cast: false or as: :hash) keeps its options, and the generated rows + # keep the documented types. + def test_a_client_with_other_query_options_still_returns_typed_rows + defaults = Mysql2::Client.default_query_options.dup + @query_sql.insert_mysql_numeric_types(**numeric_values(1).merge(c_bool: 1, c_boolean: 0, c_tinyint: 1, c_int: 7)) + @query_sql.insert_mysql_datetime_types(**datetime_values(1).merge(c_date: Date.new(2024, 2, 29), + c_datetime: ::Time.new(2024, 2, 29, 12, 30, 0))) + id = create_author('typed', nil) + expected = [@query_sql.get_mysql_numeric_types(id: 1), @query_sql.get_mysql_datetime_types(id: 1), + @query_sql.get_author!(id: id)] + Mysql2::Client.default_query_options.merge!(cast_booleans: true, symbolize_keys: true) + [{}, { cast: false }, { as: :hash, cache_rows: false }].each do |options| + client = mysql2_client + client.query_options.merge!(options) + own_options = client.query_options.dup + shared = QuerySql.new(SingleConnection.new(client)) + actual = [shared.get_mysql_numeric_types(id: 1), shared.get_mysql_datetime_types(id: 1), shared.get_author!(id: id)] + assert_equal expected, actual, options.inspect + assert_equal own_options, client.query_options + ensure + client&.close + end + ensure + Mysql2::Client.default_query_options.replace(defaults) if defaults + end + + # Documented limitation: MySQL's BEGIN commits an open transaction, so #transaction on a client already inside a + # transaction opened by another library (ActiveRecord::Base.transaction on the same raw connection) commits it. + def test_transaction_inside_a_transaction_opened_by_another_library_commits_it + client = mysql2_client + shared = QuerySql.new(SingleConnection.new(client)) + client.query('BEGIN') + client.query("INSERT INTO authors (name) VALUES ('outer')") + shared.transaction { |tx| tx.create_author_return_id(name: 'inner', bio: nil) } + client.query('ROLLBACK') + assert_equal %w[inner outer], @query_sql.list_authors.map(&:name).sort + ensure + client&.close + end + + # Reads go to a replica query object, writes to the primary: a write sent to the read-only replica raises the + # driver's error and leaves the replica usable. + def test_primary_and_read_replica_query_objects + replica = QuerySql.connect(mysql2_params.merge('init_command' => 'SET SESSION transaction_read_only = 1'), size: 1) + id = @query_sql.create_author_return_id(name: 'primary', bio: nil) + assert_equal 'primary', replica.get_author!(id: id).name + error = assert_raises(Mysql2::Error) { replica.create_author_return_id(name: 'replica', bio: nil) } + assert_equal 1792, error.error_number # ER_CANT_EXECUTE_IN_READ_ONLY_TRANSACTION + error = assert_raises(Mysql2::Error) do + replica.transaction { |tx| tx.create_author_return_id(name: 'replica', bio: nil) } + end + assert_equal 1792, error.error_number + assert_equal ['primary'], replica.list_authors.map(&:name) + end + + # Query objects memoized in a frozen constant, subclassed (caching, scoping) and wrapped (SimpleDelegator, + # prepended instrumentation forwarding `...` / `**`), also inside transactions. + def test_memoized_frozen_subclassed_and_wrapped_query_objects + memoized = QuerySql.connect(mysql2_params, size: 2).freeze + id = memoized.create_author_return_id(name: 'memoized', bio: nil) + instrumentation = Module.new do + def calls = @calls ||= [] + + def get_author(**args) + calls << args + super + end + + def list_authors(...) + calls << :list + super + end + end + caching = Class.new(QuerySql) do + prepend instrumentation + def get_author(id:) = (@cache ||= {}).fetch(id) { @cache[id] = super } + end + cached = caching.new(memoized.instance_variable_get(:@db)) + assert_equal 'memoized', cached.get_author!(id: id).name + assert_same cached.get_author(id: id), cached.get_author(id: id) + assert_equal [{ id: id }] * 3, cached.calls + in_tx = cached.transaction { |tx| [tx.class, tx.get_author!(id: id).name, tx.list_authors.size] } + assert_equal [caching, 'memoized', 1], in_tx + repository = SimpleDelegator.new(memoized) + assert_equal 'memoized', repository.public_send(:get_author, **{ id: id }).name + assert_equal ['memoized'], repository.transaction { |tx| tx.list_authors.map(&:name) } + assert_equal [1, 1], Array.new(2) { ::Thread.new { memoized.list_authors.size } }.map(&:value) + end + + # Rows cached with Marshal (Rails.cache, Sidekiq-style payloads) and serialized as JSON (API responses). + def test_rows_round_trip_through_marshal_and_json + id = create_author('json "author"', "bio\nwith é") + row = @query_sql.get_author!(id: id) + assert_equal row, ::Marshal.load(::Marshal.dump(row)) + json = JSON.generate(row.to_h) + assert_equal({ 'id' => id, 'name' => 'json "author"', 'bio' => "bio\nwith é" }, JSON.parse(json)) + assert_equal row, row.class.new(**JSON.parse(json, symbolize_names: true)) + rows = @query_sql.list_authors + assert_equal rows, ::Marshal.load(::Marshal.dump(rows)) + end end diff --git a/tests/end2end_pg.rb b/tests/end2end_pg.rb index 339e9b8..fb8a8ac 100755 --- a/tests/end2end_pg.rb +++ b/tests/end2end_pg.rb @@ -13,6 +13,10 @@ EXAMPLES_DIR = File.expand_path(ENV.fetch('EXAMPLES_DIR', File.expand_path('../examples', __dir__))).freeze require_relative 'consts' require_relative 'rbs_types' +require_relative 'app_helpers' +require 'delegate' +require 'json' +require 'uri' require File.join(EXAMPLES_DIR, 'pg/query_sql') RbsTypes.install(Object.const_get("#{ENV.fetch('EXAMPLES_MODULE', 'PgCodegen')}::QuerySql"), File.join(EXAMPLES_DIR, 'pg/query_sql.rbs')) @@ -20,6 +24,7 @@ # End-to-end tests of the code generated for examples/authors/postgresql, against a real PostgreSQL server. class TestPg < Minitest::Test include Object.const_get(ENV.fetch('EXAMPLES_MODULE', 'PgCodegen')) + include AppHelpers TEXT_PARTS = ["é", '🙂', "\t", "\n", "\r", '\\', "'", '"', ',', '\N', 'NULL'].freeze @@ -1403,4 +1408,218 @@ def test_a_statement_left_prepared_by_a_failed_prepare_is_used ensure client&.close end + + # ---- usage patterns of applications ---------------------------------------------------------------------------- + + def pg_connection(**options) + PG.connect(**pg_params.transform_keys(&:to_sym), **options) + end + + # Puma / Sidekiq / Resque fork workers from a parent that already ran queries. connection_pool drops the pooled + # connections in the child (after_fork), and a connection inherited outside a pool is reconnected there: either + # way the parent's sessions stay usable. A pooled connection the application closed before does not break the fork. + def test_a_forked_child_leaves_the_parent_connections_usable + skip_without_fork + # libpq's GSS probe crashes a forked child on macOS (the usual PGGSSENCMODE=disable advice for Puma there). + params = pg_params.merge('gssencmode' => 'disable') + pooled = QuerySql.connect(params, size: 1) + closed = QuerySql.connect(params, size: 1) + closed.list_authors + closed.instance_variable_get(:@db).with(&:close) + client = pg_connection(gssencmode: 'disable') + single = QuerySql.new(SingleConnection.new(client)) + id = pooled.create_author_return_id(name: 'before fork', bio: nil) + assert_equal 'before fork', single.get_author!(id: id).name + parent_backends = [pooled, single].map { |q| q.instance_variable_get(:@db).with(&:backend_pid) } + child = in_forked_child do + [pooled.get_author!(id: id).name, single.get_author!(id: id).name, + pooled.create_author_return_id(name: 'from child', bio: nil).class, + [pooled, single].map { |q| q.instance_variable_get(:@db).with(&:backend_pid) }.intersect?(parent_backends)] + end + assert_equal ['before fork', 'before fork', ::Integer, false], child + assert_equal ['before fork', 'from child'], pooled.list_authors.map(&:name).sort + assert_equal 2, single.list_authors.size + assert_equal 2, pooled.transaction { |tx| tx.list_authors.size } + assert_equal parent_backends, [pooled, single].map { |q| q.instance_variable_get(:@db).with(&:backend_pid) } + ensure + client&.close + end + + # A worker forked while another thread has a pooled connection checked out (a Puma thread mid-request, a heartbeat + # thread during a Resque fork), and a direct connection the child never uses: the child's plain exit leaves both of + # the parent's sessions open. + def test_a_child_exit_leaves_connections_checked_out_or_unused_at_fork_open + skip_without_fork + params = pg_params.merge('gssencmode' => 'disable') + pooled = QuerySql.connect(params, size: 2) + client = pg_connection(gssencmode: 'disable') + direct = QuerySql.new(SingleConnection.new(client)) + direct.list_authors + ready = ::Queue.new + go = ::Queue.new + worker = ::Thread.new do + pooled.transaction do |tx| + tx.list_authors + ready << true + go.pop + tx.list_authors.size + end + end + ready.pop + assert_equal 0, exit_forked_child + go << true + assert_equal 0, worker.value + assert_equal [], direct.list_authors + assert_equal 0, pooled.list_authors.size + ensure + go&.push(true) + client&.close + end + + # A raw connection the application also uses through another library (ActiveRecord's raw_connection, Sequel, a + # hand-written repository) keeps that library's type maps and field names, and the generated rows stay typed + # whatever the library sets later. + def test_a_connection_shared_with_another_library_keeps_its_settings + client = pg_connection + results = PG::TypeMapAllStrings.new + queries = PG::TypeMapByClass.new + client.type_map_for_results = results + client.type_map_for_queries = queries + client.field_name_type = :symbol + shared = QuerySql.new(SingleConnection.new(client)) + id = shared.create_author_return_id(name: 'shared', bio: nil) + assert_kind_of ::Integer, id + assert_equal [id], shared.get_authors_by_ids(param1: [id]).map(&:id) + shared.copy_authors([CopyAuthorsArgs.new(name: 'copied', bio: nil)]) + assert_equal %w[copied shared], shared.transaction { |tx| tx.list_authors.map(&:name).sort } + assert_same results, client.type_map_for_results + assert_same queries, client.type_map_for_queries + assert_equal :symbol, client.field_name_type + assert_equal [{ id: id.to_s }], client.exec("SELECT id FROM authors WHERE name = 'shared'").to_a + + client.type_map_for_results = PG::TypeMapByOid.new + client.field_name_type = :string + row = shared.get_author!(id: id) + assert_equal [id, 'shared'], [row.id, row.name] + ensure + client&.close + end + + # DATABASE_URL (Heroku, Render, Kamal, docker-compose) is passed as it is. + def test_connect_accepts_a_connection_url + params = pg_params + url = "postgresql://#{URI.encode_www_form_component(params.fetch('user'))}:" \ + "#{URI.encode_www_form_component(params.fetch('password', ''))}@#{params.fetch('host', 'localhost')}:" \ + "#{params.fetch('port', 5432)}/#{params.fetch('dbname')}" + from_url = QuerySql.connect(url, size: 1) + id = from_url.create_author_return_id(name: 'from url', bio: nil) + assert_equal 'from url', @query_sql.get_author!(id: id).name + end + + # A transaction already opened on the connection by another library (ActiveRecord::Base.transaction on the same + # raw connection) becomes the outer transaction: #transaction is a savepoint inside it. + def test_transaction_inside_a_transaction_opened_by_another_library_is_a_savepoint + client = pg_connection + shared = QuerySql.new(SingleConnection.new(client)) + client.exec('BEGIN') + client.exec("INSERT INTO authors (name) VALUES ('outer')") + shared.transaction { |tx| tx.create_author_return_id(name: 'inner committed', bio: nil) } + assert_raises(RuntimeError) do + shared.transaction do |tx| + tx.create_author_return_id(name: 'inner rolled back', bio: nil) + raise 'rollback the savepoint' + end + end + assert_equal PG::PQTRANS_INTRANS, client.transaction_status + assert_equal ['inner committed', 'outer'], shared.list_authors.map(&:name).sort + assert_empty @query_sql.list_authors # nothing committed yet + client.exec('ROLLBACK') + assert_empty @query_sql.list_authors + ensure + client&.close + end + + # Schema-per-tenant apps (apartment, acts_as_tenant with schemas) switch search_path on a pooled connection: the + # statements prepared for one tenant read the tables of the next one. + def test_queries_follow_the_search_path_of_each_tenant + client = pg_connection + tenants = %w[sqlc_tenant_a sqlc_tenant_b] + tenants.each do |tenant| + client.exec("DROP SCHEMA IF EXISTS #{tenant} CASCADE; CREATE SCHEMA #{tenant}; " \ + "CREATE TABLE #{tenant}.authors (LIKE public.authors INCLUDING ALL)") + end + shared = QuerySql.new(SingleConnection.new(client)) + tenants.each do |tenant| + client.exec("SET search_path TO #{tenant}") + shared.create_author_return_id(name: "#{tenant} author", bio: nil) + end + tenants.reverse_each do |tenant| + client.exec("SET search_path TO #{tenant}") + assert_equal ["#{tenant} author"], shared.list_authors.map(&:name) + end + client.exec('SET search_path TO public') + assert_empty shared.list_authors + ensure + tenants&.each { |tenant| client&.exec("DROP SCHEMA IF EXISTS #{tenant} CASCADE") } + client&.close + end + + # Reads go to a replica query object, writes to the primary: a write sent to the read-only replica raises the + # driver's error and leaves the replica usable. + def test_primary_and_read_replica_query_objects + replica = QuerySql.connect(pg_params.merge('options' => '-c default_transaction_read_only=on'), size: 1) + id = @query_sql.create_author_return_id(name: 'primary', bio: nil) + assert_equal 'primary', replica.get_author!(id: id).name + assert_raises(PG::ReadOnlySqlTransaction) { replica.create_author_return_id(name: 'replica', bio: nil) } + assert_raises(PG::ReadOnlySqlTransaction) do + replica.transaction { |tx| tx.create_author_return_id(name: 'replica', bio: nil) } + end + assert_equal ['primary'], replica.list_authors.map(&:name) + end + + # Query objects memoized in a frozen constant, subclassed (caching, scoping) and wrapped (SimpleDelegator, + # prepended instrumentation forwarding `...` / `**`), also inside transactions. + def test_memoized_frozen_subclassed_and_wrapped_query_objects + memoized = QuerySql.connect(pg_params, size: 2).freeze + id = memoized.create_author_return_id(name: 'memoized', bio: nil) + instrumentation = Module.new do + def calls = @calls ||= [] + + def get_author(**args) + calls << args + super + end + + def list_authors(...) + calls << :list + super + end + end + caching = Class.new(QuerySql) do + prepend instrumentation + def get_author(id:) = (@cache ||= {}).fetch(id) { @cache[id] = super } + end + cached = caching.new(memoized.instance_variable_get(:@db)) + assert_equal 'memoized', cached.get_author!(id: id).name + assert_same cached.get_author(id: id), cached.get_author(id: id) + assert_equal [{ id: id }] * 3, cached.calls + in_tx = cached.transaction { |tx| [tx.class, tx.get_author!(id: id).name, tx.list_authors.size] } + assert_equal [caching, 'memoized', 1], in_tx + repository = SimpleDelegator.new(memoized) + assert_equal 'memoized', repository.public_send(:get_author, **{ id: id }).name + assert_equal ['memoized'], repository.transaction { |tx| tx.list_authors.map(&:name) } + assert_equal [1, 1], Array.new(2) { ::Thread.new { memoized.list_authors.size } }.map(&:value) + end + + # Rows cached with Marshal (Rails.cache, Sidekiq-style payloads) and serialized as JSON (API responses). + def test_rows_round_trip_through_marshal_and_json + id = create_author('json "author"', "bio\nwith é") + row = @query_sql.get_author!(id: id) + assert_equal row, ::Marshal.load(::Marshal.dump(row)) + json = JSON.generate(row.to_h) + assert_equal({ 'id' => id, 'name' => 'json "author"', 'bio' => "bio\nwith é" }, JSON.parse(json)) + assert_equal row, row.class.new(**JSON.parse(json, symbolize_names: true)) + rows = @query_sql.list_authors + assert_equal rows, ::Marshal.load(::Marshal.dump(rows)) + end end diff --git a/tests/end2end_sqlite3.rb b/tests/end2end_sqlite3.rb index d65e70d..2f0fb9d 100644 --- a/tests/end2end_sqlite3.rb +++ b/tests/end2end_sqlite3.rb @@ -11,6 +11,9 @@ EXAMPLES_DIR = File.expand_path(ENV.fetch('EXAMPLES_DIR', File.expand_path('../examples', __dir__))).freeze require_relative 'consts' require_relative 'rbs_types' +require_relative 'app_helpers' +require 'delegate' +require 'json' require File.join(EXAMPLES_DIR, 'sqlite3/query_sql') RbsTypes.install(Object.const_get("#{ENV.fetch('EXAMPLES_MODULE', 'Sqlite3Codegen')}::QuerySql"), File.join(EXAMPLES_DIR, 'sqlite3/query_sql.rbs')) @@ -18,6 +21,7 @@ # End-to-end tests of the code generated for examples/authors/sqlite, against a real (in-memory) SQLite database. class TestSqlite3 < Minitest::Test include Object.const_get(ENV.fetch('EXAMPLES_MODULE', 'Sqlite3Codegen')) + include AppHelpers TEXT_PARTS = ["é", '🙂', "\t", "\n", "\r", '\\', "'", '"', ',', '\N', 'NULL'].freeze @@ -901,4 +905,129 @@ def test_interrupts_around_begin_and_commit_never_leave_the_database_in_a_transa @query_sql.truncate_authors end end + + # ---- usage patterns of applications ---------------------------------------------------------------------------- + + # Puma / Sidekiq / Resque fork workers: the sqlite3 gem closes writable databases inherited by a child, so a child + # opens its own SQLite3::Database (documented); the parent's database stays usable. + def test_a_forked_child_opens_its_own_database_and_leaves_the_parent_usable + skip_without_fork + Tempfile.create(['sqlite-fork-test', '.db']) do |file| + db = SQLite3::Database.new(file.path) + db.execute_batch(SCHEMA) + parent = QuerySql.new(db) + parent.create_author_return_id(name: 'before fork', bio: nil) + child = in_forked_child do + own = SQLite3::Database.new(file.path) + queries = QuerySql.new(own) + [queries.list_authors.map(&:name), queries.create_author_return_id(name: 'from child', bio: nil).class] + ensure + own&.close + end + assert_equal [['before fork'], ::Integer], child + assert_equal ['before fork', 'from child'], parent.list_authors.map(&:name).sort + assert_equal 2, parent.transaction { |tx| tx.list_authors.size } + ensure + db&.close + end + end + + # A database also used by another library (Sequel, ActiveRecord's raw_connection) keeps its results_as_hash + # setting, and the generated rows keep their layout. + def test_a_database_shared_with_another_library_keeps_its_settings + id = create_author('shared', nil) + expected = @query_sql.get_author!(id: id) + @db.results_as_hash = true + assert_equal expected, @query_sql.get_author!(id: id) + assert_equal [expected.to_h], @query_sql.transaction { |tx| tx.list_authors.map(&:to_h) } + assert @db.results_as_hash + assert_equal [{ 'id' => id }], @db.execute('SELECT id FROM authors') + end + + # A transaction already opened on the database by another library becomes the outer transaction: #transaction is + # a savepoint inside it. + def test_transaction_inside_a_transaction_opened_by_another_library_is_a_savepoint + @db.execute('BEGIN') + @db.execute("INSERT INTO authors (name) VALUES ('outer')") + @query_sql.transaction { |tx| tx.create_author_return_id(name: 'inner committed', bio: nil) } + assert_raises(RuntimeError) do + @query_sql.transaction do |tx| + tx.create_author_return_id(name: 'inner rolled back', bio: nil) + raise 'rollback the savepoint' + end + end + assert_predicate @db, :transaction_active? + assert_equal ['inner committed', 'outer'], @query_sql.list_authors.map(&:name).sort + @db.execute('ROLLBACK') + assert_empty @query_sql.list_authors + end + + # Reads go to a read-only database query object, writes to the writable one: a write sent to the read-only one + # raises the driver's error and leaves it usable. + def test_primary_and_read_replica_query_objects + Tempfile.create(['sqlite-replica-test', '.db']) do |file| + primary_db = SQLite3::Database.new(file.path) + primary_db.execute_batch(SCHEMA) + primary = QuerySql.new(primary_db) + replica_db = SQLite3::Database.new(file.path, readonly: true) + replica = QuerySql.new(replica_db) + id = primary.create_author_return_id(name: 'primary', bio: nil) + assert_equal 'primary', replica.get_author!(id: id).name + assert_raises(SQLite3::ReadOnlyException) { replica.create_author_return_id(name: 'replica', bio: nil) } + assert_raises(SQLite3::ReadOnlyException) do + replica.transaction { |tx| tx.create_author_return_id(name: 'replica', bio: nil) } + end + refute_predicate replica_db, :transaction_active? + assert_equal ['primary'], replica.list_authors.map(&:name) + ensure + replica_db&.close + primary_db&.close + end + end + + # Query objects memoized in a frozen constant, subclassed (caching, scoping) and wrapped (SimpleDelegator, + # prepended instrumentation forwarding `...` / `**`), also inside transactions. + def test_memoized_frozen_subclassed_and_wrapped_query_objects + memoized = QuerySql.new(@db).freeze + id = memoized.create_author_return_id(name: 'memoized', bio: nil) + instrumentation = Module.new do + def calls = @calls ||= [] + + def get_author(**args) + calls << args + super + end + + def list_authors(...) + calls << :list + super + end + end + caching = Class.new(QuerySql) do + prepend instrumentation + def get_author(id:) = (@cache ||= {}).fetch(id) { @cache[id] = super } + end + cached = caching.new(@db) + assert_equal 'memoized', cached.get_author!(id: id).name + assert_same cached.get_author(id: id), cached.get_author(id: id) + assert_equal [{ id: id }] * 3, cached.calls + in_tx = cached.transaction { |tx| [tx.class, tx.get_author!(id: id).name, tx.list_authors.size] } + assert_equal [caching, 'memoized', 1], in_tx + repository = SimpleDelegator.new(memoized) + assert_equal 'memoized', repository.public_send(:get_author, **{ id: id }).name + assert_equal ['memoized'], repository.transaction { |tx| tx.list_authors.map(&:name) } + assert_equal [1, 1], Array.new(2) { ::Thread.new { memoized.list_authors.size } }.map(&:value) + end + + # Rows cached with Marshal (Rails.cache, Sidekiq-style payloads) and serialized as JSON (API responses). + def test_rows_round_trip_through_marshal_and_json + id = create_author('json "author"', "bio\nwith é") + row = @query_sql.get_author!(id: id) + assert_equal row, ::Marshal.load(::Marshal.dump(row)) + json = JSON.generate(row.to_h) + assert_equal({ 'id' => id, 'name' => 'json "author"', 'bio' => "bio\nwith é" }, JSON.parse(json)) + assert_equal row, row.class.new(**JSON.parse(json, symbolize_names: true)) + rows = @query_sql.list_authors + assert_equal rows, ::Marshal.load(::Marshal.dump(rows)) + end end