MDEV-38243 Write binlog row events for changes done by cascading FK operations

[sjaakola]

This commit implements a feature which changes the handling of cascading foreign
key operations to write the changes of cascading operations into binlog.
The applying of such transaction, in the slave node, will apply just the binlog
events, and does not execute the actual foreign key cascade operation.
This will simplify the slave side replication applying and make it more predictable
in terms of potential interference with other parallel applying happning
in the node.

This feature can be turned ON/OFF by new variable:
rpl_use_binlog_events_for_fk_cascade, with default value OFF

Since the original version, this PR has changes after two reviews, by Kristian and Serg, mainly to:

• Supporting slave with old MariaDB version.
  Events logged in cascade operation are additionally flagged with
  the long-standing NO_FOREIGN_KEY_CHECKS_F, so a replica that does
  not understand FK_CASCADE_EVENTS_F still disables foreign key checks
  and does not re-execute the cascade
• SE/server API now narrows the SE role to just report the changes done by foreign key cascading,
  and server side does most of the work after that.

[maryo]

Thanks for this work. It would really help our project.
I tried it in an experiment, resetting the database between the tests of a large test suite by replaying the binlog reversed (like mariadb-binlog --flashback), and ran into the problems below.
Sections 1 and 2 reproduce on main (d45cf75) with this PR (73fe01d) as it is: the replica silently diverges, and nested cascades can crash a debug server.
Section 3 shows once section 2 is fixed. With the attached fix, all tests pass on main, 12.3.3 and 11.8.9, including our project's large test suite.

The code and this report were generated by Claude Opus 5.5; I only tested them, not reviewed, so they probably need some polishing.

1. Nested cascades lose the middle levels and can overrun a buffer

The row images live in two plain THD members, assuming one cascade action in flight per THD (design doc §6.3). But InnoDB runs the next level of an FK chain from inside the outer action (row_update_cascade_for_mysql() → row_ins_foreign_check_on_constraint()), so the inner action overwrites and frees the outer one's images:

SET SESSION rpl_use_binlog_events_for_fk_cascade=1;
CREATE TABLE gp (id INT PRIMARY KEY) ENGINE=InnoDB;
CREATE TABLE p (id INT PRIMARY KEY, gp_id INT,
  FOREIGN KEY (gp_id) REFERENCES gp(id) ON DELETE CASCADE) ENGINE=InnoDB;
CREATE TABLE c (id INT PRIMARY KEY, p_id INT,
  FOREIGN KEY (p_id) REFERENCES p(id) ON DELETE CASCADE) ENGINE=InnoDB;
INSERT INTO gp VALUES (1),(2);
INSERT INTO p VALUES (1,1),(2,2),(3,1);
INSERT INTO c VALUES (1,1),(2,2),(3,3),(4,1);
DELETE FROM gp WHERE id=1;

The binlog has the deletes of gp and c, but none of p. The events are flagged, so the replica does not re-run the cascade and keeps p 1 and 3. The inner capture also reuses the outer table's buffer: with a longer child row it overruns it (Freeing overrun buffer, then signal 11 in a debug build).

Fix: a stack of image frames, each sized for its own table, opened by the before-image capture and closed by thd_fk_cascade_row() / thd_fk_cascade_abort(). InnoDB now also calls thd_fk_cascade_abort() when the before-image capture fails, as the service contract says.

2. Self-referencing foreign keys are not captured

ha_innobase::fk_cascade_fetch_row() fails when m_prebuilt->mysql_template is NULL, although build_template() right below creates it. The table instance opened for a self-referencing FK has not read anything yet, so it never has a template:

SET SESSION rpl_use_binlog_events_for_fk_cascade=1;
CREATE TABLE s (id INT PRIMARY KEY, parent_id INT, KEY(parent_id),
  FOREIGN KEY (parent_id) REFERENCES s(id) ON DELETE CASCADE) ENGINE=InnoDB;
CREATE TABLE o (id INT PRIMARY KEY, s_id INT, KEY(s_id),
  FOREIGN KEY (s_id) REFERENCES s(id) ON DELETE CASCADE) ENGINE=InnoDB;
INSERT INTO s VALUES (1,NULL),(2,1),(3,2),(4,NULL);
INSERT INTO o VALUES (10,1),(20,2),(30,3),(40,4);
DELETE FROM s WHERE id=1;

The rows of o are captured, those of s 2 and 3 are not, so the replica does not re-run the cascade and keeps s 2 and 3.

Fix: drop the NULL check. Note that this check is also why capture does nothing on an applier (§11, rpl_fk_cascade_binlog_row_slave_option): the FK child is open there via prelocking too. Without the check, a replica with the option ON logs the cascades it re-runs, like any other session. That seems right, and the test is updated accordingly; if the applier should stay inert, it needs an explicit check.

3. Cascaded rows are logged after the statement's own rows

Once section 2 is fixed, this appears: the queue is flushed at statement end (§7), which breaks the order of execution that §6.5 promises:

SET SESSION rpl_use_binlog_events_for_fk_cascade=1;
CREATE TABLE s (id INT PRIMARY KEY, parent_id INT, KEY(parent_id),
  FOREIGN KEY (parent_id) REFERENCES s(id) ON DELETE SET NULL) ENGINE=InnoDB;
INSERT INTO s VALUES (1,NULL),(2,1);
DELETE FROM s WHERE id IN (1,2);

The binlog has DELETE s(1), DELETE s(2), then the cascaded UPDATE s(2), and the replica stops with ERROR 1032.

Fix: flush the queue in handler::binlog_log_row() before each of the statement's own rows, and start a new row event in prepare_pending_rows_event() when FK_CASCADE_DERIVED_F changes.

Smaller things

• The design doc refers to rpl_fk_cascade_binlog_row_mixed_eligibility, which is not in the PR. One written along its description is attached.
• §6.3 eligibility checks every key part in table->vfield, which also covers STORED generated columns and the hidden hash of UNIQUE ... USING HASH. Safe, but maybe more than intended.
• Cascade row images leave out virtual columns, unlike regular events with binlog_row_image=FULL. Fine for replicas, worth knowing for binlog tools.
• int prepare_for_insert(bool do_create); added to handler in sql/handler.h has no definition or callers.

Attached, both applying on top of this PR on main:

• 0001-MDEV-38243-fix-nested-and-self-referencing-FK-cascades.patch: the fix for sections 1-3, with the updated rpl_fk_cascade_binlog_row_slave_option and design document.
• 0002-MDEV-38243-tests-for-nested-and-self-referencing-FK-cascades.patch: new tests. They cover chains up to the InnoDB maximum (14 levels) and one deeper, UPDATE and SET NULL chains, a diamond, savepoints, self-referencing tables and the row order, each checked on a replica, plus the same in Galera, and the missing mixed_eligibility test.

[sjaakola]

@maryo thanks for your review. I added your patch verbatim to this PR, it appears to fix the 3 vulnerabilities you found in this work.