docs(bots): teach backend selection (Thrift/SEA/kernel) + realkernel tiers

Follow-up to the #870 review thread on --all-extras: the bot needs to know, per
issue, WHICH backend the bug is on and reproduce on that one — a Thrift bug won't
reproduce on a kernel connection, and a broad unit run with the real kernel wheel
present false-reds unless realkernel is deselected. That knowledge was tribal;
write it down.

- CONTRIBUTING.md: add a "Backends and test tiers" section — the three backends
  (Thrift default / SEA `use_sea=True` / kernel `use_kernel=True`), where each
  backend's tests live, that kernel is an opt-in extra, and the rule that
  `realkernel` tests run in their own invocation (`-m "not realkernel"` for broad
  runs), matching how CI (code-coverage.yml / code-quality-checks.yml) splits them.
- engineer/system.md: add step 0 — pick the backend the bug is on and reproduce
  there; point to the CONTRIBUTING matrix.
- engineer-followup/system.md: correct the stale "do NOT run tests/e2e" line (the
  followup job now has live creds via #870) and point at the same backend matrix.

Keeps --all-extras (both backends supported); the residual "prompt-discipline
only" risk the reviewer flagged is now backed by a documented, human-shared
convention plus explicit bot rules.

Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
Co-authored-by: Isaac

Author: eric-wang-1990

.bot/prompts/engineer-followup/system.md

@@ -27,9 +27,13 @@ Your job:
   4. End with a short summary of what changed.
 
 Repo facts you need:
-  - `poetry`-managed, Python 3.8+; `poetry install` has run on the runner, so
-    `poetry run python -m pytest tests/unit` runs the fully-mocked unit suite
-    with no warehouse. Do NOT run or add `tests/e2e` (needs live credentials).
+  - `poetry`-managed, Python 3.8+; `poetry install --all-extras` has run on the
+    runner, and the live-warehouse connection env is set — so both
+    `poetry run python -m pytest tests/unit` (mocked) AND `tests/e2e` (live) are
+    runnable. If a reviewer's ask is about connector behavior that only an E2E test
+    can verify, add/adjust a `tests/e2e` test on the RIGHT backend for the change
+    (Thrift default / `use_sea=True` / `use_kernel=True` — see CONTRIBUTING.md →
+    "Backends and test tiers"), don't settle for a mocked unit test.
   - Source is under `src/databricks/sql/`; unit tests under `tests/unit/`.
     Follow `CONTRIBUTING.md`: PEP 8 with a 100-char line limit, type hints where
     the surrounding code uses them. This is a widely-consumed connector — keep

.bot/prompts/engineer/system.md

@@ -97,11 +97,24 @@ up whatever it needs.
 
 == HOW TO WORK (bug-fix flow) ==
 
+0. **Pick the BACKEND the bug is on — reproduce on that one.** The connector has
+   three backends and a bug on one won't reproduce on another. Choose from the
+   issue:
+   - **kernel** (`use_kernel=True`) — only if the issue is specifically about the
+     Rust kernel / `use_kernel`. Repro in `tests/e2e/test_kernel_backend.py` and run
+     it alone (it needs the real wheel; see the `-m "not realkernel"` note above).
+   - **SEA** (`use_sea=True`) — if the issue implicates SEA, or the area is
+     backend-parametrized (add the `{"use_sea": True}` param case).
+   - **Thrift** (the default, no kwarg) — everything else; this is the common case.
+   See CONTRIBUTING.md → "Backends and test tiers" for the full matrix (selection
+   kwarg + where each backend's tests live).
+
 1. **Write the failing E2E test FIRST — before you deep-dive the fix.** Your first
-   substantive action is a `tests/e2e/` test that REPRODUCES the bug. Do only the
-   minimal reading needed to write it (find the API to call + how the e2e tests
-   connect). Run it with `-k` and confirm it **fails for the right reason** (the
-   bug — not a compile/setup/skip). A *skipped* test is not a reproduction.
+   substantive action is a `tests/e2e/` test (on the backend from step 0) that
+   REPRODUCES the bug. Do only the minimal reading needed to write it (find the API
+   to call + how the e2e tests connect). Run it with `-k` and confirm it **fails for
+   the right reason** (the bug — not a compile/setup/skip). A *skipped* test is not
+   a reproduction.
    - **Reproduction is a HARD GATE.** If after a focused effort (a few attempts,
      not dozens) you cannot get a test that fails for the right reason — it only
      skips, you can't reach the warehouse, or you can't trigger the bug — **STOP

CONTRIBUTING.md

@@ -144,6 +144,33 @@ The `PySQLStagingIngestionTestSuite` namespace requires a cluster running DBR ve
 
 The suites marked `[not documented]` require additional configuration which will be documented at a later time.
 
+#### Backends and test tiers
+
+The connector has **three execution backends**, selected per connection. When you
+reproduce or fix a bug, use the backend the bug is actually on — a Thrift bug won't
+reproduce on a kernel connection, and vice versa:
+
+| Backend | Select via (connect kwarg / `extra_params`) | Where its tests live |
+| --- | --- | --- |
+| **Thrift** (default) | *(nothing — the default path)* | the general `tests/e2e` suite (the `{}` parametrize case) and mocked `tests/unit` |
+| **SEA** | `use_sea=True` | the general `tests/e2e` suite (the `{"use_sea": True}` parametrize case) |
+| **Kernel** (Rust, optional) | `use_kernel=True` | the dedicated `tests/e2e/test_kernel_backend.py` / `test_kernel_tls.py`, plus the offline routing test `tests/unit/test_session.py -m realkernel` |
+
+Notes that matter when running the suite:
+
+- **Kernel is an opt-in extra**, not part of the default install. `use_kernel=True`
+  needs `pip install "databricks-sql-connector[kernel]"` (or `poetry install
+  --all-extras`); without it the connector raises a clear "install the `[kernel]`
+  extra" error. Most bugs are on the **Thrift/SEA** path — reproduce those there;
+  only reach for kernel when the issue is specifically about `use_kernel`.
+- **`realkernel` tests must run in their own pytest invocation.** When the real
+  kernel wheel is installed (`--all-extras`), several unit tests fake
+  `databricks_sql_kernel` in `sys.modules`; the `@pytest.mark.realkernel` guard test
+  detects that shadowing and **fails loudly**. So a *broad* unit run with the wheel
+  present must **deselect it**: `poetry run python -m pytest tests/unit -m "not
+  realkernel"`, and run the real-wheel tests separately with `-m realkernel` (this is
+  exactly how CI splits them — see `code-coverage.yml` / `code-quality-checks.yml`).
+
 
 ### Code formatting