History search: rank on history priors, not fuzzy score alone (APP-5650)

[warp-agent-staging]

Description

HistorySearchItem::score returned Skim's raw fuzzy-match score and nothing else, so Ctrl+R
results were ranked purely on fuzzy alignment quality — recency, frequency, session and cwd
existed on HistoryEntry but were display-only. This produced three measured failure modes
(warp#6126, #3430, #5588, #6344, #4174, #1810):

1. No history priors. Every make * candidate scored identically regardless of recency, so a
   fresh command could get buried behind older ones with the same shape of match.
2. Spaces were literal. The whole query went to Skim as one pattern, so multi-word queries
   like cd hi orm matched nothing even when every term was present in the command.
3. Boundary bias beat consecutive substrings. Skim's own word-boundary bonus made a scattered
   match (txjs-cli push for query tcp) outscore a tight, contiguous one (adb tcpip 5000) —
   the opposite of what real fzf does.

We kept SkimMatcherV2 (same Smith–Waterman family as fzf v2, so swapping matchers would be a
lateral move) and fixed the ranking instead.

The new score

Two stages: fuzzy match as a gate that produces normalized match-quality features, then a
rank that combines that quality with history priors.

final = 0.55·match + 0.30·recency + 0.08·freq + 0.05·session + 0.02·cwd − 0.03·exit_penalty
match = 0.45·exact + 0.35·skim + 0.15·consecutive + 0.05·tightness

• exact is 1.0 for a whole-line match, 0.85 for any literal substring (anywhere — this is what
  fixes failure mode 1, since OS=linux make bar now gets the same exact-tier credit as make bar), 0.55 when the fuzzy alignment starts at the very first character, else 0.
• skim is Skim's raw score, normalized by query length and soft-capped via x / (x + 30) so an
  unbounded, length-scaled Skim score can't swamp the priors.
• consecutive and tightness are computed directly from the matched character indices
  (independent of Skim's internal bonuses), which is what fixes failure mode 3.
• recency is exp(-ln2 · age_days / 3); freq is ln(1+count)/ln(21) clamped to 1.

Query text is whitespace-tokenized and every term is matched as an independent fuzzy subsequence,
ANDed together (fzf-style), which fixes failure mode 2. A minimum match-quality floor filters out
matches that are technically non-empty under Skim's DP but are visual noise (a few scattered
characters in an unrelated command).

The zero state (blank query, shown before the user types) is a special case: it bypasses ranking
entirely and gives every candidate the same score, so the mixer's stable sort preserves
History::commands_shared()'s established chronological order — the same effect Skim's uniform
0 score for an empty pattern had before this PR. Priors only apply once there's an actual query.

Ordering, not just weights

The actual sort key packs (exact_line, match_band = floor(match/0.25), final_score, age) into
one f64, with exact_line/match_band given enough headroom (100.0 and 3.0 respectively) that
they always dominate final_score (bounded to ±1). This means an older whole-line exact match
always outranks a fresher weak/scattered match — that guarantee is structural, not a side effect
of the weights, per the golden fixture older_exact_match_outranks_fresher_weak_match.

Ordering direction is unchanged: higher score is still better, consistent with the existing Skim
convention, SearchMixer's ascending (priority_tier, score, source_order) sort, and
SearchResultOrdering::BottomUp ("the highest ranked result [is] ordered last", i.e. closest to
the input box). Neither warp_search_core's sort nor crates/fuzzy_match (shared by every other
search surface) were touched.

Data plumbing

• CommandHistorySummary.count (app/src/terminal/history.rs) already counted executions but was
  discarded when joining history-file commands to their sqlite metadata. Added a small
  History::command_execution_count(session_id, command) accessor instead of widening
  HistoryEntry (whose Eq/Hash are used elsewhere).
• History-file rows with no matching sqlite record (HistoryEntry::command_only) have no
  timestamp. These fall back to a synthetic age based on how many newer candidates exist in the
  chronologically-ordered list, reusing the same 3-day half-life, so they decay gracefully instead
  of reading as infinitely old.
• The cwd prior uses the session's live-tracked working directory
  (TerminalView::current_working_directory, backed by ActiveSession), threaded through
  workspace/view.rs::show_command_search → CommandSearchView::reset_state → the history data
  source. It does not use the most recent history entry's pwd: that field is captured when a
  command starts (HistoryEntry::for_session_command), so it's stale immediately after a cd
  until the next command runs.

What a user will notice

• A command you ran a minute ago will generally rank above one you ran weeks ago, even if both
  match the query equally well (previously ties were broken by data-source registration order).
• Multi-word queries like cd hi orm now match commands containing all three terms anywhere, not
  just as one literal phrase.
• A tight substring match (tcpip for tcp) will rank above a scattered one spanning word
  boundaries.
• An exact whole-line match (e.g. re-running a command you've typed verbatim before) will
  consistently rank at the very top, regardless of age.
• The directory you're currently in gives a small boost to commands previously run from there.

Coordination with APP-5564 / APP-4542 (unbounded SkimMatcherV2 allocation)

Whitespace tokenization means a multi-word query now calls SkimMatcherV2 once per term per
candidate instead of once, which multiplies (bounded by the number of terms a user types, typically
2-4) rather than fixes the allocation concern those tickets track. To avoid making it worse: each
term is matched independently against the same candidate list size (no new collection or
candidate-set growth), and a single-word query — the common case — is unaffected (exactly one
fuzzy_indices call per candidate, same as before). The score floor also reduces the number of
items that make it into HistorySearchItem/rendering, but doesn't reduce the per-candidate
matching cost itself; the underlying allocation fix is left to that other work.

Decisions not settled by the issue

• List-position age fallback: reuses the same 3-day half-life as real timestamps, treating each
  position back in the chronological list as roughly one synthetic day of age. This is a
  judgment call the issue didn't pin down a formula for.
• SKIM_SOFT_CAP = 30.0 and MATCH_SCORE_FLOOR = 0.12: chosen so the per-character raw
  Skim scores in the issue's concrete examples (~20-23) land mid-curve; validated against the
  golden fixtures rather than a specific formula in the issue.
• The frequency default for commands with no persisted sqlite record is 1 (not 0), since the entry
  itself is proof of at least one execution.
• Cross-source score scale (raised in review): the packed sort key changes the score scale
  compared across sources in the shared mixer (Command Search registers history alongside
  workflows, notebooks, env-var collections and AI results, all at the default priority_tier).
  This is an open policy question the orchestrator has put to the requester; not addressed in this
  revision pending that decision.

Linked Issue

Linear: APP-5650
Origin: warp#6126, #4174, #3430, #5588, #6344, #1810

Testing

• Added table-driven golden fixture tests (rank_tests.rs) against a fixed clock, covering all
  three failure modes plus guard cases (older exact match beats a fresh weak match, frequency,
  session, cwd, exit-status penalty, missing-timestamp fallback, the score floor, and the blank
  query bypassing priors entirely rather than just the floor).
• End-to-end mixer tests (searcher_tests.rs) for: exact-match-over-substring ranking; blank
  query preserving chronological zero-state order despite differing priors (verified this
  actually fails against a floor-only-bypass revision, and passes against this one); and the live
  cwd prior winning over a stale last-entry pwd.
• cargo test -p warp --lib search::command_search (24 passed) and
  cargo test -p warp --lib terminal::history (20 passed).
• ./script/format and cargo clippy -p warp --all-targets --tests -- -D warnings (clean), plus
  the workspace-wide cargo clippy --workspace --exclude warp_completer --all-targets --tests -- -D warnings from script/presubmit (clean).
• Visual verification: built and launched the real Warp GUI client (cargo run --bin warp) from
  this branch, seeded real shell history, and confirmed both multi-word (space-AND) matching and
  recency-based prior ranking on the running Ctrl+R panel. See recording below.

Recording

Multi-word matching (hi orm matching a cd .../history_orm command) and recency-based ordering
(make query ranking the freshest of three similar matches closest to the input):

Ctrl+R history search demo (video)

• Screenshot: multi-word match

• Screenshot: recency-based ordering

• I have manually tested my changes locally with ./script/run

Agent Mode

• Warp Agent Mode - This PR was created via Warp's AI Agent Mode

[warp-agent-staging]

This PR was generated with Warp.

Comment @warp-factory on this PR to send it follow-up work.

  

[warp-agent-staging]

Overview

Replaces Ctrl+R history ranking with a normalized match-quality gate plus history priors, keeping SkimMatcherV2. One open policy question needs a human decision; the other review findings are already back with the author.

Concerns

• The packed history sort key changes the score scale the shared mixer compares across sources, without isolating history from them. Command Search registers history alongside workflows, notebooks, env-var collections and AI results (app/src/search/command_search/view.rs:222-331), all at default priority_tier 0, and SearchMixer compares raw score() across sources (crates/warp_search_core/src/mixer.rs:459) — non-whole-line history now tops out near 13 and whole-line history starts near 100, while the untouched sources still return raw Skim scores in the tens to hundreds. Decide the intended policy (preserve history's former cross-source position, give history a distinct priority tier, or normalize every participating source), implement it, and cover it with a mixer test using genuinely heterogeneous result types, since the two updated searcher_tests.rs assertions use two history sources and so cannot validate cross-source interleaving.

Verdict

Checks: build pass, tests not independently verified, CI red (required jobs reported skipped; no code-test failure exposed), visual proof missing

Found: 0 critical, 1 important, 0 suggestions, 0 nits

Responding as wilson: Open session · View in factory