Skip to content

P928: replace the handwritten Vim subset with embedded Neovim - #1320

Merged
trissim merged 34 commits into
mainfrom
work/vim-engine--P928
Oct 30, 2026
Merged

trissim merged 34 commits into
mainfrom
work/vim-engine--P928

Conversation

@trissim

@trissim trissim commented Oct 29, 2026 •

Copy link
Copy Markdown

Summary — real Neovim instead of a handwritten Vim subset

Toad: 18 files, +1,607/-134 against fetched main 765996b80. Textual's companion restores 15 files, +291/-936: 26 handwritten Vim commands and 62 lines of private motion geometry removed; document/_editing.py shrinks from 765 to 157 lines.

Current Toad head: 90ae1d8fe3aa6eefbc31ac4c87ac8a1cbd5ccacc, pushed and remote-verified. Textual #212 merged at bea0d377b9bc414dc377f1ebcbb51405fc7b0e03, restoring #208 over #209. pyproject.toml:46 tracks Textual main; generated uv.lock:293 and :1331 agree on the branch and actual restored-main resolution. Toad main still imports Textual's deleted VimInsert; this is its matching replacement, not a compatibility shim.

Not installed. Claude exercised all seven real-Neovim groups at 16964a0de; the separate refused-handshake control is now green in hosted run 40981237928. These establish those paths, not a full-suite or current-head installation pass. P3 selection optimization remains separate and non-gating.

Ownership before and after

Before: TextArea interpreted a handwritten command family with separate motion geometry, linewise clipboard behavior and downstream catalog hooks. Adding Vim behavior meant implementing it again in Python.

After:

EditingMode                     native TextArea defaults and engine hooks
  VimMode                       modal binding marker, no command grammar
    NeovimEditing(VimMode, MroDispatch)
KeyMode                         retained Standard/VimInsert/VimNormal/VimVisual profiles

NeovimEditing owns one lazy nvim --embed --clean -n child. AttachedChild, AsyncExitStack and join_retirement supply acquisition and cleanup. Neovim owns buffer changes, modes, operators, registers and undo. Standard stays on native TextArea editing. Saved profiles, drafts, programmatic edits, history and capture/return use those existing owners. msgpack==1.2.2 is declared and locked; CI prepares Neovim 0.12.4.

Decode MessagePack once at its boundary

Before: JSON-shaped decoding inspected raw MessagePack arguments/results and rejected real Window extensions before their selected owner saw them.

After: IdentifiedFrame validates the request ID; MethodFrame validates method/header shape and retains raw arguments; RequestFrame composes both; ResponseFrame retains raw results. API metadata acquired before UI attach supplies Buffer/Window/Tabpage handle identities and the actual channel. The unpacker's ext_hook decodes each handle once; inherited MroDispatch writes those same owners. Metadata fields use their existing FieldPlan/FieldCodec rather than another key map. Unknown extension codes and malformed arity refuse.

tests/textual/test_neovim_wire.py covers nominal Window redraw/response preservation, handle encoding, malformed arity and unknown-code refusal through the real unpacker. No author-run pass is claimed for that control.

Completion is not FIFO input acceptance

Before: waiting for deferred engine execution inside App dispatch prevented delivery of the operand or Escape needed to finish it. An idle SafeState scheduling loop then starved progress.

After: the FIFO writer can send the next key without awaiting a deferred reply. Actual changed-generation SafeState supplies completion. NeovimCompletion, NotCompleted and CompletedGeneration preserve that distinction across snapshot transitions; a failed state cannot regain proof. Pending requests or unpublished lines prevent flush. Flush refuses immediately without a mode probe, delayed automatic submit or input replay. SafeState publishes directly only when completion changes; no self-scheduling idle loop remains.

RPC-only operations also keep their real owner: NeovimLuaOperation loads Lua instead of looking for a nonexistent vim.api.nvim_exec_lua, and resize queues NeovimCall for RPC-only nvim_ui_try_resize without impersonating a buffer generation. Native API lookup asserts the actual operation name before execution.

One document, one publication path

Before: lines could wait for a UI message while a newer cursor/selection was already exposed; rendering could mutate the buffer; full transfers could replace the document retained by history.

After:

DocumentBase.replace_lines       one text replacement algorithm
  Document
    SyntaxAwareDocument          existing replace_range specialization
RangedEditResult(EditResult)      original range plus actual result
TextArea.project_engine_edit     view/cache effects, not text mutation
NeovimEditing.publish            apply preceding lines, expose snapshot, finish view batch

Regular displays and arrow callbacks share publish. Render refuses an unpublished line batch and never applies text. Replies and displays share one coalesced paint request. Full transfers mutate the retained document rather than replacing its identity; TextArea.set_document owns adoption. The competing apply_engine_text/apply_engine_lines entry points were deleted and searched across src/tests/tools in both paired roots.

Hidden documents and restored views

DetachedReceiver                no retained document or projection
  RetainedReceiver              actual document, not retired widget
    AttachedReceiver            current widget effects

Hidden child changes update the retained document. Reattachment adopts it before exposing the engine, and generic paint_selection restores view selection without echoing another engine command. UnpaintedSnapshot represents the lack of an actual presentation, including after a view rebuild; it is not a force flag. NeovimUpdated carries its owner and the prompt asks that owner to paint. Accepted flush/capture still precedes screen detachment and readiness withdrawal.

Arrow callbacks carry the actual binding identity and display through CursorBoundaryRequest, AttachedReceiver and the existing MethodBinding/HistoryCursor path. There is no duplicate arrow roster. P917's central key migration remains its own paired PR.

Repair main's refused-handshake red at the retained owner

This was a pre-existing red on main, not a Neovim regression. AgentSession.session_failed already retained failure feedback, but FailedSessionDisposition.status dropped its details while constructing SessionDisconnected. agent_session.py:113-115 now includes those retained detail lines. The existing SessionDetails disclosure presents them; no persistent ghost error Note returns.

The startup control opens the actual disconnected-status disclosure and still requires AGENT_REFUSED_TO_START, Not sent and not-ready state. The log-link control waits for the actual link instead of unrelated hidden detail. Claude reports the refused-handshake control PASSED in run 40981237928.

Main's stale mode-call test was repaired separately in merged #1341 and is included here. Its five in-flight cases use Agent.set_mode and the actual set_config_option SDK call; the deleted AgentSession.set_mode API was not resurrected.

Dependency and CI changes

Evidence and remaining work

Evidence Result and limit
Retained source inventory 566 production Python modules parsed across paired roots; zero omissions. Lua/C and dynamic binding relationships read separately. Source evidence, not execution.
Real Neovim at 16964a0de All seven groups exercised: insert, motion, visual, undo/redo/paste, keymap, hidden retained editor and reattach/delayed paint. 31/32 raw checks; the last comparison omitted the selected newline cell in the harness, not a product paint failure.
Hosted run 40981237928 Refused-handshake PASSED. Remaining reported reds: test_context_inspection_source expects message_parts on a tuple; test_channel_display_basis calls renamed Messaging.send_message rather than send_user_message. Claude identifies these as Toad/Core drift, not introduced here.
Current source head Diff check passed; Toad main fetched and merge reports already up to date; push and exact ls-remote verified; clean tree.
Installation/performance Not performed or claimed for this head. Root owns combined acceptance and installation.

No author pytest, synthetic application, replacement-head import/type checker, new environment, provider input or installation was run. A cancelled earlier suite proves neither pass nor absence of a hang.

P3 remains owned and unfinished

Lua still eagerly calls getregionpos/getregion over selected rows. Native source confirms selected-row work; it has not been optimized away. The separately preserved Textual branch work/vim-neovim--P928--textual at 4ef5a1a56 carries the WrapOffset/native-cell preparation (+32/-40). No P3 code is included in #212. After #1320 merges, fresh branches carry compact selection, native tabstop/virtualedit and deletion of the folded-word second pass. Partial-cell geometry and measured CPU/latency remain open.

26 fake Vim commands retired. CI no longer closes its eyes after finding the first body.

Assumed: Claude owns exact-head acceptance, integration of main's other API-drift repairs, merge and installed live-path verification. Merged, installed and observed working remain separate facts.

@trissim
trissim marked this pull request as ready for review October 29, 2026 18:46
@trissim
trissim enabled auto-merge October 29, 2026 18:46
Copilot AI balanced review requested due to automatic review settings October 29, 2026 18:46

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

trissim and others added 2 commits October 29, 2026 15:14
…se.sha

pull_request.base.sha is not refreshed when main moves. toad#1320 at
c29eaa1 was diffed from 6d83f98, fifteen main commits behind its real
merge-base 4c3c4f3, and charged with a NoneAbsence at screens/main.py:231
and a BroadSwallow at session_navigation.py:319 that main commits 776f7f9
and 961e7ca introduced; #1320 touches neither file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RnH6dR85kyFKcc2tQAJLhK
@trissim
trissim disabled auto-merge October 29, 2026 20:38
@trissim
trissim merged commit 36719ff into main Oct 30, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants