Skip to content

feat(extensions): run extensions - #523

Merged
scottlovegrove merged 2 commits into
feat/extensions-2-discoveryfrom
feat/extensions-3-dispatch
Sep 11, 2026
Merged

feat(extensions): run extensions#523
scottlovegrove merged 2 commits into
feat/extensions-2-discoveryfrom
feat/extensions-3-dispatch

Conversation

@scottlovegrove

@scottlovegrove scottlovegrove commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Slice 3 of 7, on top of slice 2.

The execution half of the contract: arguments reach the extension exactly as the user typed them, the three standard streams are the host's own, and the host exits with whatever the extension exited with.

Node-shebang scripts are launched with the host's own Node, which makes them work on Windows and pins them to the Node version the CLI already requires. Other scripts run directly on POSIX and through sh on Windows.

The environment adds the documented TD_* contract. No credential is ever injected: an extension that needs a token runs td auth token view. A token the user exported themselves is inherited like any other variable, which the spec records as a deliberate exception, and there is now a test pinning that behaviour so it cannot change by accident.

Stack: #521 core · #522 discovery · #523 dispatch · #524 tools · #525 install · #526 upgrade · #527 manager

@doistbot doistbot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This slice wires up extension execution: arguments pass through verbatim, the host shares the child's stdio streams, exit codes propagate, and the environment gets the TD_* contract with no credentials injected. Security-wise this looks solid — argument-array spawning with no shell interpolation on POSIX, and the Windows sh -c fallback keeps user input as positional parameters, so nothing user-controlled can escape into a command.

Few things worth tightening:

  • The Windows .cmd/.bat branch can't work: spawn() without shell: true can't launch these via CreateProcess, so valid extensions resolved by discovery will always fail with EXTENSION_NOT_EXECUTABLE. Run them through cmd.exe /d /s /c instead.
  • buildExtensionEnv leaks an inherited TD_USER into nested td invocations when no --user is given — the existing test only passes because it stubs the variable to undefined. Delete ${prefix}_USER when no user is supplied, matching the extra loop's behavior.
  • Node-shebang replacement drops the interpreter's options, so directives like #!/usr/bin/node --conditions=development break. Parse the Node directive (including env -S forms) and prepend its options before the script path.

I also included a few optional follow-up notes in the details below.

Optional follow-up notes (7)
  • P3 src/lib/extensions/dispatch.test.ts:136: expect(env.TD_EXTENSION).toBe(process.env.TD_EXTENSION) can never fail: buildExtensionEnv spreads process.env, so the two are identical by construction. If the intent is "the TDC prefix doesn't leak TD_* variables", assert expect(env.TD_EXTENSION).toBeUndefined().
  • P3 src/lib/extensions/dispatch.ts:91: ExtensionEnvOptions re-declares user and extra on top of the existing DispatchOptions in ./types.ts, which already defines user and env with the same "merged last" child-environment semantics. Compose DispatchOptions (or reuse its fields) and keep one name for the child env override — the current env/extra split is an easy way for the dispatch and manager layers to drift.
  • P3 src/lib/extensions/dispatch.test.ts:112: Hardcoded 'TODOIST_API_TOKEN' duplicates the canonical TOKEN_ENV_VAR from ../auth-store.js (also re-exported by ../auth.js). Import that constant so the pass-through test stays in sync if the token env var name changes.
  • P3 src/lib/extensions/dispatch.test.ts:41: This test asserts POSIX-only behavior but has no platform guard, so running npm test on Windows fails at expect(plan.command).toBe(join(dir, 'td-shell')) because the Windows branch routes through sh. Skip it on Windows (e.g., it.skipIf(process.platform === 'win32')) or branch the expectation.
  • P3 src/lib/extensions/dispatch.ts:49: buildSpawnPlan stats the executable here via isExecutable, then readShebang opens the same path again on line 66. Since readShebang already has a file handle, it can call handle.stat() there to answer both "is this a runnable file" and "what is its shebang" in a single open. That removes the duplicate I/O and closes the TOCTOU window where the file changes between the two checks.
  • P3 src/lib/extensions/dispatch.test.ts:22: buildSpawnPlan's Windows branches have no coverage: .exe/.cmd/.bat files that should run directly, and every other script that should become sh -c '"$0" "$@"'. That sh quoting is what preserves argument passthrough on Windows, and CI runs only on Linux, so a regression there would pass silently. Add a platform-forced test (e.g., temporarily redefine process.platform as win32 and restore it) covering both branches.
  • P3 src/lib/extensions/dispatch.test.ts:195: The signal branch in dispatchExtension is untested: a child killed by a signal must yield 128+signal (e.g. 130 for SIGINT), and a regression that resolves with code ?? 0 would make the host silently report success after an extension is killed. A fixture that runs process.kill(process.pid, 'SIGKILL') (POSIX) would pin this. The three --exit-code cases at lines 196-198 only cover the normal-close path.

Share FeedbackReview Logs

Comment thread src/lib/extensions/dispatch.ts
Comment thread src/lib/extensions/dispatch.ts
Comment thread src/lib/extensions/dispatch.ts Outdated
@scottlovegrove
scottlovegrove added this pull request to stack #528 September 10, 2026 16:03
@scottlovegrove
scottlovegrove force-pushed the feat/extensions-3-dispatch branch from e1dd57e to 855d15e Compare September 10, 2026 16:11
@scottlovegrove
scottlovegrove force-pushed the feat/extensions-3-dispatch branch from 855d15e to f4a077c Compare September 10, 2026 16:25
@scottlovegrove
scottlovegrove force-pushed the feat/extensions-3-dispatch branch from f4a077c to 384d258 Compare September 10, 2026 16:51
@scottlovegrove
scottlovegrove force-pushed the feat/extensions-3-dispatch branch from 384d258 to 82671ca Compare September 10, 2026 16:56
@scottlovegrove
scottlovegrove removed this pull request from stack #528 September 10, 2026 16:57
@scottlovegrove
scottlovegrove added this pull request to stack #529 September 10, 2026 16:58
@scottlovegrove
scottlovegrove removed this pull request from stack #529 September 10, 2026 16:59
@scottlovegrove
scottlovegrove added this pull request to stack #530 September 10, 2026 16:59
@craigcarlyle

Copy link
Copy Markdown
Contributor

The execution half of the contract: arguments reach the extension exactly
as the user typed them, the three standard streams are the host's own, and
the host exits with whatever the extension exited with.

Node-shebang scripts are launched with the host's own Node, which makes
them work on Windows and pins them to the Node version the CLI already
requires. Other scripts run directly on POSIX and through sh on Windows.

The environment adds the documented TD_* contract. No credential is ever
injected: an extension that needs a token runs `td auth token view`.
Review feedback on the dispatch slice:

- run a .cmd or .bat through the command interpreter. A batch file is not
  a program, so spawning it directly fails on every supported Node
  version, which would have made those extensions unrunnable
- keep the options a node shebang asked for, including the env -S form.
  A script that needs --conditions to resolve its imports was losing it
- clear an inherited user variable when no --user was given, so a nested
  call does not keep acting as the account the outer call chose
- ask for the file's type and its shebang through one open, which also
  removes the window in which the file could change between two checks
- the child environment override is now the same field name the manager
  already uses

Adds coverage for the Windows branches by declaring the platform, for
node shebang forms, and for a child killed by a signal.
@scottlovegrove
scottlovegrove force-pushed the feat/extensions-3-dispatch branch from 82671ca to 5482db5 Compare September 11, 2026 09:44
@scottlovegrove scottlovegrove added the 👀 Show PR PR must be reviewed before or after merging label Sep 11, 2026
@scottlovegrove
scottlovegrove merged commit 07fd458 into next Sep 11, 2026
6 checks passed
@scottlovegrove
scottlovegrove deleted the feat/extensions-3-dispatch branch September 11, 2026 10:03
scottlovegrove added a commit that referenced this pull request Sep 11, 2026
Slice 1 of 7. Replaces #520, which was one 3.7k-line PR; this is the
same work split along doistbot's suggested plan, with its review
feedback already applied.

The vocabulary every later slice builds on: the types describing an
installed extension, the naming rules derived from the host binary name
(`td` gives `td-<name>` directories, `td-extension.json` and
`.td-manifest.json`), one parser shared by install sources and by git
remotes read back off disk, manifest reading and writing including the
`package.json` fallback for Node extensions, and per-extension state
kept outside the extension directory so that deleting that directory by
hand is still a clean uninstall.

Nothing is wired up yet. See [the
spec](https://github.com/Doist/todoist-cli/blob/main/docs/specs/extensions.md)
for where this is going.

**Stack:** #521 core · #522 discovery · #523 dispatch · #524 tools ·
#525 install · #526 upgrade · #527 manager
scottlovegrove added a commit that referenced this pull request Sep 11, 2026
Slice 2 of 7, on top of slice 1.

Discovery walks the extensions directory and works out what each entry
is: a symlink or path file is a local install, a `.git` directory is a
clone, anything else is a release binary. It runs on every invocation of
the CLI, including the common case of no extensions at all, so it costs
one `readdir` plus a couple of small file reads and never spawns a
process or touches the network. The git remote is read out of
`.git/config` rather than by asking git.

Also adds the test fixture helpers, which build real extension
directories in a temporary location so later slices can install and run
something genuine rather than a mock.

**Stack:** #521 core · #522 discovery · #523 dispatch · #524 tools ·
#525 install · #526 upgrade · #527 manager
scottlovegrove added a commit that referenced this pull request Sep 11, 2026
Slice 4 of 7, on top of slice 3.

The outside world an install has to talk to, plus the one operation that
needs no installer.

- A subprocess wrapper that captures a bounded tail of output, since npm
runs extension lifecycle scripts that could otherwise print without end.
- Git operations, with working-tree state reported as clean, dirty or
unknown, so that "could not tell" is never mistaken for "safe to
delete".
- npm dependency installation, run without the CLI's own credentials,
invoked through the command interpreter on Windows because a `.cmd` shim
cannot be spawned directly on supported Node versions, and without
writing a lockfile into a clone that has none.
- The GitHub client: releases, asset downloads and checksum
verification.
- Removal, which deletes only the link for a local install and refuses a
clone with uncommitted or unreadable state unless forced.

**Stack:** #521 core · #522 discovery · #523 dispatch · #524 tools ·
#525 install · #526 upgrade · #527 manager
scottlovegrove added a commit that referenced this pull request Sep 11, 2026
Slice 5 of 7, on top of slice 4.

Installs from a release binary, a git clone, or a local directory.

Everything is assembled in a staging directory and moved into place only
once complete, with the previous install moved aside rather than
deleted, so a failure part-way through leaves the working extension
recoverable. The trust warning is printed before anything is downloaded,
cloned or executed.

Release assets are matched by platform and architecture, verified
against the release's checksums when it publishes any, and recorded in a
manifest that carries the author's description and version requirement.

**Stack:** #521 core · #522 discovery · #523 dispatch · #524 tools ·
#525 install · #526 upgrade · #527 manager
scottlovegrove added a commit that referenced this pull request Sep 11, 2026
Slice 6 of 7, on top of slice 5.

Clones fast-forward, release binaries are re-downloaded when the tag
moves, and local installs are left alone because the user's own
directory is already the source of truth.

Pins are honoured unless forced, and a forced upgrade clears the pin it
just moved past rather than leaving a stale one to skip every later
upgrade. Dependencies are reinstalled only when `package.json` or the
lockfile changed. Upgrading many extensions bounds how many talk to
GitHub at once, and warns about trust once for the whole run rather than
once per extension.

**Stack:** #521 core · #522 discovery · #523 dispatch · #524 tools ·
#525 install · #526 upgrade · #527 manager
scottlovegrove added a commit that referenced this pull request Sep 11, 2026
Slice 7 of 7, on top of slice 6. This completes the library half of
phase 1.

Ties the slices together behind `createExtensionManager`, which is the
only thing a host CLI needs to call.

Includes the version-range check behind an extension's `requires` field.
A range the running CLI does not satisfy is a warning rather than a
refusal: upgrading the CLI should not silently break an extension that
still works.

Also records the subsystem in `CODEBASE.md`, including why nothing in
the directory imports from the rest of the repo.

Next, in a separate PR: the `td extension` command group, the
`src/index.ts` wiring, the command-token lookup change, `--user`
scoping, doctor checks and `SKILL_CONTENT`.

**Stack:** #521 core · #522 discovery · #523 dispatch · #524 tools ·
#525 install · #526 upgrade · #527 manager
@doist-release-bot

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 5.4.0-next.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

released on @next 👀 Show PR PR must be reviewed before or after merging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants