Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions plugins/aidd-telemetry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,15 +135,20 @@ tokens; turning tokens into money is a separate service's job.
- **Codex needs one interactive approval.** Its hook trust is per entry, and a headless run
never sees the prompt — so a Codex session journals nothing until someone approves once,
in an interactive session, and says nothing while it does not.
- **OpenCode misses a server process's first session**, and `opencode run` is always a first
session. Measured, not asserted: one live `opencode 1.14.20` run (2026-08-31, started with
`--print-logs`) shows the plugin's own event hook firing for roughly 38 events of other
types, and its debug log shows `session.created` genuinely published on the bus after the
plugin loaded — yet it never reached the hook. Two further runs neither confirm nor refute
this: one without debug logging, one that captured no plugin events at all — see
`scripts/__tests__/fixtures/README.md`, "OpenCode's plugin events" for exactly what
each run shows. `session.idle` (the turn-end signal) is unaffected and reaches every
session.
- **OpenCode never announces a session, so the plugin opens it.** Measured, not asserted:
one live `opencode 1.14.20` run (2026-08-31, started with `--print-logs`) shows the
plugin's own event hook firing for roughly 38 events of other types, and its debug log
shows `session.created` genuinely published on the bus after the plugin loaded — yet it
never reached the hook. Two further runs neither confirm nor refute this: one without
debug logging, one that captured no plugin events at all — see
`scripts/__tests__/fixtures/README.md`, "OpenCode's plugin events" for exactly what each
run shows. `session.idle` (the turn-end signal) is unaffected and reaches every session.
Since a session nobody announced would otherwise leave the journal with no run file — and
so drop the turn-end and every task declaration after it, for every `opencode run` there
has ever been — the first call a session produces opens it, carrying the directory that
call was already going to use. What is lost is only what `session.created` alone could
have said: on a server serving more than one directory, a session it never announced is
journalled under the plugin's own init-time directory rather than its own.
- **A task is declared from a tool call's own arguments, on every host now.** Claude Code,
Codex, Copilot and Cursor each hand their hook a tool call whose own arguments can name a
file under a task folder — a `Read`, a `Bash` command line, an object keyed `path` — and
Expand Down
69 changes: 59 additions & 10 deletions plugins/aidd-telemetry/hooks/opencode-plugin.js
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,12 @@ function runJournal(event, payload) {
// delivered" - see the fixtures' own README entry for that run's log and two further
// attempts that could not distinguish the two.
//
// Every `opencode run` invocation is a first session, so this cache stays empty for it in
// practice today, and `session.idle`/`message.part.updated` fall back to `input.directory`
// below - this plugin's own init-time directory, which happens to be correct for the
// single-directory case `opencode run` is. The plugin README already named this limit
// ("OpenCode misses a server process's first session"); this comment is the same fact,
// now anchored to a measurement rather than left as an assertion.
// Every `opencode run` invocation is a session OpenCode never announced, so nothing fills
// this cache from `session.created` there, and `session.idle`/`message.part.updated` fall
// back to `input.directory` below - this plugin's own init-time directory, which is
// correct for the single-directory case `opencode run` is. `journalCallsFor` writes that
// same directory back into this cache when it opens such a session, so the session is
// opened once and every later event reads the directory its own opening used.
const directoryBySessionId = new Map();

// Mirrors `lib/task-declared.cjs`'s own `TASK_PATH_PATTERN` and `lib/host.cjs`'s
Expand Down Expand Up @@ -118,8 +118,16 @@ function declaredTaskCallFor(event, sessionDirectories, fallbackDirectory) {
/** One OpenCode event in, the journal call it produces out - or `null` for an event this
* plugin does not act on. Pure but for the one map mutation `session.created` makes on
* its way through: kept separate from `runJournal`'s spawn so a captured event can be
* asserted against without running node as a child process. */
export function journalCallFor(event, sessionDirectories, fallbackDirectory) {
* asserted against without running node as a child process.
*
* Reached as a property of the plugin below, never as a second named export. OpenCode's
* loader calls every function-valued export of a file in `plugin/` as a plugin factory of
* its own: measured against opencode 1.14.20 in a freshly installed project, this function
* exported beside the plugin was called with one argument, returned `null`, and `opencode
* run` died reading `.auth` off it before any session started - the tool this plugin
* measures, unusable in every project the framework had installed. A property is invisible
* to that loader, and a non-function export would have been ignored by it too. */
function journalCallFor(event, sessionDirectories, fallbackDirectory) {
if (event.type === "session.created") {
const sessionId = event.properties.info.id;
const cwd = event.properties.info.directory;
Expand All @@ -137,9 +145,50 @@ export function journalCallFor(event, sessionDirectories, fallbackDirectory) {
return null;
}

/** The session id an event names, whichever field its own type carries it in. */
function sessionIdOf(event) {
if (event.type === "session.created") return event.properties.info.id;
return event.properties.sessionID;
}

/** Every journal call one OpenCode event produces, in the order the journal must receive
* them - empty for an event this plugin does not act on.
*
* `journalCallFor` alone leaves `opencode run` measuring nothing. OpenCode publishes
* `session.created` on its own bus and never delivers it to a plugin's event hook
* (measured, 2026-08-31 - see plugins/aidd-telemetry/README.md, "OpenCode never announces a
* session"), and `opencode run` is always such a session. So the journal
* never receives a `session_start`, never creates the run file the rest of the session
* appends to, and drops the `turn-end` and every task declaration that follows - while
* `telemetryLocalRead` declares the tool covered and `aidd telemetry read`, which reads
* only sessions the run journal knows, can never find one. A declaration with nothing
* behind it, the same fault this plugin's own second export was.
*
* So the first call for a session nobody announced opens it. The `session-start` carries
* the directory that following call was already going to use - never a new guess: for a
* session no `session.created` named, that is this plugin's own init-time directory,
* exactly what `journalCallFor` already hands `turn-end` and `tool-used`. An announced
* session is untouched, and no session is opened twice. */
function journalCallsFor(event, sessionDirectories, fallbackDirectory) {
const sessionId = sessionIdOf(event);
const announced = sessionId !== undefined && sessionDirectories.has(sessionId);
const call = journalCallFor(event, sessionDirectories, fallbackDirectory);
if (call === null) return [];
if (announced || call.script === "session-start") return [call];
sessionDirectories.set(sessionId, call.payload.cwd);
const opening = { tool: "opencode", session_id: sessionId, cwd: call.payload.cwd };
return [{ script: "session-start", payload: opening }, call];
}

export const AiddTelemetry = async (input) => ({
event: async ({ event }) => {
const call = journalCallFor(event, directoryBySessionId, input.directory);
if (call) runJournal(call.script, call.payload);
for (const call of journalCallsFor(event, directoryBySessionId, input.directory)) {
runJournal(call.script, call.payload);
}
},
});

// The spawn-free test seams, hung off the one export rather than standing beside it - see
// journalCallFor's own comment for the measurement that rules out a second export.
AiddTelemetry.journalCallFor = journalCallFor;
AiddTelemetry.journalCallsFor = journalCallsFor;
12 changes: 10 additions & 2 deletions scripts/__tests__/aidd-telemetry-cost-skill.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -222,9 +222,17 @@ test("the plugin README gives every partly-measurable tool its reason, not just
assert.ok(readme.includes(tool), `${tool} is named`);
assert.ok(readme.includes(reason), `${tool}'s reason, not just its name`);
}
// OpenCode's limit shrank rather than vanished: the plugin now opens a session OpenCode
// never announced, so a run is journalled and readable - but a session it never announced
// is journalled under the plugin's own directory, which is only right when the server
// serves one. Both halves are pinned: the fact, and what it still costs.
assert.ok(
readme.includes("OpenCode misses a server process's first session"),
"OpenCode's own remaining limit is named, not silently dropped once it could declare a task"
readme.includes("OpenCode never announces a session"),
"OpenCode's unannounced session is named, not silently dropped once it could declare a task"
);
assert.ok(
readme.includes("journalled under the plugin's own init-time directory"),
"what an unannounced session still costs is named, not left as a solved problem"
);
});

Expand Down
118 changes: 113 additions & 5 deletions scripts/__tests__/aidd-telemetry-opencode-payloads.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,16 +30,124 @@ function loadFixture(name) {
// `opencode-plugin.js` as CommonJS and choke on its `export` syntax. OpenCode's own loader
// does not consult that field at all - the extension is the only thing that differs from
// what ships.
let journalCallForPromise;
async function journalCallFor() {
if (!journalCallForPromise) {
let pluginModulePromise;
async function pluginModule() {
if (!pluginModulePromise) {
const twin = path.join(fs.mkdtempSync(path.join(os.tmpdir(), "aidd-opencode-payloads-")), "opencode-plugin.mjs");
fs.copyFileSync(PLUGIN_SOURCE, twin);
journalCallForPromise = import(pathToFileURL(twin).href).then((mod) => mod.journalCallFor);
pluginModulePromise = import(pathToFileURL(twin).href);
}
return journalCallForPromise;
return pluginModulePromise;
}

async function journalCallFor() {
return (await pluginModule()).AiddTelemetry.journalCallFor;
}

async function journalCallsFor() {
return (await pluginModule()).AiddTelemetry.journalCallsFor;
}

// OpenCode loads every function-valued named export of a file in `plugin/` as a plugin
// factory of its own. Measured live against opencode 1.14.20 in a freshly installed
// project: a second such export returning `null` killed `opencode run` with
// `TypeError: null is not an object (evaluating 'S.auth')` before any session started, so
// installing this framework made the tool it measures unusable. A non-function export is
// ignored by that same loader, which is why the spawn-free seam this file needs rides on
// the plugin function as a property instead of standing beside it as a second export.
test("the plugin file exports one plugin factory, never a second one OpenCode would call", async () => {
const exported = await pluginModule();
const factories = Object.keys(exported).filter((name) => typeof exported[name] === "function");
assert.deepEqual(factories, ["AiddTelemetry"]);
});

// `opencode run` is always a session OpenCode never announced: measured, `session.created`
// is published on its own bus and never reaches a plugin's event hook (plugins/aidd-telemetry
// /README.md, "OpenCode misses a server process's first session"). Without these four cases the
// journal receives a turn-end for a run file that was never created, drops it, and every
// OpenCode session reads back as nothing at all - while the tool is declared covered.
test("session.idle for a session nobody announced opens it first, so the journal has a run file to write into", async () => {
const calls = await journalCallsFor();
const idle = loadFixture("opencode-session-idle.json");

const produced = calls(idle, new Map(), "/home/user/fallback");

assert.deepEqual(
produced.map((call) => call.script),
["session-start", "turn-end"],
);
assert.deepEqual(
produced.map((call) => call.payload.cwd),
["/home/user/fallback", "/home/user/fallback"],
);
assert.equal(detectHost(produced[0].payload), "opencode");
});

test("a task declaration in a session nobody announced opens it first too, never arriving before its own run file", async () => {
const calls = await journalCallsFor();
const part = loadFixture("opencode-tool-part-completed.json");

const produced = calls(part, new Map(), "/home/user/probe/project-opencode-task");

assert.deepEqual(
produced.map((call) => call.script),
["session-start", "tool-used"],
);
assert.equal(produced[0].payload.session_id, produced[1].payload.session_id);
});

test("a session already announced is never opened a second time", async () => {
const calls = await journalCallsFor();
const sessionDirectories = new Map();

calls(loadFixture("opencode-session-created.json"), sessionDirectories, "/home/user/fallback");
const produced = calls(
loadFixture("opencode-session-idle.json"),
sessionDirectories,
"/home/user/fallback",
);

assert.deepEqual(
produced.map((call) => call.script),
["turn-end"],
);
});

test("a session OpenCode did announce produces its one session-start, never a doubled one", async () => {
const calls = await journalCallsFor();

const produced = calls(
loadFixture("opencode-session-created.json"),
new Map(),
"/home/user/fallback",
);

assert.deepEqual(
produced.map((call) => call.script),
["session-start"],
);
});

test("a second session.idle on one already-opened session adds no second session-start", async () => {
const calls = await journalCallsFor();
const idle = loadFixture("opencode-session-idle.json");
const sessionDirectories = new Map();

calls(idle, sessionDirectories, "/home/user/fallback");
const produced = calls(idle, sessionDirectories, "/home/user/fallback");

assert.deepEqual(
produced.map((call) => call.script),
["turn-end"],
);
});

test("an event this plugin does not act on opens no session either", async () => {
const calls = await journalCallsFor();

assert.deepEqual(calls({ type: "message.updated", properties: {} }, new Map(), "/x"), []);
});

test("session.idle, captured live, turns into a turn-end call the journal recognises as opencode", async () => {
const builder = await journalCallFor();
const event = loadFixture("opencode-session-idle.json");
Expand Down