feat(knowledge): folder operations for the Knowledge block, and folders as search scope - #7394
feat(knowledge): folder operations for the Knowledge block, and folders as search scope#7394mzxchandra wants to merge 11 commits into
Conversation
…rs as search scope
Makes knowledge folders reachable from a workflow. The application layer already
knew how to list, create, relocate and delete them, and a v2 route already
exposed all four, but nothing between the Agent block and that layer existed:
no tools, no executor admission, no block fields.
New operations: List Folders, Create Folder, Move Folder, Delete Folder. List
answers "what is in here" - subfolders and knowledge bases together, direct
children by default, the whole subtree under Include Subfolders, subject to Max
Depth and Search. Its entries are a discriminated union on kind, so a consumer
narrows before reaching for the fields only one side has, and the listing is
capped with a truncated flag rather than being unbounded now that it carries
knowledge bases.
Search gains a Folder above its picker. A folder is a SCOPE: with no knowledge
base picked it stands for the knowledge bases in that folder, resolved when the
workflow runs, so one added later is searched without editing the block. Naming
a knowledge base is the narrower answer and wins outright - the two are never
unioned, because "this folder, specifically this one" searching more than either
alone is the opposite of what a scope means. The picker only offers that
folder's knowledge bases, so the narrower answer is the only one that travels.
The folder travels on the INTERNAL search body only. v1 declares its own
v1KnowledgeSearchBodySchema and v2 rebuilds from v1's shape, so neither inherits
this; only knowledgeSearchModeSchema is shared. Expansion reuses the existing
KNOWLEDGE_SEARCH_COST_POLICY.maxKnowledgeBases of 20 rather than introducing a
second answer to how many knowledge bases one search may span, and because a
folder scope can start breaching that cap through someone else's work, the error
names the folder and the count instead of quoting a bare bound. An empty folder
returns empty results while still running the workspace authorization check -
pointing at an empty folder must not be a way around it. Subfolders are opt-in
here, the inverse of the File block, so a folder means its own knowledge bases
unless asked otherwise.
The workspace root is the case worth reading twice. `resolveFolderPathFromIndex`
reports it as `null`, so a descending root scope must not go through `folderIds`:
that is an `inArray` over folder ids and no id equals SQL NULL, so enumerating
every folder would silently drop every knowledge base sitting loose at the root -
a "search everything" that quietly searched less. Root-with-subfolders is
therefore no folder filter at all, and root-without is `folderId: null`.
Executor admission, one caller per widening, since an authz widening with no
caller in the diff is not reviewable:
knowledge.folders.list <- knowledge_list_folders
knowledge.folders.create <- knowledge_create_folder
knowledge.folders.relocate <- knowledge_update_folder
knowledge.folders.delete <- knowledge_delete_folder
knowledge.list <- knowledge_list_folders, which answers with the
knowledge bases in a folder as well as its
subfolders, so it calls both use cases
The four folder operations take a new executor-only policy rather than the
existing copilot+executor constant. `visibility: 'user-only'` on the recursive
delete flag is an editor-role concept that holds for the agent block's model
(`createLLMToolSchema` withholds it) but collapses on Copilot's surface, which
publishes through `createUserToolSchema` and withholds only `hidden`. Granting
copilot would therefore have handed a model a cascade delete. Copilot manages
knowledge folders through `knowledge.vfs.folders.manage`, which is purpose-built
for it and stays untouched. `knowledge.list` keeps copilot because it already
had it.
No data backfill is needed - capabilities, operation ids, principal policies and
tool ids are all in-memory here, and the DB surfaces involved
(permission_group.config, folder_id, parent_id) are default-permissive or
nullable-means-root.
Dispatch is in-process through the existing executeKnowledgeTool rather than the
v2 route, matching the direction staging set when it deleted
/api/tools/file/manage. Folder expansion returns whole rows and the search
context consumes them, so a folder-scoped search does not re-read each knowledge
base it just loaded.
Path handling is the other part worth reviewing. Knowledge paths are canonical
and percent-encoded, and parseFolderPath rejects any other spelling, so a
contract-validated path is byte-identical to what buildFolderPath produces and
matching on the string is exact. Everywhere a hierarchy is walked -
selectKnowledgeDirectoryEntries, the folder-scope expansion, the picker's own
filtering - it walks parentId rather than comparing paths, so a folder genuinely
named "Q3/Q4" is one level and not two. The pure listing module is tested against
exactly that name.
Also carries one line the File slice this branch stacks on still needs: its
`SimFolderTreeSelector` barrel export landed there, but the sub-block test that
mocks that barrel wholesale was not given the matching stub, so the real control
and its React Query, socket-room and store dependencies were dragged into the
test graph.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Greptile SummaryThis PR exposes knowledge-folder operations to workflows and allows folders to act as dynamically resolved search scopes.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains in the previously reviewed paths. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| apps/sim/lib/knowledge/application/search.ts | Adds authorized runtime expansion of folder scopes and consistently validates search limits before target selection. |
| apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/knowledge-base-selector/knowledge-base-selector.tsx | Filters available knowledge bases by folder while preserving existing selections across loading, unresolved, and out-of-scope states. |
| apps/sim/blocks/blocks/knowledge.ts | Adds folder-operation fields and parameter mapping while making a specifically selected knowledge base override the broader folder scope. |
| apps/sim/lib/knowledge/application/operations.ts | Connects folder operations to the authorized knowledge application layer. |
| apps/sim/tools/knowledge/folders.ts | Defines workflow-facing folder tools and their validated inputs and outputs. |
| apps/sim/lib/api/contracts/knowledge/folders.ts | Defines bounded contracts for listing, creating, moving, and deleting knowledge folders. |
Sequence Diagram
sequenceDiagram
participant Workflow as Workflow / Agent block
participant Tool as Knowledge tool dispatcher
participant UseCase as Authorized knowledge use case
participant Folder as Folder service
participant Search as Knowledge search
Workflow->>Tool: Search with knowledgeBaseId or folderPath
Tool->>UseCase: Validate input and resolve workspace context
UseCase->>UseCase: Authorize workspace access
alt Specific knowledge base selected
UseCase->>Search: Search selected knowledge base
else Folder scope selected
UseCase->>Folder: Resolve folder knowledge bases
Folder-->>UseCase: Authorized knowledge-base set
UseCase->>Search: Search resolved set
end
Search-->>Workflow: Ranked results
Reviews (10): Last reviewed commit: "Merge remote-tracking branch 'origin/fea..." | Re-trigger Greptile
…he picker honest Greptile review round 1. Folder expansion ran inside `resolveContext`, which the authorized use-case wrapper calls BEFORE it authorizes the workspace. The fan-out error names the folder and its exact knowledge-base count, so a caller who could not read the workspace could submit an asserted workspace id plus a guessed folder path and read the difference between "no such folder", "folder with N knowledge bases", and access denied. Expansion now happens in `execute`, which only runs after authorization, and the cap check moved with it. The regression test asserts the ordering the only way it is observable: with permission denied, neither the folder index nor the knowledge-base query is ever read. The knowledge-base picker kept an already-chosen knowledge base listed even once the folder scope stopped containing it, on the reasoning that a chip needs its label. That is not neutral here: the block's params transformer treats a present knowledge base as the narrower answer and drops the folder, so the run searched a knowledge base the editor was no longer showing while displaying a folder scope that never reached the wire. An out-of-scope selection is now cleared, and a knowledge base that has not loaded yet is kept rather than cleared, because absence of data is not evidence of being out of scope. Declaration-level block comments converted to TSDoc per the repository comment convention; statement-level comments inside function bodies are left alone.
… a blank search cubic review round 1. An empty folder scope returned its empty result AFTER tag filters were resolved, so a folder-only search carrying tagFilters resolved them against zero knowledge bases and failed a search that is documented to come back empty. The early return now sits directly after the fan-out check, ahead of both tag-filter resolution and embedding generation. `search` on the folder listing used a bare `.min(1)`, which accepts whitespace. The listing reads a blank search as no search at all and returns every entry, so " " quietly became an unfiltered listing instead of the empty match asked for. Trimmed before the emptiness check. `knowledgeBaseId` and `folderPath` now say in their own descriptions that exactly one of them is required, which is what the generated integration docs render.
…r erasing a valid pick Greptile review round 2. Both findings are regressions from round 1's fixes. Moving folder expansion out of `resolveContext` left the folder branch returning before the shared `topK` check, so a folder-scoped search accepted a fractional limit that an otherwise identical knowledge-base-scoped search rejected. topK bounds the request, not the way its target was chosen, so the check now runs ahead of either branch. The out-of-scope cleanup added in round 1 could not tell "the folder query has not landed yet" from "this folder holds nothing". The knowledge-base query can resolve first, and during that window every selection looked out of scope, so reopening a workflow with a persisted folder and knowledge base could erase the knowledge base and silently widen the next run to the whole folder. The scope is now an explicit state rather than a nullable set: `pending` offers everything instead of flashing an empty picker, `unresolved` offers nothing because a path naming no folder selects no knowledge bases, and only `resolved` is allowed to erase a selection — a path that resolves to nothing is more likely a typo than a reason to discard the user's pick.
|
@cursor review |
|
Skipping Bugbot: Bugbot is disabled for this repository. Visit the Bugbot dashboard to update your settings. |
|
@cubic review |
@mzxchandra I have started the AI code review. It will take a few minutes to complete. |
Greptile review round 3, and a reversal of the approach two rounds back. The cleanup effect erased a selection the folder scope did not contain. Round 2 guarded it against the folder query still loading. Greptile then pointed out that `useFolders` sets `placeholderData: keepPreviousData`, so `isLoading` stays false through a refetch and a stale tree still reads as resolved. Adding an `isFetching` guard would not have closed it either. Knowledge-base folders have no invalidation broadcast — `useResourceFolders` documents this — so a folder created by a workflow run is absent from the client cache until the stale time elapses, with no refetch in flight and nothing to distinguish that from a correct answer. A knowledge base in that folder would still have been erased. Three rounds, three variants of the same fault: the data is not trustworthy enough to delete a user's selection from. So the erase is gone. The picker still narrows its options to the folder, an already-chosen knowledge base stays listed so its chip keeps a label, and the precedence is stated where the choice is made: "pick one and only it is searched." Nothing is silently dropped — the canvas card names the knowledge base that will actually be searched, which is the same one the run uses.
|
@cubic review |
@mzxchandra I have started the AI code review. It will take a few minutes to complete. |
…ibuting an empty search cubic review round 2. `folderScope` named only the basic half of the canonical pair, so a folder typed into the advanced Folder Path read as no folder at all: the picker offered knowledge bases the folder excludes, and picking one then took precedence over a scope the editor was still displaying. `folderScope` now carries `manualFieldId` as well, and the selector reads whichever half is filled — only one ever is. An empty folder scope searched no knowledge base, so there is none to attribute a search to, but `afterSuccess` still emitted `platform.knowledge_base.searched` with the empty id and inflated the per-base counts with a base that was never read. Skipped when no knowledge base was searched. The `list_folders` card read "List a folder" on an untouched card, which names the placeholder rather than what the operation does — listing from the workspace root is the default, not a missing value. Now an always-on "List folders" with the folder as an optional clause, matching how the other blocks phrase an operation whose scope is optional.
|
@cubic review |
@mzxchandra I have started the AI code review. It will take a few minutes to complete. |
… guards that never fired Greptile review round 4. An empty folder returned before building the provenance registry fallback the populated path builds. `prepareModelInputProvenance` returns no registry when the caller sent no provenance envelope, and the internal dispatch rejects a response without one — so the documented empty result became a 500 for exactly the callers that omit the envelope. The end-to-end check passed only because the executor always sends one. The same early return reported `rerankerStatus: 'not_requested'` when reranking had in fact been requested and simply had no candidates. That reports the caller's own configuration back to them incorrectly; `skipped` is what the populated no-candidate path returns. Two guards named things their evaluators never see: - the picker read the folder value as a raw string, missing the legacy array and JSON-array forms a persisted value can take. It now uses `readFolderPath`, the same parser the block's params transformer uses, so the picker and the run cannot scope differently. - the Include Subfolders condition tested `searchFolderRef`, a canonical param id. Conditions evaluate raw subblock values, so it matched nothing and the switch rendered on every search — the dead control it was added to prevent. It now tests whichever half of the pair is filled.
…o feat/knowledge-folder-tools
|
@cubic review |
@mzxchandra I have started the AI code review. It will take a few minutes to complete. |
…o feat/knowledge-folder-tools # Conflicts: # apps/sim/tools/generated/tool-metadata.ts
The generated artifact conflicted on merge because both branches added tools to the same single-line file. Regenerated rather than hand-merged, and verified in sync with tool-metadata:check.
|
@cubic review |
@mzxchandra I have started the AI code review. It will take a few minutes to complete. |
…o feat/knowledge-folder-tools # Conflicts: # apps/sim/blocks/types.ts
There was a problem hiding this comment.
No issues found across 32 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
Re-trigger cubic
Summary
Makes knowledge folders reachable from a workflow. The application layer already knew how to list, create, relocate and delete them, and a v2 route already exposed all four, but nothing between the Agent block and that layer existed: no tools, no executor admission, no block fields.
Stacked on #7388. This branch targets
feat/file-folder-operations, notstaging, because it imports three things that only exist there:readFolderPathfromsim-folder-tree-selector/selection.ts, theuseResourceFoldershook, andfolderScope/resourceTypeonSubBlockConfig. Against staging this is 71 files / 7060 lines and re-proposes #7388 wholesale; stacked it is 31 files / 2950 lines of only the Knowledge slice. It cannot merge until #7388 does.New operations. List Folders, Create Folder, Move Folder, Delete Folder. List answers "what is in here" — subfolders and knowledge bases together, direct children by default, the whole subtree under Include Subfolders, subject to Max Depth and Search. Entries are a discriminated union on
kind, so a consumer narrows before reaching for the fields only one side has, and the listing is capped with atruncatedflag rather than being unbounded now that it carries knowledge bases.Folders as search scope. Search gains a Folder above its picker. With no knowledge base picked, the folder stands for the knowledge bases inside it, resolved when the workflow runs — so one added later is searched without editing the block. Naming a knowledge base is the narrower answer and wins outright; the two are never unioned, because "this folder, specifically this one" searching more than either alone is the opposite of what a scope means.
The folder travels on the internal search body only. v1 declares its own
v1KnowledgeSearchBodySchemaand v2 rebuilds from v1's shape, so neither inherits this — onlyknowledgeSearchModeSchemais shared. Expansion reuses the existingKNOWLEDGE_SEARCH_COST_POLICY.maxKnowledgeBasesof 20 rather than introducing a second answer to how many knowledge bases one search may span. An empty folder returns empty results while still running the workspace authorization check.The workspace root is worth reading twice.
resolveFolderPathFromIndexreports it asnull, so a descending root scope must not go throughfolderIds: that is aninArrayover folder ids and no id equals SQLNULL, so enumerating every folder would silently drop every knowledge base sitting loose at the root — a "search everything" that quietly searched less. Root-with-subfolders is therefore no folder filter at all; root-without isfolderId: null.Executor admission, one caller per widening:
knowledge.folders.listknowledge_list_foldersknowledge.folders.createknowledge_create_folderknowledge.folders.relocateknowledge_update_folderknowledge.folders.deleteknowledge_delete_folderknowledge.listknowledge_list_folders, which answers with the knowledge bases in a folder as well as its subfoldersThe four folder operations take a new executor-only policy rather than the existing copilot+executor constant.
visibility: 'user-only'on the recursive delete flag holds for the agent block's model (createLLMToolSchemawithholds it) but collapses on Copilot's surface, which publishes throughcreateUserToolSchemaand withholds onlyhidden. Granting copilot would have handed a model a cascade delete. Copilot manages knowledge folders throughknowledge.vfs.folders.manage, which stays untouched.No data backfill. Capabilities, operation ids, principal policies and tool ids are all in-memory here, and the DB surfaces involved (
permission_group.config,folder_id,parent_id) are default-permissive or nullable-means-root. Verified against the File slice precedent, which touched nothing underpackages/db.Test Coverage
Tests: 2891 → 2892 files. 122 files / 1922 tests pass in the affected suites.
The
Q3/Q4slash-in-name trap is covered on both the pure-listing side and the block-composition side, and the root-scope branch fixed in review now has two dedicated tests.Pre-Landing Review
28 findings from 6 specialists plus Claude and Codex adversarial passes. 5 critical, all addressed:
inArrayover folder ids, which never matchesfolderId IS NULL. Fixed: root+descending is now no folder filter at all.user-onlydid not keep a model away from the recursive delete — found by security, Claude adversarial and Codex, all three ranking it top. Fixed with an executor-only operation policy; the tool comment now states precisely what the flag does and does not buy.depthsilently ignored withoutrecursive(now refused by contract and gated in the editor so the combination cannot be built),Recursiverenamed toInclude Subfoldersto match the search field,Destination Parent Path→Move Into Path, the subfolder switch no longer renders before a folder is chosen, and the delete count no longer reports the deleted folder as one of its own casualties.Known and accepted, documented rather than fixed:
list_foldersreads the workspace's folders and knowledge bases and applieslimitin memory, matching the File slice'sfile_list; and a recursive folder delete can archive knowledge bases thatknowledge.deletedoes not admit the executor for — reachable only when a human ticks the guard.Design Review
6 findings, 2 critical (copy-vs-behavior mismatch and the empty-state message), both addressed above. Field order is pinned by test: the Folder renders above the picker it narrows.
Eval Results
No prompt-related files changed — evals skipped.
Scope Drift
Scope Check: CLEAN. All 31 changed files map to a plan section. No file outside the plan's footprint.
Plan Completion
27/33 done, 2 changed (goal met differently), 4 outstanding at audit time — three of which were closed during review (contracts barrel export, the
folderPathdispatch-seam assertion, and direct operation tests). The remaining one is adescribe('Knowledge block')inblocks.test.ts; that coverage lives inknowledge-folders.test.tsinstead.Verification Results
Verified end to end against live dev servers, driven from the workflow editor and confirmed in Postgres:
SupportcreatedQ3/Q4created as one folder, not two levelstruncated: false, correct union shapeEmptyFolderrelocated underArchiveresults: [], not an errorcompleted, chunk embedded via live provider keyTest plan
tsc --noEmitcleanbun run check:audits)check:api-validationandcheck:client-boundarypasstool-metadata:check,docs:check)🤖 Generated with Claude Code