Skip to content

Commit d236a68

Browse files
waleedlatif1claude
andcommitted
fix: address the review findings raised on the v0.8.21 release PR
Eight of the thirteen threads were real. Each was verified against source before changing anything; three were pushed back on and are unchanged. Knowledge and credential groups: - The connector Access field resolved its provider through the standard-OAuth subset, which excludes Slack — Slack collects accounts through a custom bot. The field never rendered for a Slack connector, so it could not enter members mode and, worse, a per-member Slack connector had no way back to workspace mode. Resolved across all credential-group providers instead. - The v1 document delete looked the document up with an ACL scope and then ran an unscoped delete. The access-aware path already existed and v2 already used it; v1 was the last surface on the old one. The window is small and not attacker-controllable, but the divergence is worth closing. - Enrollment surfaced `CredentialGroupEnrollmentError` as a bare 500. A missing, disabled, or unconfigured credential group is the admin's to act on, so the policy now projects its 404/409 the way the credential-group routes do. - The workspace-level member-connector listing used knowledge-base concealment and answered "Knowledge base not found" where its siblings return an authorization response. It names a workspace, not a base, so it now uses an unconcealed policy — the convention the policy file already documents. Home: - Restoring a queued Build message left the search query in the URL, and the rule that forces Search whenever a query is present flipped the composer straight back. The edit was discarded and the original message dispatched. Clearing the query alongside the mode restore batches into the same nuqs update, so the forcing rule never observes the intermediate state. Docs and tooling: - `redis.mdx` claimed completed work is unaffected by losing Redis. Webhook idempotency markers live wherever the cache does, with a 7-day TTL sized to the longest provider retry window, and the bundled Redis runs without persistence — so a redelivery after a restart can re-run a finished workflow with its real side effects. Billing, checkout, and Chat-send idempotency are pinned to PostgreSQL. Corrected the same claim in the chart's values. - `/ship` Phase A never regenerated the docs manifest that Phase B hard-gates on, so adding or renaming a docs page aborted the command. - The CLI updater prints a yarn command, but the upgrade tabs offered none. - Documented that Ask may reach an integration for an explicitly requested action, not only when sources cannot answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
1 parent 67e3f6d commit d236a68

10 files changed

Lines changed: 96 additions & 24 deletions

File tree

apps/docs/content/docs/platform/self-hosting/redis.mdx

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,9 @@ Sim uses Redis as a message bus and shared cache. Both deployments ship it by de
2424
With more than one app or realtime replica and no `REDIS_URL`, users on different pods stop seeing each other's edits and live status updates. Beyond one startup log line noting single-pod mode, nothing is logged — the app looks healthy and quietly loses events. Treat Redis as mandatory the moment `replicaCount` exceeds 1.
2525
</Callout>
2626

27-
Everything Sim keeps in Redis is cache, coordination state, or an in-flight event — never committed data, which lives in PostgreSQL and object storage. Persistence is therefore not required. Losing or restarting the instance is not free, though: cancellation markers and the cross-pod half of execution streaming live here, so active runs stop streaming and a cancellation issued across the gap may not land. Completed work is unaffected.
27+
Everything Sim keeps in Redis is cache, coordination state, or an in-flight event — never committed data, which lives in PostgreSQL and object storage. Losing or restarting the instance costs active runs their streaming, and a cancellation issued across the gap may not land.
28+
29+
It also costs webhook deduplication. Webhook idempotency markers live wherever the cache does, with a 7-day TTL sized to the longest provider retry window, so a redelivery arriving after a restart can re-run a workflow that already completed — with its real side effects. Billing, checkout, and Chat-send idempotency are pinned to PostgreSQL and are never at risk. The bundled Redis runs without persistence, which is fine for coordination state; if duplicate webhook side effects would be unacceptable for your deployment, point `REDIS_URL` at a managed instance with persistence enabled.
2830

2931
## Configuration
3032

apps/sim/app/api/knowledge/member-connectors/route.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ export const GET = defineInternalJsonRoute({
1313
auth: internalSessionAuth,
1414
operation: knowledgeOperations.listWorkspaceMemberConnectors,
1515
rateLimit: internalRateLimits.none({ reason: 'One small read per visit to the Search tab' }),
16-
errorPolicy: internalKnowledgeErrorPolicies.connectors,
16+
errorPolicy: internalKnowledgeErrorPolicies.memberConnectors,
1717
mapInput: ({ query }) => ({ workspaceId: query.workspaceId }),
1818
useCase: listWorkspaceMemberConnectors,
1919
present: ({ connectors }) => ({ success: true as const, data: connectors }),

apps/sim/app/api/v1/knowledge/[id]/documents/[documentId]/route.ts

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -113,11 +113,12 @@ export const DELETE = withRouteHandler(
113113
)
114114
if (result instanceof NextResponse) return result
115115

116-
const doc = await getKnowledgeDocument(
117-
knowledgeBaseId,
118-
documentId,
119-
await resolveV1KnowledgeAccessScope(userId, rateLimit, parsed.data.query.workspaceId)
116+
const access = await resolveV1KnowledgeAccessScope(
117+
userId,
118+
rateLimit,
119+
parsed.data.query.workspaceId
120120
)
121+
const doc = await getKnowledgeDocument(knowledgeBaseId, documentId, access)
121122

122123
if (!doc) {
123124
return NextResponse.json({ error: 'Document not found' }, { status: 404 })
@@ -130,6 +131,7 @@ export const DELETE = withRouteHandler(
130131
workspaceId: parsed.data.query.workspaceId,
131132
},
132133
document: { id: documentId, filename: doc.filename },
134+
access,
133135
userId,
134136
source: 'api',
135137
requestId,

apps/sim/app/workspace/[workspaceId]/home/home.tsx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -550,9 +550,10 @@ export function Home({ chatId, userName, userId }: HomeProps) {
550550
*/
551551
const restoreQueuedMode = useCallback(
552552
(requestMode: QueuedMessage['requestMode']) => {
553+
setSearchQuery('')
553554
void setComposerMode(requestMode === 'ask' ? 'assistant' : 'build')
554555
},
555-
[setComposerMode]
556+
[setComposerMode, setSearchQuery]
556557
)
557558

558559
/** An emptied search box returns to the sources; a send in any other mode has no search to clear. */

apps/sim/app/workspace/[workspaceId]/home/types.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,8 @@ export interface FileAttachmentForApi {
2626
/**
2727
* A request mode a send asks the agent for beyond the default. `ask` is an
2828
* Assistant turn: an answer drawn from the attached knowledge bases first,
29-
* with a connected integration reached only when those cannot answer.
29+
* with a connected integration reached only when those cannot answer — live or
30+
* very recent data, or an action the person asked for outright.
3031
*/
3132
export type ChatRequestMode = 'ask'
3233

apps/sim/app/workspace/[workspaceId]/knowledge/[id]/hooks/use-connector-member-group-options.ts

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
import { useMemo } from 'react'
44
import type { ComboboxOption } from '@sim/emcn'
55
import {
6-
type CredentialGroupStandardOAuthProvider,
6+
type CredentialGroupProvider,
7+
getCredentialGroupProviderFromProviderId,
78
getCredentialGroupProviderId,
8-
getCredentialGroupStandardOAuthProviderFromProviderId,
99
isCredentialGroupProvider,
1010
} from '@/lib/credential-groups/providers'
1111
import type { ConnectorMeta } from '@/connectors/types'
@@ -30,13 +30,18 @@ export function decodeConnectorMemberGroupOption(
3030
}
3131
}
3232

33-
/** The credential-group provider that collects accounts for this connector, if any. */
33+
/**
34+
* The credential-group provider that collects accounts for this connector, if any.
35+
* Resolves across every credential-group provider, not just the standard-OAuth
36+
* subset — Slack collects accounts through a custom bot and would otherwise
37+
* resolve to none, hiding the Access field.
38+
*/
3439
function connectorMemberGroupProvider(
3540
connectorConfig: ConnectorMeta
36-
): CredentialGroupStandardOAuthProvider | null {
41+
): CredentialGroupProvider | null {
3742
if (connectorConfig.auth.mode !== 'oauth' || !connectorConfig.permissionScopedListing) return null
3843
try {
39-
return getCredentialGroupStandardOAuthProviderFromProviderId(connectorConfig.auth.provider)
44+
return getCredentialGroupProviderFromProviderId(connectorConfig.auth.provider)
4045
} catch {
4146
return null
4247
}

apps/sim/lib/knowledge/api/route-policies.ts

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import {
1010
v2OrchestrationErrorPolicy,
1111
} from '@/lib/api/server/routes'
1212
import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits'
13+
import { CredentialGroupEnrollmentError } from '@/lib/credential-groups/enrollments'
1314
import { KNOWLEDGE_DELEGATION_AUDIENCE } from '@/lib/knowledge/application/authorization'
1415
import { KnowledgeUsageLimitExceededError } from '@/lib/knowledge/application/billing'
1516
import { KnowledgeDocumentNotReadyError } from '@/lib/knowledge/application/chunk-errors'
@@ -109,7 +110,23 @@ export const internalKnowledgeErrorPolicies = {
109110
tags: concealKnowledgeBase(
110111
internalKnowledgeErrorPolicy('Failed to process knowledge tag request')
111112
),
112-
connectors: concealKnowledgeBase(internalKnowledgeErrorPolicy('Internal server error')),
113+
/**
114+
* Enrollment reaches the credential-group helpers, whose failures are the
115+
* admin's to act on — a missing group, a disabled one, or one with no active
116+
* account option — rather than a bare 500.
117+
*/
118+
connectors: concealKnowledgeBase(
119+
extendInternalErrorPolicy(internalKnowledgeErrorPolicy('Internal server error'), (error) =>
120+
error instanceof CredentialGroupEnrollmentError
121+
? internalErrorResponse(error.status, { error: error.message })
122+
: null
123+
)
124+
),
125+
/**
126+
* Workspace-scoped, like the bulk routes: the request names a workspace, not
127+
* one knowledge base, so there is no resource whose existence a 403 betrays.
128+
*/
129+
memberConnectors: internalKnowledgeErrorPolicy('Failed to fetch member connectors'),
113130
uploads: concealKnowledgeBase(internalKnowledgeUploadErrorPolicy),
114131
} as const
115132

apps/sim/lib/knowledge/orchestration/documents.test.ts

Lines changed: 41 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ const {
88
mockCaptureServerEvent,
99
mockCreateDocumentRecords,
1010
mockCreateSingleDocument,
11-
mockDeleteDocument,
11+
mockDeleteKnowledgeDocumentInKnowledgeBase,
1212
mockGetDocumentByUploadId,
1313
mockMarkDocumentAsFailedTimeout,
1414
mockProcessDocumentAsync,
@@ -21,7 +21,7 @@ const {
2121
mockCaptureServerEvent: vi.fn(),
2222
mockCreateDocumentRecords: vi.fn(),
2323
mockCreateSingleDocument: vi.fn(),
24-
mockDeleteDocument: vi.fn(),
24+
mockDeleteKnowledgeDocumentInKnowledgeBase: vi.fn(),
2525
mockGetDocumentByUploadId: vi.fn(),
2626
mockMarkDocumentAsFailedTimeout: vi.fn(),
2727
mockProcessDocumentAsync: vi.fn(),
@@ -47,7 +47,7 @@ vi.mock('@/lib/core/telemetry', () => ({
4747
vi.mock('@/lib/knowledge/documents/service', () => ({
4848
createDocumentRecords: mockCreateDocumentRecords,
4949
createSingleDocument: mockCreateSingleDocument,
50-
deleteDocument: mockDeleteDocument,
50+
deleteKnowledgeDocumentInKnowledgeBase: mockDeleteKnowledgeDocumentInKnowledgeBase,
5151
getDocumentByUploadId: mockGetDocumentByUploadId,
5252
markDocumentAsFailedTimeout: mockMarkDocumentAsFailedTimeout,
5353
processDocumentAsync: mockProcessDocumentAsync,
@@ -58,6 +58,7 @@ vi.mock('@/lib/knowledge/documents/service', () => ({
5858
vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mockCaptureServerEvent }))
5959

6060
import { OrchestrationError } from '@/lib/core/orchestration/types'
61+
import { type KnowledgeAccessScope, WORKSPACE_ACCESS_TOKENS } from '@/lib/knowledge/access/types'
6162
import {
6263
performDeleteKnowledgeDocument,
6364
performMarkKnowledgeDocumentTimedOut,
@@ -75,6 +76,7 @@ const FILE = {
7576
mimeType: 'application/pdf',
7677
}
7778
const ACTOR = { userId: 'user-1', source: 'agent' as const, requestId: 'req-1' }
79+
const ACCESS: KnowledgeAccessScope = { kind: 'workspace', tokens: WORKSPACE_ACCESS_TOKENS }
7880

7981
/**
8082
* Lets the fire-and-forget dispatch settle. Both upload paths queue indexing
@@ -403,31 +405,64 @@ describe('performUpdateKnowledgeDocument', () => {
403405
describe('performDeleteKnowledgeDocument', () => {
404406
beforeEach(() => {
405407
vi.clearAllMocks()
406-
mockDeleteDocument.mockResolvedValue({ success: true, message: 'ok' })
408+
mockDeleteKnowledgeDocumentInKnowledgeBase.mockResolvedValue(undefined)
407409
})
408410

409411
it('audits the deletion against the acting user', async () => {
410412
const outcome = await performDeleteKnowledgeDocument({
411413
...ACTOR,
412414
knowledgeBase: KB,
413415
document: { id: 'doc-1', filename: 'report.pdf', fileSize: 10, mimeType: 'application/pdf' },
416+
access: ACCESS,
414417
})
415418

416419
expect(outcome).toMatchObject({ success: true })
417-
expect(mockDeleteDocument).toHaveBeenCalledWith('doc-1', 'req-1')
418420
expect(mockRecordAudit).toHaveBeenCalledWith(
419421
expect.objectContaining({ actorId: 'user-1', resourceId: 'doc-1' })
420422
)
421423
expect(mockCaptureServerEvent).toHaveBeenCalled()
422424
})
423425

426+
it("re-applies the caller's access at the delete itself", async () => {
427+
await performDeleteKnowledgeDocument({
428+
...ACTOR,
429+
knowledgeBase: KB,
430+
document: { id: 'doc-1', filename: 'report.pdf' },
431+
access: ACCESS,
432+
})
433+
434+
expect(mockDeleteKnowledgeDocumentInKnowledgeBase).toHaveBeenCalledWith(
435+
'kb-1',
436+
'doc-1',
437+
'req-1',
438+
ACCESS
439+
)
440+
})
441+
442+
it('reports not_found when the scoped delete finds nothing to delete', async () => {
443+
mockDeleteKnowledgeDocumentInKnowledgeBase.mockRejectedValue(
444+
new OrchestrationError('not_found', 'Document not found')
445+
)
446+
447+
const outcome = await performDeleteKnowledgeDocument({
448+
...ACTOR,
449+
knowledgeBase: KB,
450+
document: { id: 'doc-1', filename: 'report.pdf' },
451+
access: ACCESS,
452+
})
453+
454+
expect(outcome).toMatchObject({ success: false, errorCode: 'not_found' })
455+
expect(mockRecordAudit).not.toHaveBeenCalled()
456+
})
457+
424458
it('emits no telemetry when the delete fails', async () => {
425-
mockDeleteDocument.mockRejectedValue(new Error('deadlock detected'))
459+
mockDeleteKnowledgeDocumentInKnowledgeBase.mockRejectedValue(new Error('deadlock detected'))
426460

427461
const outcome = await performDeleteKnowledgeDocument({
428462
...ACTOR,
429463
knowledgeBase: KB,
430464
document: { id: 'doc-1', filename: 'report.pdf' },
465+
access: ACCESS,
431466
})
432467

433468
expect(outcome).toMatchObject({ success: false, errorCode: 'internal' })

apps/sim/lib/knowledge/orchestration/documents.ts

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@ import type { BillingAttributionSnapshot } from '@/lib/billing/core/billing-attr
55
import { OrchestrationError } from '@/lib/core/orchestration/types'
66
import { PlatformEvents } from '@/lib/core/telemetry'
77
import { generateRequestId } from '@/lib/core/utils/request'
8+
import type { KnowledgeAccessScope } from '@/lib/knowledge/access/types'
89
import { dispatchDocumentProcessing } from '@/lib/knowledge/documents/processing-dispatch'
910
import {
1011
createDocumentRecords,
1112
createSingleDocument,
1213
type DocumentData,
13-
deleteDocument,
14+
deleteKnowledgeDocumentInKnowledgeBase,
1415
getDocumentByUploadId,
1516
markDocumentAsFailedTimeout,
1617
type ProcessingOptions,
@@ -440,6 +441,11 @@ export async function performUpdateKnowledgeDocument(
440441
export interface PerformDeleteKnowledgeDocumentParams extends KnowledgeOperationContext {
441442
knowledgeBase: KnowledgeBaseTarget
442443
document: { id: string; filename: string; fileSize?: number; mimeType?: string }
444+
/**
445+
* Re-applied at the delete itself, so an access change landing between the
446+
* caller's lookup and this write cannot still delete the document.
447+
*/
448+
access: KnowledgeAccessScope
443449
}
444450

445451
export type PerformDeleteKnowledgeDocumentResult = KnowledgeOrchestrationResult
@@ -448,11 +454,11 @@ export type PerformDeleteKnowledgeDocumentResult = KnowledgeOrchestrationResult
448454
export async function performDeleteKnowledgeDocument(
449455
params: PerformDeleteKnowledgeDocumentParams
450456
): Promise<PerformDeleteKnowledgeDocumentResult> {
451-
const { knowledgeBase, document, request, source } = params
457+
const { knowledgeBase, document, request, source, access } = params
452458
const requestId = params.requestId ?? generateRequestId()
453459

454460
try {
455-
await deleteDocument(document.id, requestId)
461+
await deleteKnowledgeDocumentInKnowledgeBase(knowledgeBase.id, document.id, requestId, access)
456462
} catch (error) {
457463
return classifyKnowledgeFailure(error, requestId, `Delete document ${document.id}`)
458464
}

helm/sim/values.yaml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -597,7 +597,10 @@ redis:
597597
pullPolicy: IfNotPresent
598598

599599
# No persistence is configured: Redis holds coordination state and short-lived
600-
# keys, so a restart costs in-flight live updates, not committed data.
600+
# keys, so a restart costs in-flight live updates, not committed data. It also
601+
# drops webhook idempotency markers, so a provider redelivery after a
602+
# restart can re-run an already-completed workflow — use a persistent managed
603+
# instance if that matters. Billing and checkout idempotency is on PostgreSQL.
601604
maxmemory: "512mb"
602605
maxmemoryPolicy: "noeviction"
603606

0 commit comments

Comments
 (0)