Repository navigation
docs(ui-capture): privacy, sessions, network bodies, ingest keys and limits - #989
Merged
Merged
Conversation
…limits Four guides and a reference page for the UI Testing section, written for @keploy/capture-sdk 3.1.0: - ui-capture/privacy: what each privacy mode masks and what every mode still records (URL paths, data-testid as written, and under balanced query values and the body fields no rule covers), selectors, editors, Do Not Track and Global Privacy Control (on by default since 3.1.0; Brave and DuckDuckGo send GPC by default), headers, credential names and bodies, values in URLs, secretRoutes and keepRoutes, piiPatterns, PrivacyEngine, what each batch carries, and what the dashboard, REST and MCP show. - ui-capture/sessions-and-sampling: one session per page load, every way a session ends, continuous capture, /complete, sampling per page load, custom session ids, pause and hidden tabs, batching and retries, and the server's session states. - ui-capture/network-and-bodies: which calls are recorded and from when (not before rrweb has loaded), how bodies are read, the never-read marker, large-body notes, worker redaction and not-captured markers, GraphQL terms, maskNetworkBodies, streams. - ui-capture/ingest-keys: what a key can do, creating one in the dashboard or the API, one app per environment, rotation, revocation within about 30 seconds, and who can manage keys. - ui-capture/limits: every limit in the browser and on the server. The sidebar gets a Guides category and a Limits and sizes reference item, and the overview links the new pages. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Four guides and one reference page for the UI Testing section, for
@keploy/capture-sdk3.1.0. No new screenshots: the ingest-keys page reusesui-capture-key-created.webpandui-capture-revoke.webp.New:
/docs/ui-capture/privacy/(Privacy and masking). For the developer and for the privacy or security reviewer who signs off: the three modes and the input table; what strict still records (URL paths,id,class, field names, SVG geometry, anddata-testidas written: never build one from user data); mask, unmask and block selectors; rich-text editors; visitors' privacy signals:honorPrivacySignalsis on by default from 3.1.0, so visitors whose browser sends Do Not Track or Global Privacy Control are not recorded, and Brave and DuckDuckGo send GPC by default; headers, credential names and bodies, and that the rest of a body is recorded as sent underbalanced; values in URLs (balancedkeeps a query value whose name is not a credential's: the sample shops recordGET /api/products?q=kettle);secretRoutesandkeepRoutes;piiPatternsand why they never run over URLs;PrivacyEngine; what each batch carries; what the dashboard, REST and MCP show; recognition, not a guarantee. This closes the gap that the install path never said some visitors are never recorded.New:
/docs/ui-capture/sessions-and-sampling/(Sessions and sampling). A session is one page load and its in-app navigations, so a multi-page site records one session per page load; every way a session ends (maxSessionDuration,stop(), unload, the server's 30-minute window,/complete, the per-session size limit);continuous; a tab that comes back after its session was completed;sampleRaterolled per page load, never per visitor;sessionIdGenerator; pause, resume and hidden tabs; batching, retries and the last batch; the server's session states.New:
/docs/ui-capture/network-and-bodies/(Network calls and bodies). EveryfetchandXMLHttpRequestcall, a framework's own included (Next.js's?_rsc=payload fetches); network capture starts only once rrweb has loaded, so a page's first requests are often not recorded and nothing closes that window; read whole, read throughbody, and never read ([Response body not read by the page], which replay sends as the body); the eight large-body notes; worker redaction, its budget and the three not-captured markers; GraphQLrequestTerm;maskNetworkBodies; streams and uploads; turning network capture off.New:
/docs/ui-capture/ingest-keys/(Ingest keys and environments). What a key can and cannot do; creating one in the dashboard (no expiry) or the API (name,ttlDays, the reserved name); one app per environment, because journey shares count every session an app receives; rotation; revocation within about 30 seconds; who can manage keys (testgen:run; no service-account key can).New:
/docs/ui-capture/limits/(Limits and sizes, under Reference). Every limit in the browser and on the server, and what happens past it. It states no retention period for recordings: none is set in code.Also: the sidebar's UI Testing category gets a Guides category (the four guides, open like Universal Mocker's) and Limits and sizes under Reference; the overview links the ingest-keys page from its first step, the privacy page from "Privacy at a glance", and the sessions and limits pages from "Limitations".
Sources checked
Every sentence was checked against keploy/ui-capture v3.1.0 (9bae8002), keploy/api-server main (7d97ad0db), keploy/enterprise-ui main (4a5d8c5d) and keploy/samples-typescript main (6231b4e).
READMEispackages/sdk/README.mdat v3.1.0,sdk/ispackages/sdk/src/.data-testid, replay look), 589-603 (selectors, editors), 616-654 (headers, strict header list), 655-700 (names; the page-URL exception :694-698), 701-871 (bodies; text-body pairs :798-822; patterns :829-832;maskNetworkBodies:848-871), 900-963 (URLs), 964-1124 (page-URL credentials; generated-looking :1065-1076), 1125-1143 (recognition), 1144-1239 (secretRoutes,keepRoutes), 1240-1333 (shape scrub, page text tokens), 1334-1381 (patterns,piiPatterns,PrivacyEngine), 1710-1735 (what a batch carries), 2025-2033 (Brave and DuckDuckGo send GPC by default); sdk/index.ts:163-185, 438-446 (signals read, decline before rrweb); sdk/privacy.ts:31-45 (a request URL is judged by names, never by shape), 4694-4705 (PrivacyEnginearguments; user patterns on network data only); sdk/transport.ts:210-220 (gzip only when smaller), 1160-1167 (batchId); no cookie,localStorage,sessionStorageor IndexedDB use in sdk/ (grep); samplesui-capture-react-shop/src/keploy.ts:28-52(both excerpts, byte for byte) and its README:120-136 (?q=kettle); euiapp/ui-testing/[flowId]/page.tsx:112-131, 228-279(what the flow page shows); apipkg/graph/schema.graphqlsUINetworkCallandpkg/graph/authz_resource.go:506-508(stored headers and bodies,app:view); apipkg/http/apiv1/uicapture_agent.go:637-700(the agent view).continuousends only the session onsession_completed/session_full); sdk/transport.ts:1250-1253 (backoff jitter), 1815-1838 (newest dropped on re-queue); apipkg/service/uicapture/session_sweep.go:33-52, 99-120,pkg/http/apiv1/uiingest.go:146-153, 300-361,pkg/service/uicapture/sdk_ingest.go:44-47, 308-333,pkg/service/uicapture/capture_status.go:97-137,pkg/models/uicapture.go:17-21, 416-436,pkg/uicapture/sdkwire/sdkwire.go:30-31, 83-89,pkg/service/uicapture/flowengine/extract.go:135, 440-466, 580-595anddedup.go:215(flow splits; a page only loaded gives a one-step flow); samplesui-capture-html-shop/README.md:87-92.networkCapture: false); sdk/body-redact.ts:60-76 (main-thread cut); sdk/body-jobs.ts:81, 172, 686 (budgets, the not-read marker); sdk/request-term.ts:118-130 (#pq); sdk/term-candidate.ts:34, 55-75 (body:*); samplesui-capture-next-shop/README.md:118-124(?_rsc=) andui-capture-react-shop/README.md:104-110(first request).pkg/http/apiv1/uiingest.go:95-107(CORS headers), 146-153;uiingest_auth.go:36-46, 139-141, 235-240, 265, 278-330;uicapture_agent.go:135-150, 205-308, 310-360;openapi.yaml:2418-2522, 3519, 3522-3535;response.go:95-101(envelope);pkg/service/uicapture/journeys/plan.go:103-111andpkg/models/uicapture_journey_plan.go:166-169(shares of all the app's sessions, 30 days);middleware.go:263-266(last use every 5 minutes);service_account_operation_gate.go:142-159;pkg/models/roles.go:69-71, 89-95,permissions.go:336-366, 473-486, 517-620(who holdstestgen:run);pkg/models/step_up.go:71andpkg/graph/schema.resolvers.go:3105, 3206(recent sign-in on create and revoke); euicomponents/ui-testing/CaptureSetupCard.tsx:211-262, 304-345, 408-457andapp/ui-testing/page.tsx:1143-1200(when the card shows).sdkwire.go:41, 47,uiingest.go:84-85, 244-283, 314-315,uicapture_journey_plan.go:166-226,journeyplan/service.go:429-440,journeys/plan.go:66-75,uicapture_journeys.go:197-200,uicapture_agent.go:145-150, 491-500,router.go:122, 217-218, 260-265,ip_ratelimit.go:49-90(the MCP per-IP limit applies only to requests without akep_key).Line numbers are at the revisions above. No page states how long recordings are kept: no retention period is set in code (api
pkg/models/uicapture.go:420-422).Checks
On a build box, with CI's commands (node 22, yarn 1.22), on this branch's commit:
prettier@2.8.8 --checkon the changed.mdfilesstatic/yarn install --frozen-lockfileand the yarn.lock checkyarn buildonmainand on this branch: neither prints a warning or error line (onBrokenLinksandonBrokenAnchorsthrow)verify-linkedin-tag,verify-schema,verify-lastmod,verify-moved-urlsandverify-ui-capture-sdk-links(49 error sections, 5 refused verbs, 4 README links)In the built site: all 58 heading ids of the five pages and the overview exist; the five pages are in the sitemap with their titles; a rendered-text comparison of each page's source with its HTML differs only at
ProductTier, the images and code-block titles, and the errors page differs nowhere.An independent review checked every sentence against the sources above, over three rounds (the last clean).