Repository navigation
docs: add Universal Mocker walkthroughs, guides, features and reference - #995
Merged
Merged
Conversation
The Universal Mocker section had no page that lists every keploy mock command and flag. Add universal-mocker/reference, at the end of the section's sidebar. - Commands: each keploy mock command's summary, usage, flags and help examples, in the words of the released 3.9.4 CLI's own --help. Flags and commands shown only with KEPLOY_PROVEN=1 (check, explain, judge, pin, quarantine; --lock on publish, pull, record and replay, --from on publish, --freeze-binary on record, --all and --list-main on replay) are marked so. - keploy login and keploy logout, which the cloud commands need. - The keploy.yml mock: keys and the flag each one stands for, read from the config the 3.9.4 CLI loads: a flag that is passed wins over its key. Help text is kept as the CLI prints it; only a "(default X)" tail is shown as "Default: X", and a URL in it is shown as code, not as a link. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The Universal Mocker reference listed every keploy mock command, flag and keploy.yml key, but not the environment variables, the exit codes or what a replay reports. Add them, from the 3.9.4 CLI's help and code, with every output shown taken from a run of the released 3.9.4. - Environment variables: KEPLOY_API_KEY, KEPLOY_K8S_PROXY_URL, KEPLOY_REPORT_ANSWER_TIMEOUT, KEPLOY_DISABLE_AUTO_UPDATE, KEPLOY_PROVEN, KEPLOY_LIST_MAIN, KEPLOY_RUN_RESULT, and MOCK_LOCAL, MOCK_NAME and MOCK_EMITMOCKEVENTS (the keploy.yml keys local, name and emitMockEvents in capitals). A flag you pass wins over its variable. - Exit codes: the test command's own code passed on unchanged, and Keploy's 1, 3, 4, 5, 6, 8 and 128+N, with the output of a usage error (8), a run with no sign-in (5), and a test command that exits 7. What KEPLOY_RUN_RESULT holds for each, so a script can tell the test command's code from Keploy's. - Replay results: keploy mock status --json for a set after a red replay, each set's fields, the verdicts, the last replay's fields (from keploy/<set>/last-replay.yaml) and the failedBy values. The page's description and opening now point at the new sections. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add universal-mocker/troubleshooting: what keploy mock record and replay print when something goes wrong, the cause, and the fix. It starts with the keploy mock status block and its phases, then covers a call with no recording (the Hint texts, re-record, --on-miss record), an empty mock set, a replay that verified nothing, a record with the dependencies down, a busy proxy port, and the tests that pass locally but fail in CI (no sign-in, no recording in the checkout, package downloads, sudo, a CI key that can't record). Each output is from a real Keploy 3.9.4 run, trimmed; the CI key refusal is quoted from the CLI. The reference now says that keploy.yml's top-level command, which a first replay saves from -c, is what keploy mock replay runs with no -c, also without KEPLOY_PROVEN. The page is in the sidebar after the guides and linked from the overview and the reference. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add universal-mocker/guides/network-isolation.md: what keploy mock replay does with each call (a recorded connection, an unrecorded one that gets 502 Bad Gateway, a local port nothing listens on, DNS lookups with and without a recording, and UDP, ICMP and raw sockets, which a replay doesn't intercept), the flags that let a call out on purpose (--on-miss passthrough and record, --pass-through-ports, --allow-egress dns and keploy.yml's test.allowEgress), and how to tell whether a replay reached the real network (the last line, the "real egress opened by" lines, keploy mock status and last-replay.yaml). Every output is from a run of the released Keploy 3.9.4 on Linux, trimmed. The page says plainly that, in 3.9.4, a replay's last line still reads "every dependency answered from the recording" after a lookup went out with --allow-egress dns, or after a UDP datagram reached its destination. List the guide in the sidebar after keploy.yml and keploy.lock, and link it from the overview's next steps and the troubleshooting page. The Java quickstart's step 5 also says that, with the users API up, keploy mock replay --on-miss record adds only the new call, which a run of the page on 3.9.4 showed. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A new guide, guides/changing-values.md, shows what keploy mock replay
does when a test sends a value that is new on every run, from real runs
of Keploy 3.9.4 (Linux, --local, a Go suite against a local orders API):
- A replay of the whole run answers a call with a new time, request id
or body id from the recording. It also answers GET /orders/{id} for a
new id with the recorded order and counts the call as answered, so the
page tells readers to check the ids their tests get back.
- In that mode a go test run reads the real clock, with --freeze-time on
or off, so a test that checks a recorded token's expiry fails once the
token's lifetime has passed.
- With KEPLOY_PROVEN=1 the record's judging rounds learn an echoed value
and an id used again in a later call (variable places), the replay
runs each test on a clock shifted back to its recording, and a test
that still fails with every call answered is quarantined. The page
shows keploy mock explain for both cases and how to run a quarantined
test again.
The guide is listed in the sidebar after the network isolation guide,
and linked from the overview, the troubleshooting page and the network
isolation guide.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A new guide, guides/partial-re-record.md, shows how to record a new test
or re-record a changed one without running the whole suite against the
real dependencies again, from real runs of Keploy 3.9.4 (Linux, Keploy
Cloud, a Go suite against PostgreSQL and a rates API):
- With KEPLOY_PROVEN=1 and keploy.lock, a record whose -c runs one test
(go test -run) records only that test, then replays the whole
mock.command on the recordings, and keploy.lock keeps every other
entry ("kept 3, start-up kept"). The lock's diff adds or changes one
digest.
- Keploy notes that the older tests now run later than their recording,
and keeps them because the confirming round passed. The page says
when to re-record them too.
- A replay of a few tests passes the runner's filter in -c and prints a
SUBSET line; --run-only is refused with KEPLOY_PROVEN=1.
- keploy mock diff and --markdown list the tests a branch adds or
changes against the merge base.
- Without KEPLOY_PROVEN=1, a one-test --local record replaces the whole
set (10 mocks to 5), and a replay of the suite fails the other tests,
so the page says to re-record the whole suite in that mode.
The guide is listed in the sidebar after the keploy.yml and keploy.lock
guide, and linked from the overview and that guide.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A new page, universal-mocker/features.md, lists what Universal Mocker does in Keploy 3.9.4, one row per capability, each linking to the page that shows it: record and replay, per-test checks, sharing through Keploy Cloud and keploy.lock, partial re-record, network isolation, values that change, status and next steps, and platforms. Each row restates what the page it links to says, or what the reference built from the Keploy 3.9.4 CLI's help says; it describes no behaviour of its own. The sidebar lists it before Troubleshooting, and the overview's Next steps links to it. The sidebar's partial-re-record line gets the indentation prettier gives it. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add universal-mocker/walkthroughs/vscode.md, a step-by-step tour of the Keploy VS Code extension 3.1.1: install it, let it download the CLI, sign in with a code, hand the setup to your coding agent, record, replay with the dependencies off, see a red test, stage the files, wire CI and invite the team. Each step names the label, message or button the released extension shows. The screenshots aren't taken yet: each one is a marked "Screenshot coming" note that says what the image will show. The sidebar gets a Walkthroughs category after Quickstarts, and the overview links the page. Since 3.1.1 the Marketplace lists the extension as "Keploy: API Testing, Mocks & Service Virtualization", so the VS Code page, the integrations page and the llms.txt lines use that name. The llms.txt lines also stop saying the extension generates tests or runs from a panel. The VS Code page now says Wire CI writes the workflow for a project below the repository's root too, as 3.1.1 does. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add universal-mocker/walkthroughs/dashboard.md, a tour of the Keploy Cloud dashboard for recordings made with keploy mock: open Recordings and its E2E tests view, pick the Keploy branch, read a test's recorded calls and its start-up mocks, find the branch in Branch Reviews, review its added, removed and changed mocks, commit, request a review, approve or request changes, and merge with the pull request. Every label, status and message is quoted from the dashboard's code as of its November 2026 release. Five notes mark where screenshots go. List the page under Walkthroughs in the sidebar, and link it from the overview's next steps. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add universal-mocker/guides/coding-agents.md. It shows how to install Keploy's mocking skill with keploy skill install mock, for one agent or several, into the repository or the home directory, and where each agent reads it: Claude Code, Cursor, Codex, GitHub Copilot and Antigravity. Then it shows what the agent does with it: it starts with keploy mock status, relays the NEXT block's question, runs its command and loops until the replay is green. The page explains each field and phase of the block, where the recording lives, the two-part sign-in for an agent's shell, KEPLOY_PROVEN=1, and what the skill tells the agent never to do. Every command and output is from a run of Keploy 3.9.4. The sidebar, the overview and the features page link the guide. llms.txt had no Universal Mocker entries, so add a section that lists every page of it, and add the Universal Mocker to the products in llms-full.txt. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add universal-mocker/guides/dependencies.md. It says how Keploy tells one dependency from another (by the first bytes of each connection, on any port, MySQL by the server's greeting, raw bytes when no protocol matches) and how it reads connections that start with TLS, linking the common-errors entry on Keploy's CA. Then it gives what a replay does with a PostgreSQL login (the recorded TLS answer, no password check), a MySQL login (no password compared, a recorded success served over a recorded refusal), a Redis blocking pop still waiting when the recording ended, and Amazon SQS polls under KEPLOY_PROVEN=1, plus SQS on macOS. Each note is read from the code Keploy 3.9.4 is built from. The sidebar, the overview, the features page and llms.txt link it. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add universal-mocker/guides/background-workers.md. It says where Keploy records a call made while a test is open (that test's recording, whoever made it) and one made outside every test (the set's start-up recording), and, in a table, what a replay answers each kind of call with, without KEPLOY_PROVEN and with it. It shows the sessionCalls entry that serves a call the app makes once and keeps, such as a token, to every test, what an entry serves in each mode, and what each of the four session-call hints of an HTTP miss asks you to do. It ends with polls made while a test is open, and the Redis and SQS polls Keploy 3.9.4 answers itself. Each note is read from the code 3.9.4 is built from. The sidebar, the overview, the features page and llms.txt link it. The troubleshooting row for "Each test is served the HTTP calls it recorded." now links the guide, and the reference's keploy.yml keys name the top-level sessionCalls list. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add universal-mocker/guides/branches-and-prs.md. It shows how the recordings follow git through keploy.lock: a record on a branch lists the lock under a Keploy branch of the same name, keploy mock diff --markdown prints what a pull request re-records, git merges the lock with the change, and a green replay of main lists main's lock as Keploy main (on its own in CI, with --list-main on elsewhere, or with keploy mock register --main). It covers a teammate replaying a branch, two branches that change different tests merging cleanly, and two branches that re-record the same test: the conflict in keploy.lock, the error replay and record print, and how to resolve it. Every command and output is from a run of Keploy 3.9.4 against Keploy Cloud, with a bare repository standing in for the git host. The sidebar, the overview, the features page, the keploy.yml and keploy.lock guide and llms.txt link the guide. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The CI text lived in a section of the mock-your-tests page. It is now its own guide, guides/ci.md: the CI key and the repository secret, the GitHub Actions workflow the VS Code extension writes, a pull request's job and its keploy mock diff comment, the push to main that lists Keploy main, caching the recordings, what fails the job (no key, no keploy.lock, a call the recordings don't hold), re-recording on a branch with a key a person made, and what any other CI job needs. Every output is from Keploy 3.9.4 against Keploy Cloud: the workflow's run steps ran verbatim, taken from the page, on a fresh clone as a user with passwordless sudo, with no Keploy installed and every dependency down, for a pull request and for a push to main. The mock-your-tests anchor keeps a short pointer to the guide, and the quickstarts, troubleshooting, the VS Code walkthrough, the branches guide, the sidebar, the overview, the features page and llms.txt link it. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add three screenshots to the Universal Mocker VS Code walkthrough, taken from VS Code with the released Keploy extension (3.1.1 from Open VSX) and the Keploy 3.9.4 CLI it downloads on a fresh machine: - the Set up checklist while the CLI downloads (Step 1); - the download notification with its Cancel button (Step 2); - the work-branch message that a record on main shows (Step 5). The download example in Step 2 now quotes the size that run showed. In the dashboard walkthrough's merge step, say that an app whose recording pin is on follows the keploy.lock of git's main branch, and link to the merge and listing steps of the branches and pull requests guide. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Rewrite Steps 3 to 9 of the Universal Mocker VS Code walkthrough from a run of the released extension (3.1.1 from Open VSX) and the Keploy 3.9.4 CLI it downloads, and add nine screenshots from that run: - Step 3: VS Code asks to open the sign-in page, and the sign-in notification quotes the CLI's own text. Say what happens when the approval doesn't come in time. - Step 4: where each agent's skill and task go (Claude Code, GitHub Copilot, Cursor), that the task is typed in for you to send, and what VS Code says when the agent's chat doesn't open. - Step 5: the record's last lines, with a link to the recording pin. - Step 6: the Testing view shows one "Keploy Cloud" row for recordings in Keploy Cloud, and only the runs started in it. - Step 7: the red replay's terminal, and the Test Results message with its FAILED and VERDICT lines. The Testing view has no per-test "red:" rows for recordings in Keploy Cloud. - Steps 8 and 9: the staging list, Source Control and the CI workflow. Report links, app ids and the sign-in code are blurred. The browser's sign-in page and the agent's chat still show a "Screenshot coming" box. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Add quickstarts/vscode.md: the Go sample with PostgreSQL and the rates API, recorded and replayed from the Keploy VS Code extension without a coding agent. The page adds the keploytest line to each test, writes a two-line keploy.yml, pushes main, records from the extension's work-branch offer, replays with both dependencies stopped, and stages keploy.yml and keploy.lock for the commit. Every output is from a run of the page's steps with extension 3.1.1 from Open VSX and the Keploy 3.9.4 CLI it downloads, on Linux, against Keploy Cloud. List the page in the sidebar after the Docker Compose quickstart, in the overview's next steps, and in llms.txt. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The overview's next steps linked every page of the section but the reference: add it, after Troubleshooting. The VS Code walkthrough's next steps now link the VS Code quickstart, which records and replays a sample from the extension without a coding agent. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The features page and the overview said a call with no recording fails the run, so a missing mock never passes silently. Keploy 3.9.4 doesn't keep that promise: - A call with no recording gets an error, and the replay names it. By default, the run fails when a test fails on that error, or with --strict. A default run of 3.9.4 where the test only logs the 502 exits 0, and its last line says the replay did not prove the tests run with the dependencies off. The network isolation guide now shows that run and the same run with --strict, which exits 1. - A whole-run replay can answer a new id from a similar recorded call. - A DNS lookup the recording doesn't have doesn't fail a replay, and other UDP (not DNS) isn't intercepted. The features page, the overview, the on-miss table, the network isolation table and llms-full.txt now say this, and link the guides that show each case. Also: - Troubleshooting says what a record signed in with a CI key does. By default, Keploy doesn't stop it: the record runs, isn't published, logs "the recording was not published" with Keploy Cloud's reason, and exits 1. With KEPLOY_PROVEN=1, 3.9.4 stops it up front when the key's role is CI; a service-account key, like the one the CI guide makes, still records and is refused at the upload. The CI guide says the same and links it. - The reference lists exit code 2 (a Keploy crash) and the replay's own checks that exit 1. - The VS Code walkthrough's sign-in step shows the dialog and the notification cropped on their own, without the later screen around them. - The CI guide says Wire CI also handles a project in a folder below the repository's root, as the extension page does. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Universal Mocker section had quickstarts and one guide, but no reference for
keploy mock, no troubleshooting, nothing on branches, pull requests or CI, and no tour of the VS Code extension or the Keploy Cloud dashboard. Users had to read--helpand the CLI's messages to learn them.What changed
/universal-mocker/reference/): everykeploy mockcommand and flag in the words of the released 3.9.4 CLI's--help, with the ones shown only underKEPLOY_PROVEN=1marked;keploy loginandlogout; thekeploy.ymlmock:keys and the flag each stands for; the environment variables; the exit codes, including2for a Keploy crash, withKEPLOY_RUN_RESULT; and the fields ofkeploy mock status --jsonandlast-replay.yaml./universal-mocker/troubleshooting/): what a record or replay prints when something goes wrong, the cause and the fix: the status block and its phases, a call with no recording and its hints, an empty or unverified replay, a record with the dependencies down, a busy proxy port, and tests that pass locally but fail in CI./universal-mocker/features/): one row per capability, each linking to the page that shows it. The row on calls with no recording also says what a 3.9.4 replay doesn't fail on: a new id answered from a similar recorded call, a DNS lookup the recording doesn't have, and other UDP (not DNS), which a replay doesn't intercept.keploy.lock, whatkeploy mock diff --markdownshows a pull request, how main is listed after a merge, and how to resolve akeploy.lockconflict.--strictis on. It says plainly that in 3.9.4 the replay's last line still reads "every dependency answered from the recording" after a lookup went out with--allow-egress dnsor a UDP datagram reached its destination.KEPLOY_PROVEN=1, including that a whole-run replay answersGET /orders/{id}for a new id with the recorded order.sessionCalls.keploy skill install mockfor each agent, and how an agent follows the status block'sNEXTsteps./universal-mocker/quickstarts/vscode/): the Go sample recorded and replayed from the extension, without a coding agent.How it was checked
KEPLOY_PROVEN=1), which are quoted from the 3.9.4 CLI's code. The network isolation guide's run of a test that doesn't fail on its answer, with and without--strict, used--localand noKEPLOY_PROVEN. The branches, CI and partial re-record guides ran against Keploy Cloud, with a bare repository standing in for the git host. The CI guide's workflow steps ran as written, on a fresh clone with no Keploy installed and every dependency down, for a pull request and for a push to main. Those runs signed in with a user's API key, not a read-only CI key, and their pull-request comment went to a stand-in forgh; what the read-only key may do is read from Keploy Cloud's code, and what a record signed in with a CI-role or service-account key does, by default and withKEPLOY_PROVEN=1, is read from the 3.9.4 CLI's and Keploy Cloud's code.--help; the dependency and background-call notes are read from the code 3.9.4 is built from; the dashboard walkthrough is read from the dashboard's code.yarn build(onBrokenLinksandonBrokenAnchors: throw) and every verify script, on Node 22, as CI builds it; then a build of main, andverify-moved-urlsagainst its sitemap: every moved URL redirects, and every page of the base build still leads to a page.prettier@2.8.8 --checkon every changed.jsand.mdfile, as CI runs it.