diff --git a/.agents/skills/api-doc/references/openapi-patterns.md b/.agents/skills/api-doc/references/openapi-patterns.md index 30817e9903..83ca28e0e8 100644 --- a/.agents/skills/api-doc/references/openapi-patterns.md +++ b/.agents/skills/api-doc/references/openapi-patterns.md @@ -85,9 +85,13 @@ Rules: | `/request-queues` | GET | `requestQueues_get` | | `/request-queues/{queueId}` | GET | `requestQueue_get` | | `/request-queues/{queueId}` | PUT | `requestQueue_put` | -| `/acts/{actorId}/runs` | POST | `act_runs_post` | -| `/acts/{actorId}/runs` | GET | `act_runs_get` | -| `/acts/{actorId}/runs/{runId}` | GET | `act_run_get` | +| `/actors/{actorId}/runs` | POST | `actors_runs_post` | +| `/actors/{actorId}/builds` | GET | `actors_builds_get` | +| `/actors/{actorId}/runs/last` | GET | `actor_runs_last_get` | + +Uniqueness beats the singular rule, because the operation ID becomes the page slug. GET `/actors/{actorId}/runs` can't be `actor_runs_get` - the account-wide GET `/actor-runs` already owns the `actor-runs-get` slug and the two are different scopes. So Actor-scoped `/runs` and `/builds` use plural `actors_`, while `runs/last` has no clash and stays singular. Matching redirects: `nginx.conf`. + +Never rewrite the `act_*` fragments in `x-legacy-doc-urls` or in-description links pointing at them. They're backward-compatibility anchors, not operation ID references. `docusaurus.config.js` turns every `x-legacy-doc-urls` hash into a `data-altids` sidebar attribute, and `redirectOpenApiDocs()` in `apify-docs-theme/static/js/custom.js` matches an incoming `#tag/` or `#/reference/` hash against those altids - an unmatched hash hard-bounces to `/search?...¬-found=1`. Rewrite `act_version_put` to `actor_version_put` and the old link breaks, because no page carries the new id. ## Code sample examples diff --git a/.github/workflows/openapi-ci.yaml b/.github/workflows/openapi-ci.yaml index 55d5493c74..fc02c93c87 100644 --- a/.github/workflows/openapi-ci.yaml +++ b/.github/workflows/openapi-ci.yaml @@ -84,6 +84,11 @@ jobs: - name: Validate JSON bundle run: pnpm exec redocly lint static/api/openapi.json + # Guards against docs teaching routes the contract doesn't define - notably the legacy + # `/v2/acts` alias, which responds but is absent from the published spec (#2804). + - name: Check documented API paths exist in the spec + run: pnpm test:api-paths + - name: Check bundle sizes run: | JSON_SIZE=$(stat -f%z static/api/openapi.json 2>/dev/null || stat -c%s static/api/openapi.json) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ccdcb69ac4..d16011210e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -272,7 +272,7 @@ Examples: - `/requests-queues` GET -> `requestQueues_get` - `/requests-queues/{queueId}` PUT -> `requestQueue_put` -- `/acts/{actorId}/runs` POST -> `act_runs_post` +- `/actors/{actorId}/runs` POST -> `actors_runs_post` #### Code samples @@ -337,6 +337,8 @@ Add languages by adding new folders at the appropriate path level. - **Broken links**: [Periodic GitHub Action](.github/workflows/lychee.yml) checks broken links by [lychee](https://lychee.cli.rs/). If the Action fails, we manually fix the issues. +- **API paths**: `pnpm test:api-paths` checks that every `/v2/...` route written in `sources/` exists in the bundled OpenAPI spec and isn't marked `deprecated`, so the docs can't teach a route the contract doesn't define or has already superseded. It runs in the [OpenAPI checks](.github/workflows/openapi-ci.yaml) `validate` job on every pull request. Build the bundle first (`pnpm openapi:build:json`) when running it locally. Two common failures: the legacy `/v2/acts/` prefix, which still responds but is absent from the published contract; and the Actor-scoped single-run and single-build routes such as `/v2/actors/{actorId}/runs/{runId}`, which are deprecated in favor of `/v2/actor-runs/{runId}` and `/v2/actor-builds/{buildId}`. Note that the run and build _collection_ routes (`/v2/actors/{actorId}/runs`) are current - only the singular forms moved. + - **Academy exercises**: At the end of each lesson in the academy courses, there are exercises that target real-world websites. Each exercise includes a solution, stored as a separate file containing executable code. These files are included in the docs using the `!!raw-loader` syntax. Each course has a [Bats](https://bats-core.readthedocs.io/) test file named `test.bats`. The tests run each solution as a standalone program and verify that it produces output matching the expected results. A [periodic GitHub Action](.github/workflows/test-academy.yml) runs all these tests using `pnpm test:academy`. If the Action fails, we rework the exercises. ## Pull request process diff --git a/package.json b/package.json index 25ab8b3dbe..8d91b97512 100644 --- a/package.json +++ b/package.json @@ -45,6 +45,7 @@ "format": "oxfmt", "format:check": "oxfmt --check", "test:academy": "bats --print-output-on-failure -r .", + "test:api-paths": "node ./scripts/checkApiPaths.mjs", "test:llms-size": "node ./scripts/checkLlmsSize.mjs", "test:nav-headers": "node ./scripts/checkNavHeaders.mjs", "postinstall": "patch-package", diff --git a/scripts/checkApiPaths.mjs b/scripts/checkApiPaths.mjs new file mode 100644 index 0000000000..2133567315 --- /dev/null +++ b/scripts/checkApiPaths.mjs @@ -0,0 +1,169 @@ +// Fail when a documented `/v2/...` API path is absent from the OpenAPI contract (issue #2804). +// +// The `/v2/acts/...` prefix still responds - apify-core registers it as an alias to the same +// controllers - but it is not in the published contract at api.apify.com/v2/openapi.json, which +// only defines `/v2/actors` and `/v2/actor-runs`. An agent that generates a client from the spec +// therefore gets a different contract than an agent that copies a route out of our docs. This +// script closes that gap by making the docs prove every route they teach exists in the spec. +// +// Reads the bundled spec off disk rather than fetching it, so the check runs against the spec in +// the same commit as the docs. In CI the `validate` job already has the bundle as an artifact. +// +// Usage: +// pnpm openapi:build:json && pnpm test:api-paths +// node scripts/checkApiPaths.mjs [specPath] + +import { glob, readFile } from 'node:fs/promises'; +import { relative, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const ROOT = fileURLToPath(new URL('..', import.meta.url)); +const SPEC = process.argv[2] ? resolve(process.argv[2]) : resolve(ROOT, 'static/api/openapi.json'); + +// `sources/api/` is gitignored output generated from the spec itself, so it can only ever agree +// with the spec - and it legitimately carries the documented `/v2/acts/` compatibility notice. +const SOURCES = 'sources/**/*.{md,mdx}'; +const SKIP_DIR = 'sources/api/'; + +// Only `api.apify.com` serves our v2 API. Other hosts have their own unrelated `/v2/` namespaces +// (registry.hub.docker.com/v2/repositories/... is a real, valid link in the tree). +const API_HOST = 'api.apify.com'; + +// Trailing hostname in the text before `/v2/`, e.g. `https://api.apify.com`. A bare `/v2/...` with +// no host is still checked - the most agent-visible legacy route this issue was filed over lived +// in a bare code span in the agent-onboarding quick-reference table, so a host-anchored match +// would have passed while missing the one occurrence that mattered most. +const TRAILING_HOST = /([A-Za-z0-9][A-Za-z0-9.-]*\.[A-Za-z]{2,})$/; + +// Where a path reference ends. Markdown and code fences wrap them in all of these. +const PATH_CHARS = /[A-Za-z0-9{}$@%!*+,:;.=_~/-]/; + +const spec = JSON.parse(await readFile(SPEC, 'utf8')); +const specPaths = Object.keys(spec.paths ?? {}); +if (specPaths.length === 0) throw new Error(`No paths found in ${SPEC} - is the bundle built?`); + +// Existing-in-the-spec is necessary but not sufficient. `/v2/actors/{actorId}/runs/{runId}` is in +// the contract yet carries `deprecated: true` - "endpoints related to run of the Actor were moved +// under new namespace actor-runs" - so a docs page can teach a contract-valid route that we've +// already superseded. Treat that as drift too, otherwise the check blesses the deprecated form. +const deprecatedPaths = new Set( + specPaths.filter((p) => Object.values(spec.paths[p]).some((op) => op?.deprecated === true)), +); + +// A doc path matches a spec path when they have the same number of segments and every spec segment +// is either a `{template}` (any concrete value the docs put there - `apify~web-scraper`, +// `ACTOR_ID`, a real id) or literally equal. +const specSegments = specPaths.map((p) => ({ path: p, segs: p.split('/') })); + +// All spec paths a doc path could be. Templates make this genuinely ambiguous: a docs page writing +// `/v2/actors/X/runs/last` matches both `/v2/actors/{actorId}/runs/last` and the deprecated +// `/v2/actors/{actorId}/runs/{runId}`, so callers must not assume a single match. +function matchSpecPaths(docPath) { + const docSegs = docPath.split('/'); + return specSegments + .filter( + ({ segs }) => + segs.length === docSegs.length && segs.every((seg, i) => seg.startsWith('{') || seg === docSegs[i]), + ) + .map(({ path }) => path); +} + +// Template interpolation leaves fragments a static check can't resolve: `${storeId}`, +// f-string `{os.environ[...]}`, or a literal ellipsis standing in for the rest of a route. +// These aren't drift, so skip them rather than reporting them. +function isUnresolvable(docPath) { + if (docPath.includes('$') || docPath.includes('...')) return true; + return docPath.split('/').some((seg) => { + const open = (seg.match(/{/g) ?? []).length; + const close = (seg.match(/}/g) ?? []).length; + return open !== close; + }); +} + +// Returns the `/v2/...` path at `index`, or null when the reference should not be checked. +function extractRef(line, index) { + const before = line.slice(0, index); + + // A relative or absolute link into our own API reference pages - `/api/v2/actor-get` is a + // Docusaurus slug, not an API route. This is the largest false-positive class by far. + if (before.endsWith('/api')) return null; + + const host = before.match(TRAILING_HOST)?.[1]; + if (host && host !== API_HOST) return null; + + let end = index + '/v2/'.length; + while (end < line.length && PATH_CHARS.test(line[end])) end += 1; + + // Trailing punctuation belongs to the prose, not the path. + const path = line.slice(index, end).replace(/[.,:;/-]+$/, ''); + if (path === '/v2') return null; + + return path; +} + +const missing = []; +const deprecated = []; +let checked = 0; +let skipped = 0; + +for await (const entry of glob(SOURCES, { cwd: ROOT })) { + const file = entry.split('\\').join('/'); + if (file.startsWith(SKIP_DIR)) continue; + + const lines = (await readFile(resolve(ROOT, entry), 'utf8')).split('\n'); + + lines.forEach((line, i) => { + for (let at = line.indexOf('/v2/'); at !== -1; at = line.indexOf('/v2/', at + 1)) { + const path = extractRef(line, at); + if (!path) continue; + if (isUnresolvable(path)) { + skipped += 1; + continue; + } + checked += 1; + + const matches = matchSpecPaths(path); + if (matches.length === 0) { + missing.push(`${file}:${i + 1} ${path}`); + continue; + } + + // Only drift when every candidate is deprecated - if any current path also matches, + // the docs are on a supported route. + if (matches.every((m) => deprecatedPaths.has(m))) { + deprecated.push(`${file}:${i + 1} ${path} (matches deprecated ${matches.join(', ')})`); + } + } + }); +} + +// The counts are load-bearing output, not decoration: a regex regression that stops matching would +// otherwise show up as a silent pass. If `checked` drops toward zero, the extractor is broken. +console.log( + `Checked ${checked} API path reference(s) in sources/**/*.{md,mdx} against ${specPaths.length}` + + ` paths in ${relative(ROOT, SPEC)}` + + `\n(${skipped} skipped as template interpolation)\n`, +); + +if (missing.length > 0) { + console.error(`āŒ ${missing.length} documented API path(s) are absent from the OpenAPI contract:\n`); + for (const f of missing) console.error(` ${f}`); + console.error( + '\nUse the canonical route from the spec. The `/v2/acts/...` prefix is a deprecated alias' + + '\nthat still responds but is not part of the published contract - see issue #2804.', + ); +} + +if (deprecated.length > 0) { + console.error(`\nāŒ ${deprecated.length} documented API path(s) are deprecated in the OpenAPI contract:\n`); + for (const f of deprecated) console.error(` ${f}`); + console.error( + "\nThese exist but are superseded. Read the operation's description in the spec for the" + + '\nreplacement - Actor-scoped run and build routes moved to `/v2/actor-runs` and' + + '\n`/v2/actor-builds`.', + ); +} + +if (missing.length > 0 || deprecated.length > 0) process.exit(1); + +console.log('āœ… Every documented API path exists in the OpenAPI contract and none are deprecated'); diff --git a/sources/academy/tutorials/api/run_actor_and_retrieve_data_via_api.md b/sources/academy/tutorials/api/run_actor_and_retrieve_data_via_api.md index 98520d5e3f..490b226502 100644 --- a/sources/academy/tutorials/api/run_actor_and_retrieve_data_via_api.md +++ b/sources/academy/tutorials/api/run_actor_and_retrieve_data_via_api.md @@ -27,7 +27,7 @@ If the Actor being run via API takes 5 minutes or less to complete a typical run > If you are unsure about the differences between an Actor and a task, you can read about them in the [tasks](/actors/running/tasks) documentation. In brief, tasks are pre-configured inputs for Actors. -The API endpoints and usage (for both sync and async) for [Actors](/api/v2#tag/ActorsRun-collection/operation/act_runs_post) and [tasks](/api/v2/actor-task-runs-post) are essentially the same. +The API endpoints and usage (for both sync and async) for [Actors](/api/v2/actors-runs-post) and [tasks](/api/v2/actor-task-runs-post) are essentially the same. To run, or **call**, an Actor/task, you will need a few things: @@ -45,7 +45,7 @@ The URL of [POST request](https://developer.mozilla.org/en-US/docs/Web/HTTP/Meth https://api.apify.com/v2/actors/ACTOR_NAME_OR_ID/runs?token=YOUR_TOKEN ``` -For tasks, we can switch the path from **acts** to **actor-tasks** and keep the rest the same: +For tasks, we can switch the path from **actors** to **actor-tasks** and keep the rest the same: ```cURL https://api.apify.com/v2/actor-tasks/TASK_NAME_OR_ID/runs?token=YOUR_TOKEN @@ -239,7 +239,7 @@ When we run the Actor with the [usual API call](#run-an-actor-or-task) shown abo Replace the `RUN_ID` in the following URL with the ID you extracted earlier: ```cURL -https://api.apify.com/v2/actors/ACTOR_NAME_OR_ID/runs/RUN_ID +https://api.apify.com/v2/actor-runs/RUN_ID ``` Once a status of `SUCCEEDED` or `FAILED` has been received, we know the run has finished and can cancel the interval and finally [collect the data](#collect-the-data). diff --git a/sources/academy/tutorials/api/using_apify_from_php.md b/sources/academy/tutorials/api/using_apify_from_php.md index bb366e5e0d..4799b4a26c 100644 --- a/sources/academy/tutorials/api/using_apify_from_php.md +++ b/sources/academy/tutorials/api/using_apify_from_php.md @@ -59,7 +59,7 @@ The [API reference](/api/v2/actors-runs-post) states that an Actor's input shoul // To identify the Actor, you can use its ID, but you can also pass // the full Actor name [username]~[actorName] or just ~[actorName] for // your own Actors -$response = $client->post('acts/vdrmota~contact-info-scraper/runs', [ +$response = $client->post('actors/vdrmota~contact-info-scraper/runs', [ // Actors usually accept JSON as input. When using the `json` key in // a POST request's options, guzzle sets proper request headers // and serializes the array we pass in @@ -128,7 +128,7 @@ All the available parameters are described in [our API reference](/api/v2/datase Datasets are great for structured data, but are not suited for binary files like images or PDFs. In these cases, Actors store their output in [key-value stores](/storage/key-value-store). One such Actor is the **HTML String To PDF** ([mhamas/html-string-to-pdf](https://apify.com/mhamas/html-string-to-pdf)) converter. Let's run it. ```php -$response = $client->post('acts/mhamas~html-string-to-pdf/runs', [ +$response = $client->post('actors/mhamas~html-string-to-pdf/runs', [ 'json' => [ 'htmlString' => '

Hello World

' ], @@ -183,7 +183,7 @@ It takes some time for an Actor to generate its output. Some even have Actors th For Actors that are expected to be quick, we can use the `waitForFinish` parameter. Then, the running Actor's endpoint does not respond immediately but waits until the run finishes (up to the given limit). Let's try this with the HTML String to PDF Actor. ```php -$response = $client->post('acts/mhamas~html-string-to-pdf/runs', [ +$response = $client->post('actors/mhamas~html-string-to-pdf/runs', [ 'json' => [ 'htmlString' => '

Hi World

' ], @@ -217,7 +217,7 @@ $webhooks = \base64_encode(\json_encode([ 'requestUrl' => '', ], ])); -$response = $client->post('acts/mhamas~html-string-to-pdf/runs', [ +$response = $client->post('actors/mhamas~html-string-to-pdf/runs', [ 'json' => [ 'htmlString' => '

Hello World

' ], diff --git a/sources/platform/get-started/agent-onboarding.md b/sources/platform/get-started/agent-onboarding.md index 7fa94453bf..069f884fdd 100644 --- a/sources/platform/get-started/agent-onboarding.md +++ b/sources/platform/get-started/agent-onboarding.md @@ -255,9 +255,9 @@ For HTTP-native integrations or languages without a dedicated client. Base URL: | Action | Method | Endpoint | | :--- | :--- | :--- | | [Search Actors in Store](/api/v2/store-get) | `GET` | `/v2/store` | -| [Get Actor details](/api/v2/actor-get) | `GET` | `/v2/acts/{actorId}` | -| [Run an Actor](/api/v2/actors-runs-post) | `POST` | `/v2/acts/{actorId}/runs` | -| [Run Actor (sync, get results)](/api/v2/actor-run-sync-get-dataset-items-post) | `POST` | `/v2/acts/{actorId}/run-sync-get-dataset-items` | +| [Get Actor details](/api/v2/actor-get) | `GET` | `/v2/actors/{actorId}` | +| [Run an Actor](/api/v2/actors-runs-post) | `POST` | `/v2/actors/{actorId}/runs` | +| [Run Actor (sync, get results)](/api/v2/actor-run-sync-get-dataset-items-post) | `POST` | `/v2/actors/{actorId}/run-sync-get-dataset-items` | | [Get run status](/api/v2/actor-run-get) | `GET` | `/v2/actor-runs/{runId}` | | [Get dataset items](/api/v2/dataset-items-get) | `GET` | `/v2/datasets/{datasetId}/items` | diff --git a/sources/platform/integrations/programming/github.md b/sources/platform/integrations/programming/github.md index 90b3d62a01..2c3fc41684 100644 --- a/sources/platform/integrations/programming/github.md +++ b/sources/platform/integrations/programming/github.md @@ -55,7 +55,7 @@ After you link an Actor to a GitHub repository, add a webhook in GitHub to trigg 1. In Apify Console, open the Actor's **API** dropdown and select **API endpoints**. Copy the **Build Actor** endpoint URL. It has this format: ```text - https://api.apify.com/v2/acts/YOUR-ACTOR-NAME/builds?token=YOUR-TOKEN&version=0.0&tag=latest&waitForFinish=60 + https://api.apify.com/v2/actors/YOUR-ACTOR-NAME/builds?token=YOUR-TOKEN&version=0.0&tag=latest&waitForFinish=60 ``` :::note API token