Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
c2f5a87
feat: add a concurrency limit for named child runs
vdusek Oct 9, 2026
a20b3c7
refactor: keep the child run status max age in seconds and mark missi…
vdusek Oct 29, 2026
fc48db7
docs: document that a start waiting for a slot raises once the parent…
vdusek Oct 29, 2026
0e84cde
test: cover a recorded child run the API no longer knows under the co…
vdusek Oct 29, 2026
70fefc9
fix: count a named start already in flight when the concurrency limit…
vdusek Oct 29, 2026
5d03404
fix: count a named start that finishes while the active child runs ar…
vdusek Oct 29, 2026
158c3e6
perf: skip fetching finished child runs when counting toward the conc…
vdusek Oct 29, 2026
ca902ab
docs: add docstrings to the child run module constants
vdusek Oct 29, 2026
3447fab
docs: add docstrings to the child run registry attributes
vdusek Oct 29, 2026
a4569a7
refactor: rename the child run concurrency limit internals and method
vdusek Oct 30, 2026
403f118
Merge remote-tracking branch 'origin/master' into feat/child-run-conc…
vdusek Oct 30, 2026
206db98
refactor: use asyncio.timeout when waiting under the child run concur…
vdusek Oct 30, 2026
56f2cfe
refactor: cache child run statuses in a named tuple
vdusek Oct 30, 2026
10fc99c
fix: don't start a waiting abort_with_parent child run once the paren…
vdusek Oct 30, 2026
c81ae14
test: rename the child run limit test and document its helper
vdusek Oct 30, 2026
6d957ec
docs: clarify when the child run concurrency limit refetches statuses…
vdusek Oct 30, 2026
c5a23ab
refactor: type child run statuses as a Literal
vdusek Nov 4, 2026
37a6e87
docs: reword when waiting child run starts are woken
vdusek Nov 4, 2026
c834b21
Merge remote-tracking branch 'origin/master' into feat/child-run-conc…
vdusek Nov 4, 2026
5c6780a
refactor: persist the child run registry with RecoverableState
vdusek Nov 4, 2026
8cab4ee
refactor: extract waking the starts waiting under the child run limit
vdusek Nov 4, 2026
41614ac
fix: don't delay aborting child runs behind a start waiting under the…
vdusek Nov 4, 2026
2f0ac3c
refactor: replace set_named_child_run_limits with an Actor option
vdusek Nov 9, 2026
578b722
fix: resolve the inherited child run timeout once a waiting start starts
vdusek Nov 9, 2026
0154449
fix: don't fetch child runs while holding the concurrency limit condi…
vdusek Nov 9, 2026
7cc79b7
fix: keep the last seen status of a child run that fails to be fetched
vdusek Nov 9, 2026
d3c56cd
Merge remote-tracking branch 'origin/master' into feat/child-run-conc…
vdusek Nov 9, 2026
c678e5e
feat: track child runs started without a `run_name` in `Actor.child_r…
vdusek Nov 11, 2026
a0201f8
feat: support `abort_with_parent` for child runs started without a `r…
vdusek Nov 11, 2026
cde0b10
feat: count child runs started without a `run_name` toward the concur…
vdusek Nov 11, 2026
061c89b
docs: fix the section order in the `start_unnamed` docstring
vdusek Nov 11, 2026
c5b182e
docs: describe unnamed child runs and the limit in the agent rules
vdusek Nov 11, 2026
844da76
Merge remote-tracking branch 'origin/master' into feat/child-run-conc…
vdusek Nov 12, 2026
3d0e6d6
perf: fetch a stale child run status once for all waiting starts
vdusek Nov 12, 2026
0f808d0
refactor: store unnamed child runs under the `SDK_UNNAMED_CHILD_RUNS`…
vdusek Nov 12, 2026
0c64da0
fix: reject a non-integer child run concurrency limit
vdusek Nov 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Don't introduce a shared Pydantic base model to centralize `model_config`. Each

- **`Actor`** (`_actor.py`) is a `lazy-object-proxy` wrapping `_ActorType`. It works both as a class (`Actor.is_at_home()`) and as a context manager (`async with Actor:`); on `__aenter__` the proxy's `__wrapped__` is replaced with the active `_ActorType` instance.
- **`Configuration`** (`_configuration.py`) extends Crawlee's `Configuration` and is populated from `APIFY_*` environment variables.
- **Named child runs** (`_child_runs.py`) — runs started with `run_name` are persisted in the default key-value store so they can be reattached after a migration or resurrection; exposed through `Actor.child_runs`.
- **Child runs** (`_child_runs.py`) — runs started with `run_name` are persisted in the default key-value store under their name, so they can be reattached after a migration or resurrection; runs started without a name are persisted under their run ID. All of them are exposed through `Actor.child_runs` and count toward the optional `max_concurrent_child_runs` limit.
- **Storage clients** (`storage_clients/`) implement Crawlee's storage client interface: `ApifyStorageClient` talks to the Apify API (separate sub-clients for single vs. shared request queues), `FileSystemStorageClient` adds Apify-specific key-value store behavior, and `SmartApifyStorageClient` delegates to the cloud client on the platform and to the local one elsewhere.
- **Events** — `ApifyEventManager` receives platform events over a WebSocket.
- **Errors** — `apify.errors` only re-exports the exceptions from `apify_client.errors`; the SDK defines no error classes of its own, and API calls raise the client exceptions unchanged.
Expand Down
Loading
Loading