Repository navigation
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1154 +/- ##
==========================================
+ Coverage 93.93% 94.10% +0.16%
==========================================
Files 54 54
Lines 3828 3970 +142
==========================================
+ Hits 3596 3736 +140
- Misses 232 234 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
5bbadc2 to
033624c
Compare
7322d8e to
7c3d462
Compare
7c3d462 to
6f9300e
Compare
6f9300e to
c2f5a87
Compare
Pijukatel
left a comment
There was a problem hiding this comment.
Thanks. It reads way better with new names. Just two more nits, otherwise good.
|
Btw. not using RecoverableState for persitence? |
Refactored to use it. |
…urrency-limit # Conflicts: # src/apify/_child_runs.py
|
I'll scope this PR back to the concurrency limit for child runs (both named and unnamed) and merge it in that form. Support for child runs without a
And later we can try to implement Pepa's idea: #1154 (comment) . |
Adds the
max_concurrent_child_runsActor option. While that many child runs areREADY,RUNNING,ABORTINGorTIMING-OUT,Actor.start,Actor.call,Actor.start_taskorActor.call_taskwaits to start or resurrect a run until one of them finishes. Reattaching to a recorded run never waits. Runs with and without arun_nameboth count.The count comes from the child run registry, so children started before a migration count too. A child's status is fetched again before counting once it's more than 10 seconds old, so a finished run nobody waited on stops counting.
Actor.callandActor.call_taskstop counting their run as soon as it finishes. A start counts toward the limit until its run is recorded, which keeps concurrent starts within the limit.Once the parent receives
ABORTING, a start still waiting under the limit raises aRuntimeErrorwithout starting a run. Waiting further would block the abort of the other children.Runs started without a
run_nameare now recorded too, under their run ID in a separate__ACTOR_UNNAMED_CHILD_RUNSkey. So they show up inActor.child_runs(keyed by run ID) and supportabort_with_parent. No later call reattaches to them, so surviving a migration still needs arun_name. An unnamedActor.call/Actor.call_tasknow starts the run and waits for it in the SDK, so the run can be recorded.The named record keeps its key and JSON shape. Both records are persisted through crawlee's
RecoverableStateand saved on every change.Closes: #1138
✍️ Drafted by Claude Code