Skip to content

fix(docker): preserve failed crawl results - #2134

Merged
ntohidi merged 1 commit into
unclecode:developfrom
nightcityblade:fix/issue-2133
Aug 11, 2026
Merged

fix(docker): preserve failed crawl results#2134
ntohidi merged 1 commit into
unclecode:developfrom
nightcityblade:fix/issue-2133

Conversation

@nightcityblade

Copy link
Copy Markdown
Contributor

Summary

Fixes #2133.

Keep unsuccessful /crawl outcomes as normal per-URL results instead of converting an all-failed batch into an opaque HTTP 500 response. This preserves the crawler's error_message, including unmatched wait_for details, for API clients.

List of files changed and why

  • deploy/docker/server.py - Return the structured crawl response even when every URL failed.
  • deploy/docker/tests/test_crawl_failure_response.py - Add an offline route regression test for an all-failed crawl.
  • tests/docker/test_server_requests.py - Correct the existing integration expectation to HTTP 200 with failed results.

How Has This Been Tested?

  • .venv/bin/pytest -q deploy/docker/tests/test_crawl_failure_response.py (1 passed)
  • .venv/bin/ruff check --select E9,F63,F7,F82 deploy/docker/server.py deploy/docker/tests/test_crawl_failure_response.py tests/docker/test_server_requests.py
  • .venv/bin/black --check deploy/docker/tests/test_crawl_failure_response.py
  • .venv/bin/python -m py_compile deploy/docker/server.py deploy/docker/tests/test_crawl_failure_response.py tests/docker/test_server_requests.py

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas — N/A; the route behavior is straightforward.
  • I have made corresponding changes to the documentation — N/A; this fixes the existing documented per-URL result contract.
  • I have added/updated unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@ntohidi
ntohidi merged commit ea26abb into unclecode:develop Aug 11, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants