Skip to content

test: Test fixes#1874

Merged
hallvictoria merged 20 commits into
devfrom
gaaguiar/ci_pipelines_update
Jul 2, 2026
Merged

test: Test fixes#1874
hallvictoria merged 20 commits into
devfrom
gaaguiar/ci_pipelines_update

Conversation

@gavin-aguiar

@gavin-aguiar gavin-aguiar commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Description

Fixes the ServiceBus SDK binding failing on the classic worker (Python ≤ 3.12), which caused emulator tests to return  404  and function apps using the extension to fail indexing. The root cause was the worker's protobuf vendoring shadowing the app's newer protobuf.

Version-aware protobuf vendoring:

• The worker now inspects whichever  google.protobuf  is first on  sys.path  (via  find_spec ) and activates the vendored copy only when the app's protobuf is older than the vendored version. Same-or-newer protobuf (including 6.x) is used directly.
• Result: protobuf-6 extensions load, the worker keeps fast  upb , and older customer protobuf is still shadowed for safety.

Launchers:

• Removed the forced  _AZFUNC_USE_VENDORED_PROTOBUF=1  override so the worker autodetects. Local Core Tools now matches production behavior.

Uniform top-level imports:

• Worker pb2 stubs,  loader.py , and  nullable_converters.py  all import top-level  google.protobuf  so the entire worker runs in one descriptor pool.  REWRITE_PROTOBUF=False  keeps generated stubs top-level.

Pack script updates ( eng/pack/templates/{win,nix,nix_arm64,macos_64}_env_gen.yml )

Because the worker now imports the top-level  google.protobuf , the packaged artifact must include it:

• Removed the  !google/**  exclusion from the classic-worker copy step (it was dropping the top-level  google  package, causing  ModuleNotFoundError: No module named 'google'  at startup when an app shipped no protobuf).
• Added  google  back to the classic-worker validation list so packaging fails fast if it's ever missing.

Performance

• Switch pip → uv in all install scripts and CI templates ( install-dependencies.sh ,  test-sdk.sh ,  test-extensions.sh ,  ci-dependency-check.yml ,  ci-fc-tests.yml ) for faster dependency resolution. Batched the worker installs into a single resolver pass.
• Enable real pytest-xdist parallelism: unit tests  -n auto , emulator tests  -n 2 , E2E kept serial (shared Azure resources).  --dist loadfile  alone was previously a no-op without  -n .
• Build only the WebHost project instead of the whole  WebJobs.Script.sln  (~2.7x faster, far less disk, avoids restoring unrelated projects from the internal feed). Added  set -e  to  test-setup.sh  so build failures surface instead of being masked.
• Removed unused dev deps ( ptvsd ,  pre-commit ,  pytest-sugar ,  pytest-randomly ) from all three  pyproject.toml .
• Split the install step into separate "Python deps (uv)" and "webhost build (dotnet)" stages so timings are visible.

Test reliability

• Fixed flaky  test_unhandled_error  (all 3 variants): the traceback is forwarded to the host asynchronously, so the log-check snapshot raced it. Added  WebHostTestCase.wait_for_host_log()  to wait for the line before asserting.
• Replaced the racy  time.sleep(10)  webhost startup with a real readiness probe polling  /admin/host/status  for  Running  (with master key).
• Fixed Azurite "InvalidHeaderValue" in emulator tests via  --skipApiVersionCheck --loose ; added a ServiceBus emulator readiness wait.

Python 3.14 / version compatibility

• Made  runtimes/v{1,2}  installs and the uamqp-based ServiceBus binding install conditional (uv enforces  requires-python ); loosened the  fastapi  pin to resolve a  dev  vs  test-http-v2  conflict.
• Gated  TestServiceBusSDKFunctions  to Python 3.13 only (the binding needs protobuf ≥6.32/grpcio ≥1.74, incompatible with the ≤3.12 worker's pins).

New coverage

• Added a Flex Consumption test verifying blob SDK (deferred) bindings load under Flex ( BlobSdkBindings.zip  fixture +  test_blob_sdk_bindings , gated to 3.14).

Fixes #


Pull Request Checklist

Host-Worker Contract

  • Does this PR impact the host-worker contract (e.g., gRPC messages, shared interfaces)?
    • If yes, have the changes been applied to:
      • azure_functions_worker (Python <= 3.12)
      • proxy_worker (Python >= 3.13)
    • If no, please explain why:

Worker Execution Logic

  • Does this PR affect worker execution logic (e.g., function invocation, bindings, lifecycle)?
    If yes, please answer the following:

Python Version Coverage

  • Does this change apply to both Python <=3.12 and 3.13+?
  • If yes, have the changes been made to:
    • azure_functions_worker (Python <= 3.12)
    • runtimes/v1 / runtimes/v2 (Python >= 3.13)
  • If no, please explain why:

Programming Model Compatibility (for Python 3.13+)

  • Does this change apply to both:
    • V1 programming model (runtimes/v1)?
    • V2 programming model (runtimes/v2)?
  • Explanation (if limited to one model):

@gavin-aguiar gavin-aguiar marked this pull request as ready for review June 9, 2026 22:09
Comment thread eng/templates/jobs/ci-library-unit-tests.yml
Comment thread eng/templates/jobs/ci-emulator-tests.yml
Comment thread runtimes/v2/tests/unittests/test_deferred_bindings.py Outdated
Comment thread workers/tests/emulator_tests/test_servicebus_functions.py Outdated
Comment thread workers/tests/unittests/test_third_party_http_functions.py
Comment thread workers/tests/utils/testutils.py Outdated
Comment thread workers/pyproject.toml
Comment thread workers/pyproject.toml
@gavin-aguiar gavin-aguiar force-pushed the gaaguiar/ci_pipelines_update branch from f0715b4 to d66c05b Compare June 30, 2026 16:23
@hallvictoria hallvictoria merged commit cb778e2 into dev Jul 2, 2026
47 checks passed
@hallvictoria hallvictoria deleted the gaaguiar/ci_pipelines_update branch July 2, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants