Skip to content

Fix Python typing for bytes in typespec - #11233

Merged
Yuchao Yan (msyyc) merged 9 commits into
mainfrom
copilot/fix-python-typing-for-bytes
Jul 16, 2026
Merged

Yuchao Yan (msyyc) merged 9 commits into
mainfrom
copilot/fix-python-typing-for-bytes

Conversation

Copilot AI commented Jul 13, 2026 •

Copy link
Copy Markdown
Contributor

Design

design is approved in teams channel

Summary

Add an IO[bytes] overload alongside bytes for binary bytes bodies in the Python emitter, to keep backward compatibility for services migrating from swagger (whose binary bodies were typed as IO).

  • Add IO overload for binary bytes bodies in preprocess (add_body_param_type)
  • Fix body serialization to avoid redundant/mypy-breaking isinstance branch when both overloads write raw content
  • Describe bytes overload content-type param as "byte" body
  • Add unit tests (test_bytes_body_overloads.py) — passing
  • Add mock_api tests exercising the IO overload (sync + async)
  • Regenerate all specs; encode-bytes & streaming-jsonl verified (mypy assignment error resolved)
  • Full typecheck:generated run (was in progress at timeout)
  • Add changelog entry (chronus)

Code diff

Azure/azure-sdk-for-python#48040

Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:client:python Issue for the Python client emitter: @typespec/http-client-python label Jul 13, 2026
Copilot AI requested a review from Yuchao Yan (msyyc) July 13, 2026 09:26
@msyyc

Copy link
Copy Markdown
Contributor

Copilot continue commit for this PR and current progress is : #11233 (comment)

NOTE: CI repports the following error:

=== Running lint checks ===
npm run lint

> @typespec/http-client-python@0.34.1 lint
> tsx ./eng/scripts/ci/lint.ts


=== Linting TypeScript ===

[RUN] eslint emitter/ eng/scripts/ --max-warnings=0
(node:9151) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated.
(Use `node --trace-deprecation ...` to show where the warning was created)
[PASS] eslint emitter/ eng/scripts/ --max-warnings=0 completed successfully

=== Linting Python Generator (pygen) ===

[RUN] /mnt/vss/_work/1/s/packages/http-client-python/venv/bin/python -m pylint generator/pygen --rcfile=eng/scripts/ci/config/pylintrc --recursive=y --py-version=3.10
************* Module pygen.preprocess
generator/pygen/preprocess/__init__.py:406:12: R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions)

------------------------------------
Your code has been rated at 10.00/10

[FAIL] /mnt/vss/_work/1/s/packages/http-client-python/venv/bin/python failed with code 8
Invoke-LoggedCommand: /mnt/vss/_work/1/s/packages/http-client-python/eng/scripts/Build-Packages.ps1:87
Line |
  87 |          Invoke-LoggedCommand "npm run lint" -GroupOutput
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | ##[error]Command completed with exit code 1 (00:00:15.1126133): npm run
     | lint

##[error]PowerShell exited with code '1'.
Finishing: Run build script

Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>

Copilot AI commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Copilot continue commit for this PR and current progress is : #11233 (comment)

NOTE: CI repports the following error:
...

Fixed the pylint too-many-boolean-expressions (R0916) failure in add_body_param_type by extracting the multipart/binary content-type checks into is_multipart and is_binary_body locals, reducing the if to 5 boolean terms. Pylint now passes (10.00/10) locally. Commit 32eacae.

Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
@msyyc

Copy link
Copy Markdown
Contributor

Copilot fix the following typing check error in generated code of spector case


************* Module streaming.jsonl.basic.operations._operations
generated/azure/streaming-jsonl/streaming/jsonl/basic/operations/_operations.py:9: [W0611(unused-import), ] Unused IOBase imported from io
************* Module streaming.jsonl.basic.aio.operations._operations
generated/azure/streaming-jsonl/streaming/jsonl/basic/aio/operations/_operations.py:9: [W0611(unused-import), ] Unused IOBase imported from io

------------------------------------------------------------------
Your code has been rated at 9.99/10 (previous run: 9.99/10, +0.00)


------------------------------------
Your code has been rated at 10.00/10


-------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 9.99/10, +0.01)

ERROR:root:/mnt/vss/_work/1/s/packages/http-client-python/tests/generated/azure/streaming-jsonl/streaming exited with linting error 4
************* Module encode.bytes.requestbody.operations._operations
generated/azure/encode-bytes/encode/bytes/requestbody/operations/_operations.py:9: [W0611(unused-import), ] Unused IOBase imported from io
************* Module encode.bytes.requestbody.aio.operations._operations
generated/azure/encode-bytes/encode/bytes/requestbody/aio/operations/_operations.py:9: [W0611(unused-import), ] Unused IOBase imported from io

Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>

Copilot AI commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Copilot fix the following typing check error in generated code of spector case

...

Fixed in bf61eb3. need_import_iobase now returns False when the combined body type contains a ByteArraySchema (the binary bytes + IO[bytes] case). In that case the body serialization collapses to a single assignment with no isinstance(..., (IOBase, bytes)) branch, so IOBase was imported but unused. Regenerated encode-bytes and streaming-jsonl — the from io import IOBase unused import is gone from the requestbody/basic operations (sync + async), while model bodies that still emit the isinstance branch keep the import. Pylint reports no unused-import warnings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Python HTTP client emitter to support backward-compatible typing for binary bytes request bodies by adding an IO[bytes] overload alongside bytes, and adjusts codegen to avoid redundant runtime branching that breaks mypy type narrowing.

Changes:

  • Extend preprocess add_body_param_type to emit a combined bytes | IO[bytes] body type for non-JSON, non-multipart bytes bodies (excluding typeddict mode).
  • Update request-body serialization generation to collapse the isinstance branch when both overloads serialize to raw request content, preventing mypy narrowing issues.
  • Add unit + mock API coverage (sync/async) and a Chronus changelog entry for the fix.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/http-client-python/tests/unit/test_bytes_body_overloads.py Unit tests validating the new combined body type and overload behavior.
packages/http-client-python/tests/mock_api/shared/test_encode_bytes.py Mock API sync test exercising passing io.BytesIO for binary bytes bodies.
packages/http-client-python/tests/mock_api/shared/asynctests/test_encode_bytes_async.py Mock API async test exercising passing io.BytesIO for binary bytes bodies.
packages/http-client-python/generator/pygen/utils.py Update body-type description helper to label bytes as “byte” in overload content-type docs.
packages/http-client-python/generator/pygen/preprocess/init.py Add preprocess logic to introduce IO[bytes] overload for binary bytes bodies.
packages/http-client-python/generator/pygen/codegen/serializers/builder_serializer.py Collapse redundant binary/non-binary isinstance branching when both paths serialize to raw content.
packages/http-client-python/generator/pygen/codegen/models/operation.py Avoid importing IOBase when the combined type is the bytes/IO[bytes] case and the runtime branch is removed.
.chronus/changes/http-client-python_fix-bytes-body-io-overload-2026-7-14-3-14-0.md Changelog entry documenting the fix for @typespec/http-client-python.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@msyyc Yuchao Yan (msyyc) changed the title [WIP] Fix Python typing for bytes in typespec Fix Python typing for bytes in typespec Jul 15, 2026
@pkg-pr-new

pkg-pr-new Bot commented Jul 15, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-python@11233

commit: c3b23aa

@github-actions

Copy link
Copy Markdown
Contributor

All changed packages have been documented.

  • ✅ @typespec/http-client-python
Show changes

@typespec/http-client-python - fix ✏️

Add an IO[bytes] overload alongside bytes for binary bytes bodies, keeping backward compatibility for services migrating from swagger whose binary bodies were typed as IO.

…es body as binary

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 082cab3e-1c55-4eb5-951a-819a9842137f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 082cab3e-1c55-4eb5-951a-819a9842137f

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

packages/http-client-python/generator/pygen/utils.py:118

  • The PR description calls out describing the bytes overload content-type parameter as a "byte" body, but get_body_type_for_description maps bytes to "binary", so the generated description will say "binary body" for both bytes and IO[bytes] overloads.
        return "binary"
    if body_parameter["type"]["type"] == "string":
        return "string"
    return "JSON"


…y-statements

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 082cab3e-1c55-4eb5-951a-819a9842137f
@l0lawrence

Copy link
Copy Markdown
Member

@msyyc
Yuchao Yan (msyyc) added this pull request to the merge queue Jul 16, 2026
Merged via the queue into main with commit 6f86a25 Jul 16, 2026
36 checks passed
@msyyc
Yuchao Yan (msyyc) deleted the copilot/fix-python-typing-for-bytes branch July 16, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:python Issue for the Python client emitter: @typespec/http-client-python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python] typing for bytes in typespec

5 participants