httpx fails on IDN domains with emoji

[mimi89999]

httpx.get("https://xn--8h8haaaa.ws") raises idna.core.InvalidCodepoint while requests.get() works fine (returns 525).

import httpx
httpx.get("https://xn--8h8haaaa.ws")
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    httpx.get("https://xn--8h8haaaa.ws")
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/venv/lib/python3.13/site-packages/httpx/_api.py", line 195, in get
    return request(
        "GET",
    ...<9 lines>...
        trust_env=trust_env,
    )
  File "/dev/shm/venv/lib/python3.13/site-packages/httpx/_api.py", line 109, in request
    return client.request(
           ~~~~~~~~~~~~~~^
        method=method,
        ^^^^^^^^^^^^^^
    ...<8 lines>...
        follow_redirects=follow_redirects,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/dev/shm/venv/lib/python3.13/site-packages/httpx/_client.py", line 812, in request
    request = self.build_request(
        method=method,
    ...<9 lines>...
        extensions=extensions,
    )
  File "/dev/shm/venv/lib/python3.13/site-packages/httpx/_client.py", line 378, in build_request
    return Request(
        method,
    ...<8 lines>...
        extensions=extensions,
    )
  File "/dev/shm/venv/lib/python3.13/site-packages/httpx/_models.py", line 419, in __init__
    self._prepare(headers)
    ~~~~~~~~~~~~~^^^^^^^^^
  File "/dev/shm/venv/lib/python3.13/site-packages/httpx/_models.py", line 455, in _prepare
    if not has_host and self.url.host:
                        ^^^^^^^^^^^^^
  File "/dev/shm/venv/lib/python3.13/site-packages/httpx/_urls.py", line 191, in host
    host = idna.decode(host)
  File "/dev/shm/venv/lib/python3.13/site-packages/idna/core.py", line 430, in decode
    s = ulabel(label)
  File "/dev/shm/venv/lib/python3.13/site-packages/idna/core.py", line 328, in ulabel
    check_label(label)
    ~~~~~~~~~~~^^^^^^^
  File "/dev/shm/venv/lib/python3.13/site-packages/idna/core.py", line 277, in check_label
    raise InvalidCodepoint(
        "Codepoint {} at position {} of {} not allowed".format(_unot(cp_value), pos + 1, repr(label))
    )
idna.core.InvalidCodepoint: Codepoint U+1F33F at position 1 of '🌿🌿🌿🌿🌿' not allowed

import requests
requests.get("https://xn--8h8haaaa.ws")
# <Response [525]>

httpx version: 0.28.1
Python: 3.13

I'm already providing a punycode-encoded address, so I don't understand why there's an IDN decoding problem.

[amasen02]

Why this happens

This issue stems from a conflict between IDNA 2008 standards compliance and httpx's internal URL normalization:

1. IDNA 2008 strictly disallows Emoji in domain names

httpx relies on the Python idna library, which implements IDNA 2008 (RFC 5891 / RFC 5892).
Under RFC 5892, emoji codepoints (such as U+1F33F 🌿) are categorized as DISALLOWED. While the older IDNA 2003 and Unicode Technical Standard #46 (UTS #46) allow emoji mappings (which is how the .ws registry permits emoji domains), IDNA 2008 strictly forbids them. When idna.decode("xn--8h8haaaa") decodes the label, its mandatory check_label() step validates every codepoint and raises idna.core.InvalidCodepoint.

In contrast, requests (via urllib3) keeps raw ASCII Punycode as-is without attempting to decode it back into Unicode during request header preparation.

2. Why httpx calls idna.decode even on ASCII Punycode

When you call httpx.get("https://xn--8h8haaaa.ws"), Request._prepare() checks if a Host header needs to be generated:

if not has_host and self.url.host:
    ...

Accessing self.url.host triggers the property getter in httpx._urls.URL, which attempts to decode any Punycode labels starting with xn-- back into Unicode for human-readable representation. Because idna.decode() is called without catching idna.IDNAError, the exception propagates up and crashes the request before any socket is opened.

---

Workaround

You can bypass the self.url.host property access during header preparation by explicitly supplying the Host header:

import httpx

url = "https://xn--8h8haaaa.ws"
headers = {"Host": "xn--8h8haaaa.ws"}

# Passing Host explicitly avoids the URL.host lookup in Request._prepare
response = httpx.get(url, headers=headers)
print(response.status_code)

The Upstream Fix for httpx

In httpx/_urls.py, URL.host should catch idna.IDNAError and fall back to returning the raw Punycode string rather than failing:

try:
    host = idna.decode(host)
except idna.IDNAError:
    # If decoding fails (e.g. UTS #46 emoji domains disallowed in IDNA 2008),
    # keep the raw ASCII punycode
    pass