test: derive the realtime integration UTS tier

test/uts/README.md

@@ -195,6 +195,64 @@ Tests in this package are given 300 seconds each: a cold cache downloads the bin
 before the first of them runs, and a specification that provokes a timeout sits through
 the delay it asked the proxy for.
 
+`realtime/integration/` runs against the same sandbox over a real WebSocket — twenty
+specifications, thirteen of them straight to the sandbox and seven under `proxy/`. It
+provisions an app of its own, which arrives as the `realtime_sandbox` fixture, so that a
+realtime test entering presence or publishing to a channel cannot be seen by a REST test
+reading the same channel name. The app carries the same `key(i)`, `key_str` and `app_id`
+members the `sandbox` fixture does.
+
+```python
+async def test_rtl7a_subscribe_all_messages(realtime_sandbox):
+    client = sandbox_realtime_client(realtime_sandbox.key_str)
+    channel = client.channels.get('test-rtl7a-' + random_id())
+```
+
+Five specifications carry a `## Protocol Variants` section — `channel_history`,
+`channels/channel_publish`, `delta_decoding`, `mutable_messages` and `presence_lifecycle` —
+and take the tier's own `use_binary_protocol` fixture, passing it to every client they
+build. The other fifteen are json only.
+
+The connections are real, so the waits are wall-clock here too.
+`await_connection_state(client, state, timeout)` and `await_channel_state(channel, state,
+timeout)` are the specifications' `AWAIT_STATE`, and ten seconds is the figure the
+specifications give for reaching CONNECTED over a network, against the five those helpers
+default to for a mock. A state the client passes through in a millisecond cannot be waited
+for after the fact — DISCONNECTED after a drop from CONNECTED is one, the retry being a
+`loop.call_soon` — so a test that needs it registers a `connection.on(...)` recorder before
+connecting and waits on the recorded list.
+
+Tests here are given 120 seconds each, from the package's own `conftest.py`, as in the REST
+integration tier.
+
+`realtime/integration/proxy/` puts the same pinned `uts-proxy` between the client and the
+sandbox, with the same `proxy_control` and `proxy_session` fixtures and the same two
+environment variables, `UTS_PROXY_LOCAL_PATH` and `UTS_PROXY_CONTROL_URL`, that
+[helpers/proxy.py](helpers/proxy.py) documents. What these specifications fault is the
+WebSocket rather than an HTTP request — a frame suppressed, replaced or injected, a socket
+closed, an upgrade refused — and the event log is read for the frames that crossed:
+
+```python
+async def test_rtn15a_disconnect_triggers_resume(realtime_sandbox, proxy_session):
+    session = await proxy_session(rules=[{
+        'match': {'type': 'delay_after_ws_connect', 'delayMs': 1000},
+        'action': {'type': 'close'},
+        'times': 1,
+    }])
+    client = sandbox_realtime_client(
+        auth_callback=jwt_auth_callback(realtime_sandbox.key_str),
+        endpoint='localhost', port=session.proxy_port, tls=False,
+        use_binary_protocol=False, auto_connect=False)
+```
+
+`endpoint='localhost'` disables the fallback hosts by itself (REC2c2), so every attempt
+lands on the one session port and appears in the one event log. A realtime connection
+carries its credentials in the WebSocket's query string, so a plain `key=` does work over
+the session; the modules here sign an Ably JWT locally instead, through a file-local
+`jwt_auth_callback` built on `generate_jwt`, because it costs no round trip and so adds
+nothing to the log a test is counting. Tests in this package are given 300 seconds each,
+as in the REST proxy package.
+
 ## Running
 
 ```
@@ -207,10 +265,12 @@ The offline tiers alone, which need no network:
 uv run --frozen --extra crypto --extra dev pytest test/uts/rest/unit test/uts/realtime/unit test/uts/helpers -q
 ```
 
-The integration tier alone, which provisions a sandbox app and needs network access:
+Either integration tier alone, each of which provisions a sandbox app and needs network
+access:
 
 ```
 uv run --frozen --extra crypto --extra dev pytest test/uts/rest/integration -q
+uv run --frozen --extra crypto --extra dev pytest test/uts/realtime/integration -q
 ```
 
 `--frozen` is required: without it dependency resolution reaches past the

test/uts/realtime/integration/auth/token_renewal_test.py

@@ -0,0 +1,68 @@
+"""Derived from uts/realtime/integration/auth/token_renewal_test.md in ably/specification.
+
+Spec points: RSA4b, RTN14b
+
+The client is given a JWT that lives five seconds and then long-lived ones. Measured against
+the sandbox, the server answers the expiry with DISCONNECTED carrying `40142`, and the
+client is back in CONNECTED about a tenth of a second later, having called the auth callback
+a second time — so the specification's thirty-second poll has a wide margin.
+
+The specification records `initial_connection_id` before the expiry and never asserts on
+it, so nothing here reads it: ably-python has no `Connection#id` to read it from, and
+adding the adaptation described in [deviations.md](../../../deviations.md) would only
+introduce a value no assertion consumes.
+"""
+
+from ably.realtime.connection import ConnectionState
+from test.uts.helpers.client import await_connection_state, sandbox_realtime_client, wall_clock_poll_until
+from test.uts.helpers.sandbox import extract_key_name, extract_key_secret, generate_jwt
+
+# The two lifetimes the specification issues: one short enough for the server to expire
+# during the test, and one that outlives it.
+SHORT_TTL = 5000
+LONG_TTL = 3600000
+
+# The specification's `poll_until(interval: 1000ms, timeout: 30s)` and the fifteen seconds
+# it then gives the reconnection.
+RENEWAL_TIMEOUT = 30.0
+RENEWAL_INTERVAL = 1.0
+RECONNECT_TIMEOUT = 15.0
+
+
+# UTS: realtime/integration/RSA4b/token-renewal-on-expiry-0
+async def test_rsa4b_token_renewal_on_expiry(realtime_sandbox):
+    api_key = realtime_sandbox.key_str
+    key_name = extract_key_name(api_key)
+    key_secret = extract_key_secret(api_key)
+    callback_count = []
+
+    async def auth_callback(params):
+        callback_count.append(params)
+        ttl = SHORT_TTL if len(callback_count) == 1 else LONG_TTL
+        return generate_jwt(key_name=key_name, key_secret=key_secret, ttl=ttl)
+
+    client = sandbox_realtime_client(auth_callback=auth_callback, auto_connect=False)
+
+    client.connect()
+    await await_connection_state(client, ConnectionState.CONNECTED, RECONNECT_TIMEOUT)
+
+    assert len(callback_count) == 1
+
+    # The callback is invoked from the token-error handler, which can run while the
+    # connection still reports the CONNECTED it opened with. `await_connection_state`
+    # returns at once in that case, so the reconnection is counted as it happens
+    # instead.
+    reconnections = []
+    client.connection.on(lambda change: reconnections.append(change.current))
+
+    await wall_clock_poll_until(
+        lambda: len(callback_count) >= 2, timeout=RENEWAL_TIMEOUT, interval=RENEWAL_INTERVAL,
+        description='the auth callback to be invoked for a renewed token')
+
+    await wall_clock_poll_until(
+        lambda: reconnections.count(ConnectionState.CONNECTED) >= 1,
+        timeout=RECONNECT_TIMEOUT,
+        description='the connection to reach CONNECTED again on the renewed token')
+
+    assert len(callback_count) >= 2
+    assert client.connection.state is ConnectionState.CONNECTED

test/uts/realtime/integration/auth/token_request_test.py

@@ -0,0 +1,64 @@
+"""Derived from uts/realtime/integration/auth/token_request_test.md in ably/specification.
+
+Spec points: RSA9, RSA9a, RSA9g
+
+Both tests split the credentials in two: a REST client holding the API key signs
+TokenRequests, and a realtime client with no key of its own connects with whatever that
+callback hands it. Reaching CONNECTED is therefore the server's verdict on the HMAC the
+creator computed, which is what RSA9g is about.
+
+`Connection#id` is not a member of ably-python's `Connection`, so `connection_id` below
+reads the value off the connection manager. See
+[deviations.md](../../../deviations.md) for the house ruling on a missing accessor.
+
+`create_token_request` takes the specification's `TokenParams` as a plain dict in snake
+case, so `TokenParams(clientId: x)` is `{'client_id': x}`.
+"""
+
+from ably.realtime.connection import ConnectionState
+from test.uts.helpers.client import await_connection_state, sandbox_realtime_client, sandbox_rest_client
+from test.uts.helpers.sandbox import random_id
+
+# The wait the specification gives each connection.
+CONNECT_TIMEOUT = 15.0
+
+
+def connection_id(client):
+    """The specification's `connection.id`, held on the connection manager."""
+    return client.connection.connection_manager.connection_id
+
+
+# UTS: realtime/integration/RSA9a/token-request-server-accepted-0
+async def test_rsa9a_token_request_server_accepted(realtime_sandbox):
+    creator = sandbox_rest_client(realtime_sandbox.key_str)
+
+    async def auth_callback(params):
+        return await creator.auth.create_token_request()
+
+    client = sandbox_realtime_client(auth_callback=auth_callback, auto_connect=False)
+
+    client.connect()
+    await await_connection_state(client, ConnectionState.CONNECTED, CONNECT_TIMEOUT)
+
+    assert client.connection.state is ConnectionState.CONNECTED
+    assert connection_id(client) is not None
+    assert client.connection.error_reason is None
+
+
+# UTS: realtime/integration/RSA9/token-request-with-clientid-0
+async def test_rsa9_token_request_with_clientid(realtime_sandbox):
+    test_client_id = 'token-request-client-' + random_id()
+
+    creator = sandbox_rest_client(realtime_sandbox.key_str)
+
+    async def auth_callback(params):
+        return await creator.auth.create_token_request({'client_id': test_client_id})
+
+    client = sandbox_realtime_client(
+        auth_callback=auth_callback, client_id=test_client_id, auto_connect=False)
+
+    client.connect()
+    await await_connection_state(client, ConnectionState.CONNECTED, CONNECT_TIMEOUT)
+
+    assert client.connection.state is ConnectionState.CONNECTED
+    assert client.auth.client_id == test_client_id

test/uts/realtime/integration/auth_test.py

@@ -0,0 +1,138 @@
+"""Derived from uts/realtime/integration/auth.md in ably/specification.
+
+Spec points: RTC8, RTC8a, RTC8c, RSA8, RSA7
+
+Every client here authenticates through an `auth_callback` returning an Ably JWT, which is
+what the specification's third-party JWT library produces; `generate_jwt` signs it HS256
+over the key secret.
+
+`Connection#id` is not a member of ably-python's `Connection`, so `connection_id` below
+reads the value off the connection manager. See
+[deviations.md](../../deviations.md) for the house ruling on a missing accessor.
+
+`authorize()` on a CONNECTED connection sends AUTH and then awaits the next state change
+before it returns, so the UPDATE it provokes has already been delivered to a listener
+registered beforehand by the time the call completes. The reauth test needs no settle.
+"""
+
+from ably.realtime.connection import ConnectionState
+from test.uts.helpers.client import await_connection_state, sandbox_realtime_client
+from test.uts.helpers.deviations import deviation
+from test.uts.helpers.sandbox import extract_key_name, extract_key_secret, generate_jwt, random_id
+
+# The lifetime every JWT here is issued with, as the specification's `ttl: 3600000`.
+TOKEN_TTL = 3600000
+
+# The wait for the mismatched-clientId connection to be failed by the server. The SDK spends
+# its `disconnected_retry_timeout` — 15 seconds by default — between the first attempt and
+# the one the server rejects, so the specification's unstated wait has to clear that.
+FAIL_TIMEOUT = 20.0
+
+
+def connection_id(client):
+    """The specification's `connection.id`, held on the connection manager."""
+    return client.connection.connection_manager.connection_id
+
+
+def jwt_callback(api_key, client_id=None):
+    """The specification's `auth_callback`, answering with a freshly signed Ably JWT."""
+    key_name = extract_key_name(api_key)
+    key_secret = extract_key_secret(api_key)
+
+    async def auth_callback(params):
+        return generate_jwt(key_name=key_name, key_secret=key_secret, ttl=TOKEN_TTL, client_id=client_id)
+
+    return auth_callback
+
+
+# UTS: realtime/integration/RTC8a/in-band-reauth-connected-0
+async def test_rtc8a_in_band_reauth_connected(realtime_sandbox):
+    client = sandbox_realtime_client(
+        auth_callback=jwt_callback(realtime_sandbox.key_str), auto_connect=False)
+
+    client.connect()
+    await await_connection_state(client, ConnectionState.CONNECTED)
+
+    connection_id_before = connection_id(client)
+
+    state_changes = []
+
+    def record(change):
+        state_changes.append(change)
+
+    client.connection.on(record)
+
+    token = await client.auth.authorize()
+
+    connection_id_after = connection_id(client)
+
+    assert token is not None
+    assert isinstance(token.token, str)
+
+    assert connection_id_after == connection_id_before
+
+    state_transitions = [change for change in state_changes if change.current != change.previous]
+    assert state_transitions == []
+
+
+# UTS: realtime/integration/RTC8c/authorize-initiates-connection-0
+async def test_rtc8c_authorize_initiates_connection(realtime_sandbox):
+    client = sandbox_realtime_client(
+        auth_callback=jwt_callback(realtime_sandbox.key_str), auto_connect=False)
+
+    assert client.connection.state is ConnectionState.INITIALIZED
+
+    token = await client.auth.authorize()
+
+    await await_connection_state(client, ConnectionState.CONNECTED)
+
+    assert token is not None
+    assert client.connection.state is ConnectionState.CONNECTED
+    assert connection_id(client) is not None
+
+
+# UTS: realtime/integration/RSA8/token-auth-connect-0
+async def test_rsa8_token_auth_connect(realtime_sandbox):
+    client = sandbox_realtime_client(
+        auth_callback=jwt_callback(realtime_sandbox.key_str), auto_connect=False)
+
+    client.connect()
+    await await_connection_state(client, ConnectionState.CONNECTED)
+
+    assert client.connection.state is ConnectionState.CONNECTED
+    assert connection_id(client) is not None
+    assert client.connection.error_reason is None
+
+
+# UTS: realtime/integration/RSA7/matching-clientid-succeeds-0
+@deviation
+async def test_rsa7_matching_clientid_succeeds(realtime_sandbox):
+    test_client_id = 'test-client-' + random_id()
+
+    client = sandbox_realtime_client(
+        auth_callback=jwt_callback(realtime_sandbox.key_str, client_id=test_client_id),
+        client_id=test_client_id, auto_connect=False)
+
+    client.connect()
+    await await_connection_state(client, ConnectionState.CONNECTED)
+
+    assert client.connection.state is ConnectionState.CONNECTED
+    assert client.auth.client_id == test_client_id
+
+
+# UTS: realtime/integration/RSA7/mismatched-clientid-fails-1
+async def test_rsa7_mismatched_clientid_fails(realtime_sandbox):
+    # UTS SPEC ERROR: the test step expects the `Realtime` constructor to throw, while the
+    # assertions below it say the key assertion is that the connection enters FAILED with
+    # 40102. Both cannot hold — a constructor has no token to compare a clientId against,
+    # and the specification's own note concedes the point — so the client is constructed
+    # and the FAILED assertion the specification names is the one made.
+    client = sandbox_realtime_client(
+        auth_callback=jwt_callback(realtime_sandbox.key_str, client_id='token-client-id'),
+        client_id='wrong-client-id', auto_connect=False)
+
+    client.connect()
+    await await_connection_state(client, ConnectionState.FAILED, FAIL_TIMEOUT)
+
+    assert client.connection.state is ConnectionState.FAILED
+    assert client.connection.error_reason.code == 40102