`typing`: one `None`-valued member makes `isinstance()` on a `runtime_checkable` Protocol O(N) per call

[adamtheturtle]

A class satisfying a @runtime_checkable Protocol with any class attribute set to None still passes isinstance(), but is orders of magnitude slower, scaling linearly with member count. Best of 5 runs of 20,000 calls, at 5/20/50/110 members: 25x/84x/316x/735x slower.

import timeit
from typing import Protocol, runtime_checkable

@runtime_checkable
class P(Protocol):
    @property
    def a(self) -> bool: ...
    @property
    def b(self) -> bool: ...

class Good:
    a = b = True

class Bad:
    a, b = True, None

for x in (Good(), Bad()):
    print(isinstance(x, P), timeit.timeit(lambda: isinstance(x, P), number=100_000))

On 3.13.13: True 0.0121 then True 0.1207. Only the timing differs.

Linked PRs

• gh-156413: Let a None-valued non-callable member keep the Protocol fast path #156451

[picnixz]

1. There could be caching?
2. Does it happen on every versions?
3. Can you bisect the bad commit, if any?

[adamtheturtle]

1. Yes. _ProtocolMeta.__instancecheck__ starts with if _abc_instancecheck(cls, instance): return True. _proto_hook accepts Good, so it goes into ABCMeta's positive cache and every later call is a cache hit. It rejects Bad, which goes into the negative cache, so Bad re-runs the O(N) getattr_static loop every time.

>>> [r() for r in _abc._get_dump(P)[1]], [r() for r in _abc._get_dump(P)[2]]
([<class 'Good'>], [<class 'Bad'>])   # positive, negative

The two paths don't agree on what = None means. _proto_hook treats any None in a class __dict__ as "not implemented". __instancecheck__ only treats it that way for callable members (attr not in cls.__non_callable_proto_members__).

2. No, 3.12 and up. On 3.9 to 3.11 both classes take the slow path and the ratio is 1.0x. At 110 members, best of 5 runs of 20k, Good goes from 26615 ns/call on 3.11.14 to 101 on 3.14.4, while Bad stays at 26645 and 52966. Good is what changed. So this is Bad missing the 3.12 fast path rather than a slowdown.

3. There is nothing to bisect. The asymmetry starts at the commit that added the fast path, 47753ec (Performance of typing._ProtocolMeta._get_protocol_attrs and isinstance #74690, gh-74690: typing: Simplify and optimise _ProtocolMeta.__instancecheck__ #103159, 3.12.0b1). It dropped the _is_callable_members_only(...) guard and moved super().__instancecheck__(instance) to the top, which is what made the ABC cache reachable for protocols with non-callable members. Bad never satisfies _proto_hook, so it was never covered by it.

[picnixz]

There is nothing to bisect. The asymmetry starts at the commit that added the fast path,

Sorry, but for me this looks like the bad commit I wanted :') I wanted to know whether it was already like that when we introduced typing.Protocol but no.

I don't think it's that bad, considering that for 100+ members you get 101ns for good classes and 50ms otherwise. However, what's annoying is that it's easy to have a property that could return None or some other type and one class decides to be None while an other decides to be non-None.

OTOH, I didn't use @runtime_checkable a lot and we document it as follows:

An isinstance() check against a runtime-checkable protocol can be surprisingly slow compared to an isinstance() check against a non-protocol class. Consider using alternative idioms such as hasattr() calls for structural checks in performance-sensitive code.

I don't have a good idea of how to tackle this correctly but if the patch is un-necessarily complex and could cause breakage, then it's also possible to leave things as is.

cc @JelleZijlstra @AlexWaygood

[adamtheturtle]

Thank you for looking at this deeply @picnixz . It is a bit beyond my understanding. I asked the AI to generate a PR, and it doesn't look particularly big, but I prefer to leave that to the experts on whether it is a worthwhile trade-off.