O03, dunders and the slots behind them

GLOSSARY.md

@@ -8,7 +8,7 @@ This file is generated from `pyxray/src/pyxray/glossary.py`. Edit that and run `
 
 ## Index
 
-[ASDL](#asdl) | [Argument Clinic](#argument-clinic) | [EXTENDED_ARG](#extended_arg) | [GC pre header](#gc-pre-header) | [JIT](#jit) | [PEG parser](#peg-parser) | [PyVarObject](#pyvarobject) | [Pyodide](#pyodide) | [WebAssembly](#webassembly) | [abstract syntax tree](#abstract-syntax-tree) | [adaptive instruction](#adaptive-instruction) | [arena](#arena) | [assembler](#assembler) | [backtrace](#backtrace) | [basic block](#basic-block) | [binding](#binding) | [block](#block) | [blurb](#blurb) | [borrowed reference](#borrowed-reference) | [bytecode](#bytecode) | [cell](#cell) | [class cell](#class-cell) | [closure](#closure) | [code generation](#code-generation) | [code object](#code-object) | [coding cookie](#coding-cookie) | [cold block](#cold-block) | [computed goto](#computed-goto) | [configure](#configure) | [constant folding](#constant-folding) | [control flow graph](#control-flow-graph) | [cycle collector](#cycle-collector) | [deallocation](#deallocation) | [debug build](#debug-build) | [deoptimization](#deoptimization) | [devguide](#devguide) | [dispatch](#dispatch) | [environment changed](#environment-changed) | [eval loop](#eval-loop) | [evaluation order](#evaluation-order) | [exception table](#exception-table) | [f string](#f-string) | [finalizer](#finalizer) | [frame](#frame) | [free threaded build](#free-threaded-build) | [free variable](#free-variable) | [gdb](#gdb) | [generated file](#generated-file) | [generation](#generation) | [grammar](#grammar) | [header file](#header-file) | [heap type](#heap-type) | [immortal object](#immortal-object) | [indent and dedent](#indent-and-dedent) | [inline cache](#inline-cache) | [instance dictionary](#instance-dictionary) | [instruction](#instruction) | [interning](#interning) | [left recursion](#left-recursion) | [line table](#line-table) | [magic number](#magic-number) | [marshal](#marshal) | [metaclass](#metaclass) | [monitoring events](#monitoring-events) | [new reference](#new-reference) | [object](#object) | [object header](#object-header) | [obmalloc](#obmalloc) | [oparg](#oparg) | [opcode](#opcode) | [parser generator](#parser-generator) | [pdb](#pdb) | [pointer](#pointer) | [pool](#pool) | [product type](#product-type) | [profile guided optimization](#profile-guided-optimization) | [pseudo instruction](#pseudo-instruction) | [pyc file](#pyc-file) | [pyconfig](#pyconfig) | [reference count](#reference-count) | [reference cycle](#reference-cycle) | [reference leak](#reference-leak) | [regen](#regen) | [regrtest](#regrtest) | [replacement field](#replacement-field) | [resource](#resource) | [scope](#scope) | [segmentation fault](#segmentation-fault) | [short circuiting](#short-circuiting) | [small integer cache](#small-integer-cache) | [soft keyword](#soft-keyword) | [specialization](#specialization) | [stack depth](#stack-depth) | [stack effect](#stack-effect) | [static type](#static-type) | [stolen reference](#stolen-reference) | [struct](#struct) | [sum type](#sum-type) | [symbol table](#symbol-table) | [symbol table pass](#symbol-table-pass) | [t string](#t-string) | [test case](#test-case) | [tier one](#tier-one) | [tier two](#tier-two) | [token](#token) | [tokenizer](#tokenizer) | [trace function](#trace-function) | [type object](#type-object) | [underflow](#underflow) | [value stack](#value-stack) | [weak reference](#weak-reference)
+[ASDL](#asdl) | [Argument Clinic](#argument-clinic) | [EXTENDED_ARG](#extended_arg) | [GC pre header](#gc-pre-header) | [JIT](#jit) | [PEG parser](#peg-parser) | [PyVarObject](#pyvarobject) | [Pyodide](#pyodide) | [WebAssembly](#webassembly) | [abstract syntax tree](#abstract-syntax-tree) | [adaptive instruction](#adaptive-instruction) | [arena](#arena) | [assembler](#assembler) | [backtrace](#backtrace) | [basic block](#basic-block) | [binding](#binding) | [block](#block) | [blurb](#blurb) | [borrowed reference](#borrowed-reference) | [bytecode](#bytecode) | [cell](#cell) | [class cell](#class-cell) | [closure](#closure) | [code generation](#code-generation) | [code object](#code-object) | [coding cookie](#coding-cookie) | [cold block](#cold-block) | [computed goto](#computed-goto) | [configure](#configure) | [constant folding](#constant-folding) | [control flow graph](#control-flow-graph) | [cycle collector](#cycle-collector) | [deallocation](#deallocation) | [debug build](#debug-build) | [deoptimization](#deoptimization) | [devguide](#devguide) | [dispatch](#dispatch) | [environment changed](#environment-changed) | [eval loop](#eval-loop) | [evaluation order](#evaluation-order) | [exception table](#exception-table) | [f string](#f-string) | [finalizer](#finalizer) | [frame](#frame) | [free threaded build](#free-threaded-build) | [free variable](#free-variable) | [gdb](#gdb) | [generated file](#generated-file) | [generation](#generation) | [grammar](#grammar) | [header file](#header-file) | [heap type](#heap-type) | [immortal object](#immortal-object) | [indent and dedent](#indent-and-dedent) | [inline cache](#inline-cache) | [instance dictionary](#instance-dictionary) | [instruction](#instruction) | [interning](#interning) | [left recursion](#left-recursion) | [line table](#line-table) | [magic number](#magic-number) | [marshal](#marshal) | [metaclass](#metaclass) | [monitoring events](#monitoring-events) | [new reference](#new-reference) | [object](#object) | [object header](#object-header) | [obmalloc](#obmalloc) | [oparg](#oparg) | [opcode](#opcode) | [parser generator](#parser-generator) | [pdb](#pdb) | [pointer](#pointer) | [pool](#pool) | [product type](#product-type) | [profile guided optimization](#profile-guided-optimization) | [pseudo instruction](#pseudo-instruction) | [pyc file](#pyc-file) | [pyconfig](#pyconfig) | [reference count](#reference-count) | [reference cycle](#reference-cycle) | [reference leak](#reference-leak) | [regen](#regen) | [regrtest](#regrtest) | [replacement field](#replacement-field) | [resource](#resource) | [scope](#scope) | [segmentation fault](#segmentation-fault) | [short circuiting](#short-circuiting) | [slot](#slot) | [slot wrapper](#slot-wrapper) | [small integer cache](#small-integer-cache) | [soft keyword](#soft-keyword) | [specialization](#specialization) | [stack depth](#stack-depth) | [stack effect](#stack-effect) | [static type](#static-type) | [stolen reference](#stolen-reference) | [struct](#struct) | [sum type](#sum-type) | [symbol table](#symbol-table) | [symbol table pass](#symbol-table-pass) | [t string](#t-string) | [test case](#test-case) | [tier one](#tier-one) | [tier two](#tier-two) | [token](#token) | [tokenizer](#tokenizer) | [trace function](#trace-function) | [type object](#type-object) | [underflow](#underflow) | [value stack](#value-stack) | [weak reference](#weak-reference)
 
 ## Reading the source
 
@@ -642,6 +642,22 @@ Tuples, lists and bytes objects all hold a count of how many items they have, an
 
 Also written variable sized object, `ob_size`. First met in O01. See also [object header](#object-header), [object](#object). In the source: [`Include/object.h:174-178@v3.15.0rc1#PyVarObject`](https://github.com/python/cpython/blob/v3.15.0rc1/Include/object.h#L174-L178).
 
+### slot
+
+**One of the function pointer fields in a type object.**
+
+`tp_repr`, `tp_hash`, `tp_call` and about seventy others. The interpreter reads them directly, so calling `repr(x)` is a load and an indirect call rather than a dictionary lookup. Python code never assigns to one. It defines a dunder method and a table walk fills the slot in, which is what makes the two spellings feel like the same thing.
+
+Also written type slot. First met in O03. See also [slot wrapper](#slot-wrapper), [type object](#type-object). In the source: [`Objects/typeobject.c:11584-11590@v3.15.0rc1#slotdefs`](https://github.com/python/cpython/blob/v3.15.0rc1/Objects/typeobject.c#L11584-L11590).
+
+### slot wrapper
+
+**A dunder method that is really a C slot with a Python callable wrapped around it.**
+
+When a type written in C is made ready, `add_operators` walks the slot table and puts one of these into the class dict for every slot that has a function in it. That is where `int.__add__` and `object.__repr__` come from: nobody wrote them as methods, they are `nb_add` and `tp_repr` made callable. `type(int.__add__).__name__` is `wrapper_descriptor`, which is how you tell one from an ordinary method.
+
+Also written `wrapper_descriptor`. First met in O03. See also [slot](#slot), [type object](#type-object). In the source: [`Objects/typeobject.c:12456-12470@v3.15.0rc1#add_operators`](https://github.com/python/cpython/blob/v3.15.0rc1/Objects/typeobject.c#L12456-L12470).
+
 ### static type
 
 **A type object written out as a C literal and compiled into the binary.**

README.md

@@ -118,6 +118,7 @@ A lesson can also end with a boss fight, which is a problem the text does not so
 | F12 | [What ends up on disk](lessons/f12-what-ends-up-on-disk/f12.ipynb) | The other direction: a code object flattened into bytes and read straight back, the sixteen byte header decoded field by field, the type byte and the reference table that make marshal small, a fifty line reader checked against the real one, and a handmade .pyc that imports with no .py anywhere | M3 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/f12-what-ends-up-on-disk/f12.ipynb) |
 | O01 | [The header, byte by byte](lessons/o01-the-header-byte-by-byte/o01.ipynb) | The two fields everybody quotes, read out of your own interpreter with ctypes, and the first of them turning out to be three: a thirty two bit count, sixteen bits named and unused, and the flags that say whether an object can ever be freed, plus why an immortal starts in the middle of the range and where a length lives when a type has one | M4 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/o01-the-header-byte-by-byte/o01.ipynb) |
 | O02 | [Following the type pointer](lessons/o02-following-the-type-pointer/o02.ipynb) | The biggest struct in the interpreter, read from Python: what basicsize and itemsize say about an instance and why list breaks the arithmetic, the one flag bit behind cannot set attribute of immutable type, the minus one in tp_dictoffset that is a sentinel rather than an offset, the thirty two bytes reserved in front of every instance of a class you wrote, and the five steps a class statement actually compiles to | M4 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/o02-following-the-type-pointer/o02.ipynb) |
+| O03 | [Dunders and the slots behind them](lessons/o03-dunders-and-slots/o03.ipynb) | Where int.__add__ comes from when nobody wrote it, why setting __repr__ on an instance does nothing and setting it on a class works an hour later, how one __getitem__ makes a class iterable without an __iter__, why a subclass __radd__ beats a base class __add__, and the three steps between defining __eq__ and getting unhashable type | M4 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/o03-dunders-and-slots/o03.ipynb) |
 
 More are landing in order. [lessons/README.md](lessons/README.md) explains how one is put together and how to run them locally.
 

citations.lock.json

@@ -695,6 +695,51 @@
       "first_line": "void",
       "lines": 20
     },
+    "Objects/typeobject.c:10524-10530@v3.15.0rc1": {
+      "digest": "bd97cf89d9d7c794",
+      "first_line": "#define SLOT0(FUNCNAME, DUNDER) \\",
+      "lines": 7
+    },
+    "Objects/typeobject.c:10573-10600@v3.15.0rc1": {
+      "digest": "88cff08be482a9fa",
+      "first_line": "#define SLOT1BINFULL(FUNCNAME, TESTFUNC, SLOTNAME, DUNDER, RDUNDER) \\",
+      "lines": 28
+    },
+    "Objects/typeobject.c:11522-11531@v3.15.0rc1": {
+      "digest": "4ad77fa864aa2cae",
+      "first_line": "/*",
+      "lines": 10
+    },
+    "Objects/typeobject.c:11546-11553@v3.15.0rc1": {
+      "digest": "0385bd444c1da88c",
+      "first_line": "#define TPSLOT(NAME, SLOT, FUNCTION, WRAPPER, DOC) \\",
+      "lines": 8
+    },
+    "Objects/typeobject.c:11584-11590@v3.15.0rc1": {
+      "digest": "f494591191e7ce56",
+      "first_line": "static pytype_slotdef slotdefs[] = {",
+      "lines": 7
+    },
+    "Objects/typeobject.c:11740-11753@v3.15.0rc1": {
+      "digest": "77bf33636ec9ffc9",
+      "first_line": "MPSLOT(__len__, mp_length, slot_mp_length, wrap_lenfunc,",
+      "lines": 14
+    },
+    "Objects/typeobject.c:12086-12096@v3.15.0rc1": {
+      "digest": "69170f42ee69fe8e",
+      "first_line": "/* Update the slots after assignment to a class (type) attribute. */",
+      "lines": 11
+    },
+    "Objects/typeobject.c:12129-12138@v3.15.0rc1": {
+      "digest": "9e6a8d9ad5b83628",
+      "first_line": "dict. */",
+      "lines": 10
+    },
+    "Objects/typeobject.c:12456-12470@v3.15.0rc1": {
+      "digest": "b3eb73a5e54ec845",
+      "first_line": "add_operators(PyTypeObject *type)",
+      "lines": 15
+    },
     "Objects/typeobject.c:2512-2531@v3.15.0rc1": {
       "digest": "9ac3593478808a3d",
       "first_line": "_PyType_AllocNoTrack(PyTypeObject *type, Py_ssize_t nitems)",
@@ -705,6 +750,16 @@
       "first_line": "PyTypeObject PyType_Type = {",
       "lines": 6
     },
+    "Objects/typeobject.c:8808-8818@v3.15.0rc1": {
+      "digest": "42d8cbff41ef620b",
+      "first_line": "static int",
+      "lines": 11
+    },
+    "Objects/typeobject.c:9370-9391@v3.15.0rc1": {
+      "digest": "ff402e2b47b1287a",
+      "first_line": "tp_dict['__hash__'] equal to None.",
+      "lines": 22
+    },
     "Objects/weakrefobject.c:1001-1024@v3.15.0rc1": {
       "digest": "2990efebc60f458b",
       "first_line": "/* This function is called by the tp_dealloc handler to clear weak references.",

lessons/CLAIMS.md

@@ -12,7 +12,7 @@ header, what the allocator does with a freed block, the shape of the eval loop.
 marked with the reason, and a lesson is allowed at most 3 of them. The cap is the point.
 Without it the exception becomes the rule and this goes back to being a book.
 
-257 claims across 30 lessons, 21 of them not observable from Python.
+263 claims across 31 lessons, 21 of them not observable from Python.
 
 ## B01. Building CPython, and whether you need to
 
@@ -207,6 +207,17 @@ Without it the exception becomes the rule and this goes back to being a book.
 | an instance of a class with a managed dict is allocated with room for its attribute values after the object, and that room is in neither of the numbers getsizeof adds together | not observable from Python: the inline values array is sized from a table on the type and no Python level call reports it |
 | a class statement compiles to a call to __build_class__, and calling type with a name, bases and a namespace produces a type with the same flags, size and mro | [`o02-20`](o02-following-the-type-pointer/o02.ipynb) |
 
+## O03. Dunders and the slots behind them
+
+| Claim | Proved by |
+| --- | --- |
+| the dunder methods on a builtin type are slot wrappers generated from its C slots, and they are a different kind of object from both the plain methods on the same type and the functions you write in a class body | [`o03-07`](o03-dunders-and-slots/o03.ipynb) |
+| a dunder assigned to an instance is a real attribute you can call by name, and the built in function that would use it never sees it, because the slot looks the name up on the type | [`o03-11`](o03-dunders-and-slots/o03.ipynb) |
+| assigning a dunder to a class updates the slot at once, on that class and on every subclass of it, and deleting it puts the inherited behaviour back | [`o03-13`](o03-dunders-and-slots/o03.ipynb) |
+| a class defining only __getitem__ can be iterated over and used with in, because that one name fills the sq_item slot and the old sequence protocol is still what iteration falls back on | [`o03-15`](o03-dunders-and-slots/o03.ipynb) |
+| a subclass that defines __radd__ itself is called before the base class __add__, and an unrelated class defining __radd__ is not, because both names share one slot and the dispatcher checks the subclass relationship first | [`o03-17`](o03-dunders-and-slots/o03.ipynb) |
+| a class that defines __eq__ and not __hash__ ends up with a real None stored under __hash__ in its class dict, and putting a hash back is a one line assignment | [`o03-19`](o03-dunders-and-slots/o03.ipynb) |
+
 ## T01. One line, seven stages
 
 | Claim | Proved by |

lessons/README.md

@@ -34,6 +34,7 @@ Each lesson is a notebook you can run. There is nothing to install and nothing t
 | [F12. What ends up on disk](f12-what-ends-up-on-disk/f12.ipynb) | What a .pyc actually holds, the four header fields and which two decide staleness, why half the magic number is a carriage return and a newline, the one byte that names every marshalled object, how a repeated value costs five bytes, a marshal reader written here, and a .pyc assembled by hand and imported with no source file on disk | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/f12-what-ends-up-on-disk/f12.ipynb) |
 | [O01. The header, byte by byte](o01-the-header-byte-by-byte/o01.ipynb) | Reading the object header out of memory with ctypes and checking every field against something Python will tell you another way, the thirty two bit count and the two fields sharing its word, why an immortal object parks at 3 << 30 rather than at the top, the two flag bits that say how it got there, why a string CPython uses itself is immortal and yours is not, and the two words in front of the header that only the collector sees | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/o01-the-header-byte-by-byte/o01.ipynb) |
 | [O02. Following the type pointer](o02-following-the-type-pointer/o02.ipynb) | Where the second word of an object goes, why type is its own type, the two numbers a type keeps about how big its instances are and the one type they do not predict, the flag bit that decides whether you can assign to a type, the negative offsets that are not offsets and the space reserved in front of every instance, and the class statement turning out to be an ordinary function call | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/o02-following-the-type-pointer/o02.ipynb) |
+| [O03. Dunders and the slots behind them](o03-dunders-and-slots/o03.ipynb) | The table of ninety four rows that connects every dunder name to a C slot, read forwards when you write a class and backwards when a type written in C is made ready, why a dunder set on an instance is ignored and one assigned to a class takes effect at once, the names that fill two slots and give you truthiness and iteration for free, the two names that share one slot and have to argue about who goes first, and the two mechanisms that put a literal None in __hash__ | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/o03-dunders-and-slots/o03.ipynb) |
 
 ## The three programs