O01, the object header read byte by byte

[tamnd]

The first lesson of M4, the object model.

T08 already said that every object starts with a reference count and a pointer to its type. That is what almost every article about CPython says, and it is a good enough summary right up until you look at the bytes. On an ordinary sixty four bit build the count is thirty two bits wide, not sixty four, and the rest of that word is doing two other jobs.

So this lesson reads the header out of the reader's own interpreter with ctypes, and checks every field against something Python will tell you another way. The address comes from id(), the count is checked against sys.getrefcount, the type pointer is checked against id(type(x)), and the length field is checked against len().

What is in it:

• the first word split into ob_refcnt at thirty two bits, ob_overflow at sixteen and ob_flags at sixteen, and the fact that ob_overflow is declared in that struct and appears nowhere else in the entire source tree
• why an immortal object parks its count at 3 << 30 when the line is at 2 ** 31 and the field ends at 2 ** 32, which is so that an extension compiled against 3.11 can increment and decrement None without knowing about immortality and be about a billion out in either direction
• the two flag bits that record how an object became immortal, and why that matters: shutting down has to free the ones promoted at runtime and must not touch the ones compiled into the binary
• the string result, which is the one people will remember: "self" and "append" in your own code are immortal, because CPython ships a generated table of the identifiers it uses itself and the compiler interns your literal to it, while a string CPython has never heard of is an ordinary heap object with an ordinary count. sys.intern does not immortalise anything, and the lesson shows that too
• where a length lives for the types that have one, why str keeps its length in the same place without being a PyVarObject, and why Py_SIZE asserts that its argument is not an int and not a bool
• the two words in front of the header that only the cycle collector sees, measured rather than asserted, as the gap between value.__sizeof__() and sys.getsizeof(value)
• and the free threaded header, twice the size, with the count split into a local half the owning thread can touch without an atomic instruction and a shared half for everybody else

Every offset is computed from ctypes.sizeof(ctypes.c_void_p) rather than hardcoded as 8 and 16. That is what lets the same cells run in the browser probe, where Python is a thirty two bit build, the header is eight bytes, the pre header gap is eight rather than sixteen, and a static immortal parks at 7 << 28. Each of those is written down in a per cell note.

3.14 and 3.15 print identical output for every cell in this lesson, which does not happen often and is said in the text.

Six diagrams as .excalidraw and .svg, all under 1100 px. Ten citations into the pinned v3.15.0rc1, all resolving. Two glossary terms, PyVarObject and GC pre header, which takes GLOSSARY.md to 108. Both README tables updated.

just check and just versions are clean locally: 29 notebooks run, 940 citations resolve, 310 cells run end to end in the browser probe, and the new cells classify as 1 declared and 5 noted with nothing undeclared and nothing stale.

Part of #23.