F11. Two tables on the side

[tamnd]

The twenty fifth lesson, and the eleventh of the front end part. F10 opened the code object and named co_linetable and co_exceptiontable without saying what is in them. This one decodes both.

The angle is that the two tables are the same idea twice. A traceback needs to know which line and which columns an instruction came from. A raise needs to know which handler covers an offset. Neither is needed while a program is going right, so neither is allowed to cost anything while it is. Both got moved out of the instruction stream into a compressed blob that nothing reads until something has already gone wrong.

What the lesson does

It opens with the zero cost claim and then checks it rather than asserting it. Wrap a loop in a try, take the instructions inside the covered range, and compare them against the same loop with no try at all. Same list. SETUP_FINALLY and POP_BLOCK still exist in the compiler, and F08 watched the optimizer move them around, but the assembler deletes both and writes a table instead, so neither reaches the bytecode.

Then it decodes an exception table entry byte by byte. Four numbers, all in code units, six bits to a byte, and the top bit of the first byte marking where an entry starts. That marker is the interesting part: it is what lets a raise binary search a table whose entries are not all the same length, because you can land anywhere and walk backwards to a boundary. Twelve lines of Python decode the whole thing and the cell checks its answer against dis._parse_exception_table.

The second half is the location table. Four numbers per instruction, six entry forms, and the assembler picks the smallest one that fits. The lesson rebuilds a traceback's caret from the column numbers by hand, then writes a thirty line decoder and runs it against 299 code objects out of argparse, dataclasses, dis and json.decoder. Zero disagreements with co_positions().

It closes on a detail I did not expect. The two tables use the same six bit varint and put the chunks in opposite orders. parse_varint reads most significant first, write_varint writes least significant first, and they are forty lines apart in the same header. It is not sloppiness. The exception table is binary searched on the first number of each entry, and reading the big end first is what makes that comparison cheap.

One thing that took a while

My location decoder disagreed with co_positions() at exactly one instruction, and the internal documentation did not explain why. write_location_info_long_form in Python/assemble.c writes loc.col_offset + 1 rather than loc.col_offset, so that a stored zero can mean "no column here", and advance_with_locations takes the one back off on the way out. Neither InternalDocs/code_objects.md nor the format comment mentions it. The lesson says so, because it is a decent reminder that the source is the specification.

Checks

Every code cell has a varies note, because all seven of them print offsets or byte counts and 3.14 compiles these functions a little smaller than 3.15. The prose holds on both, and the two lines that have to be exact, dis agrees on every one of them and 0 of them decoded differently, say the same thing on both.

Six diagrams, .excalidraw and .svg, all under 1100 px. The lesson runs end to end in Pyodide. Both READMEs have their rows. just check and just versions are green locally.

Part of M3, issue #22.