Skip to content

Repository files navigation

Pystrector

The Py(thon) Str(uct) (Refl)ector

icon


The package for displaying and modifying Python's internal structures.
Do you want to see how Python objects are arranged inside?
Then this package is for you.

from pystrector import Binder
binder = Binder()
some_object = 1
reflector = binder.bind(some_object)
print(reflector.ob_base.ob_refcnt.pretty_value)

Python

To install pystrector, enter the command.

python3 -m pip install pystrector

The package version matches the CPython version whose internal structures it describes (e.g. pystrector 3.12.x works with Python 3.12).


Supported platforms

The bundled struct layouts are generated by preprocessing the headers of one CPython version on one platform, so they are not portable. pystrector validates this on import:

Assumption On mismatch
CPython 3.12 UnsupportedPlatformError
little endian UnsupportedPlatformError
64-bit pointers UnsupportedPlatformError
LP64 (sizeof(long) == 8) — rules out Windows UnsupportedPlatformError
not a Py_TRACE_REFS build UnsupportedPlatformError
same OS and architecture as the bundled layouts PlatformMismatchWarning
same CPython patch release as the bundled layouts PlatformMismatchWarning
plain char has the same signedness as where the layouts were generated PlatformMismatchWarning

The warnings are not fatal, but they are worth acting on. Core object layouts (PyObject, list, int, ...) are the same across LP64 platforms, while platform specific structs (pthread types, thread state, arena bookkeeping) are not. Patch releases move fields too — 3.12.7 added statically_allocated to PyASCIIObject, which shifts everything behind it. The char warning is milder: no offset moves, but C leaves the signedness of a plain char to the platform (it is unsigned on ARM, RISC-V and LoongArch Linux, signed on x86 and on every Apple target), so a char field with its top bit set reads with the wrong sign. To be exact, regenerate the layouts for your own interpreter:

make update-python-source python-version=v3.12.9
make generate-core-datatypes

Preprocessing needs a real gcc — clang emits extensions pycparser cannot read. Override the default with make update-python-source python-version=v3.12.9 cc=gcc-15.



Git

git clone https://github.com/bontail/pystrector.git

Documentation

This README is the English documentation; a Russian translation is also available.

To access the representation of the basic structures, you need to create an anchor object

from pystrector import Binder
binder = Binder()

Now you can call the binding method to get a class object representing the structure

some_object = 1
reflector = binder.bind(some_object)

The display object has all the same fields as the internal structure

// core structure
struct _longobject {
     PyObject ob_base;
    _PyLongValue long_value;
};
class _longobject(DataType, is_union=False):
    ob_base = _object()
    long_value = _PyLongValue()

If an object contains an anonymous_var, then you can go straight to the fields of this object

class anonymous_1(DataType, is_union=True):
    ob_refcnt = LongLong()
    ob_refcnt_split = UnsignedInt[2]

    
class _object(DataType, is_union=False):
    anonymous_var_1 = anonymous_1()
    ob_type = Pointer(datatype="_typeobject")


some_object = 1
reflector = binder.bind(some_object).cast_to(_object)
# the short form below is the same as
# reflector.anonymous_var_1.ob_refcnt.pretty_value
print(reflector.ob_refcnt.pretty_value)

For each type, you can call pretty_value and bytes_value
pretty_value - will result in the most similar type in Python
bytes_value - always returns bytearray

print(reflector.long_value.lv_tag.pretty_value)
print(reflector.long_value.lv_tag.bytes_value)

You can also set values
bytes_value - accepts only bytearray
pretty_value - accepts a similar Python type
without parameters - takes another object from the mapper

reflector.ob_base.ob_refcnt.bytes_value = bytearray((1000).to_bytes(8, "little"))  # length must match the field size
reflector.ob_base.ob_refcnt.pretty_value = 1000
reflector.ob_base.ob_refcnt = binder.bind(7).ob_base.ob_refcnt

There is also work with pointers and arrays as in C

x = [1, 2, 3]
print(binder.bind(x).ob_item[0][1])
print(+(binder.bind(x).ob_item[0]))
print(+(binder.bind(x).ob_item[0] + 1))

Dereferencing a NULL pointer raises ValueError rather than reading address 0. Nothing checks the other invalid addresses, though: a pointer into unmapped memory takes the whole interpreter down with a segfault, and there is no exception left to catch.

You can convert mappers of some data types to others

from pystrector.core_datatypes import _longobject

x = [1, 2, 3]

binder.bind(x).ob_item[0][0].cast_to(_longobject) # need to cast because list saves PyObjects

Or use auto cast (works only with PyObject)

x = [1, 2, 3]

binder.bind(x).ob_item[0][0].cast()

More examples can be seen in tests

About

Package for displaying and changing core Python structures

Topics

Resources

Stars

15 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages