Skip to content

escape the hostname every marker publishes - #709

Open
dxbjavid wants to merge 5 commits into
tox-dev:mainfrom
dxbjavid:soft-rw-marker-hostname-grammar
Open

escape the hostname every marker publishes#709
dxbjavid wants to merge 5 commits into
tox-dev:mainfrom
dxbjavid:soft-rw-marker-hostname-grammar

Conversation

@dxbjavid

@dxbjavid dxbjavid commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Every marker format in this package holds the hostname to printable non-space ASCII. socket.gethostname() reports whatever the kernel stores: a space, a newline, or a byte no codec encodes, which Python hands back as a surrogate. Publishing one raw makes a holder write a record it cannot read back.

Measured against 3.32.3, on a host named host with space:

  • SoftReadWriteLock.acquire_read takes the slot, then the heartbeat stops on its first tick, so the next peer ages the marker out and takes the lock from a live reader.
  • SoftReadWriteLock.acquire_write never completes. The writer cannot recognize its own claim, so it waits out the timeout, evicts, rewrites, and loops.
  • StrictSoftFileLock.acquire raises SoftFileLockProtocolError, since the claim validator rejects a hostname holding whitespace.

On a host whose name is not valid UTF-8, every writer raises UnicodeEncodeError before the marker exists, the soft read/write state lock included. A newline forges an extra line in a protocol 1 marker, so SoftFileLock.is_lock_held_by_us reports False for a lock this process holds.

Escaping only in the soft read/write writer leaves the strict and protocol 1 formats broken, so the escape lives in host_name(), the one place all four writers read the hostname. ? is illegal in a hostname, so escaping it alongside every out-of-grammar byte as ?<hex> leaves a conventional hostname untouched and still keeps two hosts distinct, which owner_is_stale relies on to refuse to probe a foreign PID.

A marker written raw by an older release on such a host no longer matches the escaped name, so owner_is_stale reads it as foreign and declines to reclaim it, which fails closed rather than detaching a live holder.

Tests pin the escape table, and drive each lock family through its public API on four out-of-grammar hostnames. Reverting host_name() to the raw name fails 11 of them.

a host whose socket.gethostname() carries a space or a non-ascii byte made the writer publish a marker its own heartbeat could not parse, so the heartbeat stopped and a peer could evict the still-held marker; a non-ascii name made the ascii encode raise and the acquire fail. normalise the hostname where the marker is written so it always parses back.
@gaborbernat gaborbernat changed the title keep a soft read/write marker's hostname within the reader grammar 🐛 fix: escape the hostname every marker publishes Aug 22, 2026
The soft read/write writer was not the only one bound by a grammar its
own reader enforces. A hostname carrying a space also fails the
StrictSoftFileLock claim validator, and one carrying a byte outside
UTF-8 (which Python hands back as a surrogate) fails every writer with
UnicodeEncodeError, the soft read/write state lock included. A newline
forges an extra line in a protocol 1 marker, so its own holder no longer
recognizes it.

Escape in host_name() instead, where all four writers read the hostname,
and drop the soft read/write copy. '?' is illegal in a hostname, so
escaping it alongside every out-of-grammar byte as '?<hex>' leaves a
real hostname untouched and keeps two hosts distinct, which
owner_is_stale needs to refuse to probe a foreign PID.
@gaborbernat
gaborbernat force-pushed the soft-rw-marker-hostname-grammar branch from 1776f27 to 798fe90 Compare August 22, 2026 22:19
@gaborbernat gaborbernat changed the title 🐛 fix: escape the hostname every marker publishes escape the hostname every marker publishes Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants