Skip to content

Sort parsed documents recursively with sort_keys - #617

Open
Sreekant13 wants to merge 2 commits into
python-poetry:masterfrom
Sreekant13:fix-614-recursive-sort-keys
Open

Sreekant13 wants to merge 2 commits into
python-poetry:masterfrom
Sreekant13:fix-614-recursive-sort-keys

Conversation

@Sreekant13

Copy link
Copy Markdown

Fixes #614.

Problem

dumps(doc, sort_keys=True) sorts only the top-level keys of a document produced by parse() / loads(). Nested tables, inline tables and arrays of tables keep their original order, even though the same option applied to a plain dict sorts recursively.

import tomlkit
doc = tomlkit.parse("[zeta]\nb = 2\na = 1\n\n[alpha]\nd = 4\nc = 3\n")
print(tomlkit.dumps(doc, sort_keys=True))
# top level is sorted (alpha before zeta), but inside each table b still comes before a

Cause

item() returns an already-parsed Table/InlineTable/AoT unchanged. For a parsed document the top-level container reaches the dict branch and is rebuilt sorted, but each nested value is already such an item, so the recursive item(value, _sort_keys=True) call returns it as is and never sorts its keys. A plain dict has plain-dict children, which recurse and sort.

Change

Under _sort_keys, item() returns a sorted copy of a parsed table-like item. The copy:

  • reorders each container's body by key;
  • keeps every key's leading comment and whitespace lines with it;
  • orders a value that renders as its own section (a table or array of tables) after the inline values, matching the plain-dict branch, so a key never folds under a preceding section header;
  • keeps inline tables inline;
  • leaves the input document unmutated (it works on a deep copy).

A container holding an out-of-order table is left in place, since its entries cannot be linearly reordered, but its nested tables are still sorted.

Tests

Added test_dumps_with_sorted_keys_recurses_into_a_parsed_document, covering nested tables, an inline table, an array of tables, a scalar kept before a sub-table section, and that the document is not mutated. It fails on main and passes with this change. The rest of the suite passes (the toml-test conformance submodule, which this change does not touch, is not exercised here).

dumps(doc, sort_keys=True) sorted only the top-level keys of a parsed
document: item() returned an already-parsed Table/InlineTable/AoT
unchanged, so only the top-level container reached the sorting rebuild.
A plain dict, whose nested values are plain dicts, already sorted
recursively.

item() now returns a sorted copy of a parsed table-like item. The copy
reorders each container's body by key, keeps every key's leading comment
and whitespace lines with it, orders a value that renders as its own
section after the inline values so a key never folds under a preceding
header, keeps inline tables inline, and leaves the input document
unmutated. A container holding an out-of-order table is left in place,
since its entries cannot be linearly reordered, but its nested tables
are still sorted.
The _sort_container_body annotation referenced Container, which items.py
otherwise imports lazily inside item(); add it to the TYPE_CHECKING block
so ruff no longer flags F821 (undefined name).
@Master-Norna

Copy link
Copy Markdown

Verified locally: full suite passes, the new test fails on master as claimed, and I spot-checked the edge cases (comments following keys, out-of-order fragments, no input mutation, deep nesting, top-level AoT). Looks good to me — the (a)/(b) direction call is for the maintainers, but I lean (a) per the comment on #614.

@Sreekant13

Copy link
Copy Markdown
Author

Thanks for the thorough local verification, and agreed on (a): sort_keys=True reading the same on a parsed document as on a plain dict is the least surprising behavior, and since each key keeps its leading comments only the order changes.

On your two observations, both are pre-existing and I would keep them out of this PR:

  • The top-level leading-comment drop comes from the dict-rebuild branch that already runs for parsed documents on master, so it is orthogonal to the nested-sort fix and is better as its own change.
  • Inline tables inside a plain array: this path does not recurse into Array elements, and as you note the plain-dict path renders such values as an array of tables, so the two paths do not yet agree on the shape. I left it alone rather than encode one side of that inconsistency here.

Happy to follow up on either separately if the maintainers would like.

@Master-Norna

Copy link
Copy Markdown

Agreed on the scoping — keeping both observations out of this PR is the right call. Nothing further from my side; the rest is up to the maintainers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dumps(sort_keys=True) only sorts the top level of a parsed document (nested tables / inline tables / AoT keep original order)

2 participants