docs(cleaning): document partition TTL

[deepakpanda93]

Describe the issue this Pull Request addresses

Closes #19141.

Partition TTL has existed since 1.0.0, but it appears nowhere in prose on the site. The only trace is the generated
config reference, which lists the keys and defaults without saying what they do together. Confirmed the gap rather than
assuming it: partition.ttl matches only configurations.md across website/docs, website/versioned_docs and
website/learn.

Summary and Changelog

One file, website/docs/cleaning.md (+121). A Partition TTL section is added before Related Resources.

cleaning.md is the natural home: that page is already about retention, and TTL is the partition-level counterpart to
the cleaner's file-version retention. The section opens on exactly that contrast, since it is the thing most likely to
confuse someone who has already read the cleaner docs:

Cleaning bounds how many versions of a file are kept, but it never removes a partition: an old partition whose files
have all been cleaned down to a single version still sits in the table forever.

The thing a config table cannot tell you

hoodie.partition.ttl.strategy.days.retain defaults to -1, and KeepByTimeStrategy#getExpiredPartitionPaths returns
an empty list whenever the resulting retention is zero or less:

if (!lastCompletedInstant.isPresent() || ttlInMilis <= 0
    || !hoodieTable.getMetaClient().getTableConfig().getPartitionFields().isPresent()) {
  return Collections.emptyList();
}

So TTL does nothing at all until a positive retention is set, even when enabled. That gets a :::caution, phrased
around the failure mode rather than the default value: a misconfigured job looks like a working one, because it runs,
reports no expired partitions, and deletes nothing.

That same guard shows two further silent no-ops which the section also states: a table with no completed commit, and an
unpartitioned table.

What the section covers beyond the configs

Both built-in strategies, not just the default. They age a partition against different timestamps, which is the
practically important difference:

Strategy	Ages against
KEEP_BY_TIME (default)	The partition's last commit time, so an actively written partition survives
KEEP_BY_CREATION_TIME	The partition's created commit time, so writing to it does not extend its life

Also that hoodie.partition.ttl.strategy.class takes precedence over the strategy type when both are set
(PartitionTTLStrategyType#getPartitionTTLStrategyClassName).

All three ways to run TTL, not only the inline config. Inline after each commit, the standalone
org.apache.hudi.utilities.HoodieTTLJob, and the run_ttl Spark SQL procedure. The procedure is cross-referenced to its
existing entry on the procedures page rather than duplicated. Whichever path is used, TTL lands as a replace commit, the
same commit type delete_partition uses (startDeletePartitionCommit then managePartitionTTL, committed with
REPLACE_COMMIT_ACTION).

The three bounding configs framed around the first run, which is where they matter, since every historical partition
becomes a candidate at once. max.delete.partitions caps a run at 1000 so one replace commit cannot grow unmanageably
large, and the code comment says exactly that ("Avoid a single replace commit too large") — a backlog therefore drains
over several runs. partition.selected restricts the candidate set and is the safe way to trial a policy.
stats.max.parallelism bounds the parallelism for collecting candidate commit times.

A config table and a worked example, the example following the quick start guide's own val tableName /
val basePath = "file:///tmp/..." convention so it is runnable as written, and restricting the first pass to one
partition so the effect can be checked before it applies to the whole table. A closing caution states plainly that TTL
deletes data.

One correction worth flagging

The max-partitions config is hoodie.partition.ttl.strategy.max.delete.partitions, not the shorter
hoodie.partition.ttl.max.partitions.to.delete form that circulates in discussion of this feature. Everything defined
through PARTITION_TTL_STRATEGY_PARAM_PREFIX carries the strategy. prefix, so days.retain, partition.selected,
max.delete.partitions and stats.max.parallelism all sit under it. Note also the asymmetry in Hudi's own naming, which
the section reproduces faithfully rather than tidying: the strategy type is
hoodie.partition.ttl.management.strategy.type while the strategy class is hoodie.partition.ttl.strategy.class.

Version scope

website/docs only, deliberately. hoodie.partition.ttl.strategy.stats.max.parallelism is new in 1.3.0 (#19137) and is
absent from release-1.2.0, verified against the tag. The next docs are therefore the only place where every config
described here exists; versioned copies would each need that row removed, which I would rather do as a follow-up if
reviewers want it than ship as six near-duplicate sections.

Verification

Everything read from master, not from the ticket: HoodieTTLConfig for the keys, defaults and since-versions,
KeepByTimeStrategy and KeepByCreationTimeStrategy for expiry semantics and the no-op guard, PartitionTTLStrategy
for candidate selection, BaseHoodieTableServiceClient for the inline trigger, HoodieTTLJob for the CLI parameters,
and RunTTLProcedure for the procedure name and its config mapping.

npm run build passes with the warning block byte-identical to a baseline built from the same base commit
(5971a1ac3ba3), 13,265 lines each. That parity also confirms the new procedures.md#run_ttl cross-reference resolves,
since a dangling anchor would have added a warning. Rendering checked under npm run serve: all six headings resolve
with unique anchors and 12 TOC entries, both cautions render, and the heading is Partition TTL configs rather than a
second Configs, which would have collided with the existing cleaning ### Configs and produced a #configs-1 anchor.

One limitation stated plainly: this is verified by reading the code, not by running TTL against a table. The claim I would
most like a committer to confirm is the practical consequence of the days.retain default, since that is the line
readers will act on.

Impact

Documentation only. No code, config, or behaviour change.

Risk Level

none

Documentation Update

This PR is the documentation update — the cleaning page, /docs/next/cleaning#partition-ttl.

Contributor's checklist

• Read through contributor's guide
• Enough context is provided in the sections above
• Adequate tests were added if applicable

cc @wangxianghu (who filed the issue), @yihua

[hudi-agent]

⚠️ 🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

Thanks for the docs update! This adds a much-needed prose section on Partition TTL to cleaning.md, covering the two built-in strategies, the three ways to run TTL, first-run safety configs, and a worked example. Six of the seven cited configs match the generated config reference exactly and the run_ttl cross-reference is correct; the main thing to double-check is the parallelism config below, plus a minor redundancy in the spark-submit example. Please have a Hudi committer (e.g. @yihua) confirm before merging.

[hudi-agent]

🤖 This config, hoodie.partition.ttl.strategy.stats.max.parallelism, is the one TTL key in this section that does not appear in the generated config reference (configurations.md) — the other six all do, with matching Config Param names. Could you confirm the exact key name and its 200 default against HoodieTTLConfig in the source? If it exists but isn't surfaced in the reference, it may be worth noting; if the key is slightly off, a user setting it would silently get no effect. The accompanying claim that "the effective value is the smaller of that and the candidate count" would also be good to verify against the strategy implementation. @yihua could you sanity-check this config key/default?

_{⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.}

[deepakpanda93]

Good catch on the discrepancy, and thanks for asking rather than assuming the key was wrong. Addressed in 55c8b48.

The key and default are correct. From HoodieTTLConfig on master:

public static final String PARTITION_TTL_STRATEGY_PARAM_PREFIX = "hoodie.partition.ttl.strategy.";

public static final ConfigProperty<Integer> STATS_MAX_PARALLELISM = ConfigProperty
    .key(PARTITION_TTL_STRATEGY_PARAM_PREFIX + "stats.max.parallelism")
    .defaultValue(200)
    .sinceVersion("1.3.0")

so the full key is hoodie.partition.ttl.strategy.stats.max.parallelism, default 200.

The min claim also holds. KeepByTimeStrategy#getLastCommitTimeForPartitions:

int statsParallelism = Math.min(partitionPaths.size(), writeConfig.getPartitionTTLStatsMaxParallelism());

You found a real gap, though, just not the one you suspected. The reason it is missing from configurations.md is sinceVersion("1.3.0"): the config does not exist in release-1.2.0 (I checked the tag directly, zero occurrences), and the generated reference reflects a released version. So a reader on 1.2.0 who copied that line would set a key that silently does nothing, which is exactly the failure mode you were worried about, arriving by a different route.

Rather than drop the config, the section now says so explicitly. The prose adds:

This config is new in 1.3.0, so it has no effect on earlier releases and does not yet appear in the generated configuration reference; the other six configs above do.

and the table row is marked Since 1.3.0.

This is also why the PR is scoped to website/docs only, which the description covers: the versioned copies would each need this row removed, since 1.2.0 and earlier genuinely lack the config.

Worth noting for completeness that your count is right on the other side too. I verified each of the seven keys against docs/configurations.md: six match with one occurrence each, and this one is the sole absentee.

[hudi-agent]

🤖 The standalone-job example both pulls the utilities slim bundle via --packages and passes a locally built hudi-utilities-slim-bundle-*.jar (via the backtick ls) as the application jar. That's a bit contradictory — a reader with a local build wouldn't also need --packages for the same bundle, and a reader without one won't have the jar the ls expects. It might help to pick one path (either the Maven coordinates or the local jar) so the command is copy-paste runnable.

_{⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.}

[deepakpanda93]

You're right, and it was contradictory in exactly the way you describe. Fixed in 55c8b48.

The command now takes one path only, the self-contained utilities bundle as the application jar:

spark-submit --master local \
  --class org.apache.hudi.utilities.HoodieTTLJob \
  hudi-utilities-bundle_2.12-1.2.0.jar \
  --base-path file:///tmp/events_table \
  --hoodie-conf hoodie.partition.ttl.strategy.days.retain=30

followed by a line saying the bundle is self-contained so no --packages is needed, and where to get it: Maven Central, or packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.12-*.jar from a local build. I confirmed hudi-utilities-bundle_2.12 is published on Maven Central before pointing readers at it.

One thing I should own up to about how the redundancy got there. I did not write that invocation from scratch: I copied the convention from the HoodieCleaner examples further up this same page, on the reasoning that matching the page's existing style was safer than inventing my own. Those examples do the identical thing, --packages for the slim bundle plus a locally built slim bundle jar via ls, at lines 157 and 181 of cleaning.md. So the flaw was inherited rather than introduced, which is a decent argument for not treating "consistent with the surrounding page" as the same thing as "correct".

I have deliberately not changed those HoodieCleaner examples. They carry the same redundancy and the same local-build assumption, and fixing them would be worth doing, but it is unrelated to partition TTL and would widen a docs PR about one feature into an edit of the page's long-standing cleaner examples. Happy to do it as a follow-up if a committer agrees the full-bundle form is the one the project wants; if instead the slim-bundle-plus---packages form is deliberate for some reason I am not seeing, then this new command should probably be reverted to match it rather than the other way round. That is the one open question here.

Build passes with the warning block still byte-identical to a baseline at the same base commit, and I checked the rendered page to confirm the TTL command region contains neither --packages nor slim-bundle, while the 12 remaining slim-bundle mentions are all in the untouched cleaner examples.

[hudi-agent]

⚠️ 🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

Thanks for the docs update! This adds a well-structured Partition TTL section to cleaning.md, and I verified the config keys, defaults, since-versions, both strategy behaviors, the HoodieTTLJob CLI args, and the run_ttl signature all match the source. One correctness concern in the worked example around the partition path format; please have a Hudi committer (e.g. @yihua) do a final pass.

[hudi-agent]

🤖 The partition.selected value here uses a Hive-style path (event_date=2026-01-01), but the write in this example does not set hoodie.datasource.write.hive_style_partitioning, which defaults to false on the DataSource write path. With the default, the on-disk partition path is just the value (2026-01-01), so partition.selected=event_date=2026-01-01 would match no partition — and TTL would silently delete nothing, which is exactly the no-op failure mode this section warns about. It might help to either use event_date=2026-01-01 alongside an explicit .option("hoodie.datasource.write.hive_style_partitioning", "true"), or change the selected value to 2026-01-01 to match the default layout.

_{⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.}