Skip to content

Revision cleanup: should the age of a revision be the age of its content, or the time it was replaced? #3735

Closed
opencloud-eu/reva
#895

Description

@pbleser-oc

Context

With #1419, OpenCloud deletes old revisions (versions) of files according to a cleanup strategy (OC_REVISIONS_CLEANUP_STRATEGY), e.g. 30d deletes the revisions older than 30 days, and auto keeps fewer revisions the older they get. The cleanup also runs on demand with opencloud storage-users revisions cleanup.

Durations and the time slots of auto are compared with the age of each revision. There are two ways to define that age, and they lead to quite different results for files that are not changed often. This issue is meant to decide which one we want, before the cleanup is used widely.

Current behaviour: the time the content was written

Today, the age of a revision is the time its content was written, i.e. the modification time the file had before it was changed. When a file is changed, its previous content is turned into a revision, and that revision is named after the previous modification time of the file (<node id>.REV.<time>), which is also the date the version list shows for it. Revisions are never modified afterwards.

It is not the time the file was changed, which turned that content into a revision, i.e. when the content was replaced by a newer one. That time is not recorded anywhere today.

For files that change often, both times are close together. For files that stay unchanged for a long time, they are far apart:

  1. On August 14, a document is saved.
  2. Nobody touches it for two months.
  3. On October 14, it is changed. A revision is created to keep the previous content, dated August 14.
  4. The cleanup runs right after the change. The revision counts as 61 days old.

With a strategy that deletes by duration, e.g. 30d, auto, 30d or 7d, 30d, that revision is deleted right away, and the content the document had until that morning is gone. Strategies that keep a number of revisions (e.g. 5, 30d) or auto alone (one per week, however old) don't have this problem.

This is documented (in the README of the storage-users service and in the description of OC_REVISIONS_CLEANUP_STRATEGY), with the advice to combine durations with a number of revisions.

Alternative: the time the content was replaced

The age of a revision would be the time its content stopped being current, i.e. when the revision was created. In the example above, the revision would count as 0 days old on October 14, and 30d would keep it until November 13.

That arguably matches what users expect from "keep the versions of the last 30 days": being able to go back to any state the file had during the last 30 days.

What it would take:

  • Record the creation time of every new revision as metadata, wherever revisions are created: uploads that replace a file (decomposed, decomposeds3, posix), restoring a revision, and with posix, revisions created from changes on disk (enable_fs_revisions).
  • Existing revisions don't have it. A fallback is needed, e.g. the modification time of the next newer revision (the content that replaced it), or of the current file for the newest revision. That is only an approximation: modification times can come from clients.
  • The reasons the cleanup gives for its decisions (in its log and in the output of the command) would say e.g. "replaced before …" instead of "older than …".
  • The version list would still show the time the content was written. The cleanup would then sometimes keep a version that looks older than the configured duration; that needs to be explained in the documentation.

Related: revisions dated in the future

Modification times come from clients (X-OC-Mtime, wrong client clocks) or from changes on disk with posix (touch, restores, skewed clocks on network storage). So a revision can be dated in the future.

Today, such a revision counts as the newest one:

  • it takes one of the first places of a number of revisions to keep, e.g. with 2, it can push out one of the actual 2 newest revisions;
  • no duration deletes it before its date has passed;
  • auto keeps it in a time slot in the future.

The cleanup logs a warning for every such revision and mentions it in its reason, but doesn't change its decision. Alternatives that were analysed and rejected:

  • counting it as "now" (clamping) has no effect on numbers and durations, and makes auto delete the actual current revision;
  • counting it as the oldest one trades one risk of losing data for another.

A creation time recorded by the server (the alternative above) would solve this as well, as it doesn't depend on the clocks of clients.

Questions for the team

  1. Which age should the cleanup use: the time the content of a revision was written (current), or the time it was replaced?
  2. If the time of replacement: is the fallback for existing revisions good enough, or should existing revisions keep the current behaviour (e.g. until they are gone)?
  3. Should the version list (and the API) show the time a version was replaced as well, so that what users see matches what the cleanup does?
  4. How should revisions dated in the future be handled, if we keep the time the content was written?

Notes

  • The current behaviour and its limits are documented, so this is not blocking: admins can avoid the problem today by combining a duration with a number of revisions (e.g. 5, 30d) or by using auto.
  • Recording the creation time is a change of its own in reva (both the decomposed and the posix trees), and should come with the change of the cleanup in one go, so that the cleanup never mixes both meanings for new revisions.

Activity

  1. butonic commented on Oct 21, 2026

    @butonic
    Member

    The current implementation is bug compatible with how oc10 named revisions. Which is arguably wrong. So I would mark this as a bug. Currently, all revisions have the original mtime in their filename, however, the mtime of the revision file should be the time the content was replaced ... unless we explicitly set the mtime manually in the code.

    Merge as bug compatible, then fix? Because I'm not sure if we need a migration or if we can rely on the mtime of the revision.

  2. changed the issue type fromtoon Oct 21, 2026
  3. 5 remaining items

  4. moved this from Qualification to Prio <= 3 in OpenCloud Team Boardon Oct 28, 2026
  5. pbleser-oc commented on Nov 18, 2026

    @pbleser-oc
    MemberAuthor

    An update: in his review of opencloud-eu/reva#895, @aduffeck pointed out that going by the content mtime isn't only surprising, but can be abused. Anyone who can write a file, including through a public link with edit rights, chooses that mtime with X-OC-Mtime:

    • with auto, a revision dated one second after an existing one lands in the same slot and pushes that one out;
    • revisions dated years ahead take the places of a number of revisions to keep, e.g. with 5, 30d, so that every new version is deleted, also after the writer lost access to the file.

    So I've implemented the time of replacement, as a separate last commit in each of the three PRs: opencloud-eu/reva#895 (39a017723), #3778 (80091c1) and opencloud-eu/docs#1222 (81ccf73b). On the questions above:

    1. Which age: the time a revision was created, i.e. when its content was replaced, by the clock of the server. CreateRevision (decomposed and posix) and the posix assimilation record it in a new attribute of the revision, user.oc.revision.created. The cleanup orders the revisions and measures their age by it. auto now keeps the oldest revision of each slot: it holds the content the file had when the slot began, while keeping the newest would drop a version that was current for months as soon as the file is saved twice within a slot.
    2. Existing revisions: rather than falling back to the content age, as I thought in my last comment, they get an estimate of when they were replaced: the content mtime of the next newer revision, or the mtime of the file for the newest one, capped at the current time. That's still based on client mtimes, but only for revisions that exist already, and the cap keeps revisions dated in the future from taking the places of the ones created from now on. Old and new revisions are then treated the same way, up to the accuracy of the estimate.
    3. Version list: unchanged, it still shows the content mtime, as do the revision names and X-OC-Mtime. The docs explain that the cleanup goes by another date than the one the list shows, and the verbose output of the command shows the creation time.
    4. Revisions dated in the future: solved by the above, as the server's clock decides. Only a server clock that was set back could still produce one, and that still gets the warning.

    As this changes what 30d means, I'd like more opinions from the team before it's merged: the description of opencloud-eu/reva#895 asks not to merge it before that's decided. If we go with it, opencloud-eu/reva#895 fixes this issue; if not, the three commits are dropped and this issue stays open.

  6. aduffeck commented on Nov 19, 2026

    @aduffeck
    Member

    I vote for keeping the three commits. Besides closing the hole from the review, going by the time of replacement is the less confusing behaviour.

    • 30d then means what people read into it. Every state the file had during the last 30 days can be restored. With the content age, a document that was unchanged for two months loses its previous content the moment it is edited. Undoing the latest edit is the main reason to have versions at all, and nobody would expect 30d to exclude it.
    • The simple configuration does the right thing. With the content age, admins have to know that a duration needs a count next to it to be safe, and that client clocks can move revisions around. With the replacement time only the server clock counts and both caveats go away.
    • The mismatch that remains errs on the safe side. The version list still shows the content date, so the cleanup sometimes keeps a version that looks older than the configured duration. That is much easier to explain than a version that was current this morning and is gone. We can show the replaced time in the list later (question 3). I wouldn't block on that.
    • There is precedent. S3 lifecycle rules for noncurrent versions count the days since a version became noncurrent.
    • Changing it later is the confusing option. If we release with the content age and switch afterwards, 30d changes its meaning on running instances. Every revision created until then also has no recorded time and depends on the estimate for as long as it exists. Nobody uses the cleanup yet, so the change costs nothing now.

    On question 2, I think the estimate is the right approach, since old and new revisions then follow one rule.

  7. pbleser-oc commented on Nov 19, 2026

    @pbleser-oc
    MemberAuthor

    Thanks @aduffeck for the arguments. With @butonic's okay too, we go with the time of replacement: the age of a revision is the time it was created, recorded by the server, as implemented in the last commit of opencloud-eu/reva#895, #3778 and opencloud-eu/docs#1222. The three commits stay, and opencloud-eu/reva#895 now closes this issue when it is merged.

    Showing the time a version was created in the version list, next to or instead of the date of its content, can come later, as a separate issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions