Store an edit the server won't take as a conflict

docs/explanation/adapters.rst

@@ -85,24 +85,29 @@ A member that comes back without data, as python-caldav returns a 404 entry, or
 
 .. _caldav-fingerprint:
 
-Why the fingerprint leaves out three properties
+Why the fingerprint leaves out what servers set
 -----------------------------------------------
 
-Tested servers rewrite what they store: they add ``DTSTAMP``, reorder properties and parameters, fold lines at other points, and reorder the parts of a recurrence rule, which may come in any order (:rfc:`5545#section-3.3.10`). A server sets ``DTSTAMP`` and ``LAST-MODIFIED`` itself (:rfc:`5545#section-3.8.7.2`, :rfc:`5545#section-3.8.7.3`), and ``PRODID`` names the product that created the object (:rfc:`5545#section-3.7.3`). So the canonical form unfolds the lines, sorts the components, the properties and parameters within each, and the parts of each recurrence rule, and leaves those three properties out. A server's copy of your own write then fingerprints the same as what you sent, on every tested server. ``UID`` is kept, since it's the logical key, and so is everything else, because any other difference can be a real edit. Calendar data is UTF-8 unless a charset says otherwise (:rfc:`5545#section-3.1.4`): python-caldav hands the adapter what a server sent as UTF-8, and the adapter refuses an edit that isn't.
+Tested servers rewrite what they store: they add ``DTSTAMP``, reorder properties and parameters, fold lines at other points, and reorder the parts of a recurrence rule, which may come in any order (:rfc:`5545#section-3.3.10`). A server sets ``DTSTAMP`` and ``LAST-MODIFIED`` itself (:rfc:`5545#section-3.8.7.2`, :rfc:`5545#section-3.8.7.3`), and ``PRODID`` names the product that created the object (:rfc:`5545#section-3.7.3`). A server that schedules an event adds a ``SCHEDULE-STATUS`` parameter to its attendees and organizer (:rfc:`6638#section-7.3`), and keeps no ``SCHEDULE-FORCE-SEND`` parameter (:rfc:`6638#section-7.2`). So the canonical form unfolds the lines, sorts the components, the properties and parameters within each, and the parts of each recurrence rule, and leaves out those three properties and those two parameters. A server's copy of your own write then fingerprints the same as what you sent, on every tested server. The price is that an edit changing only those properties or parameters isn't pushed; a client shouldn't change ``SCHEDULE-STATUS`` anyway, and a server doesn't keep ``SCHEDULE-FORCE-SEND``. ``UID`` is kept, since it's the logical key, and so is everything else, because any other difference can be a real edit. Calendar data is UTF-8 unless a charset says otherwise (:rfc:`5545#section-3.1.4`): python-caldav hands the adapter what a server sent as UTF-8, and the adapter refuses an edit that isn't.
 
 Why writes are conditional
 --------------------------
 
 Updates and deletes send ``If-Match`` with the base ETag (:rfc:`9110#section-13.1.1`), so the server applies them only if the member still has that ETag, and creates send ``If-None-Match: *`` (:rfc:`9110#section-13.1.2`), so a create never replaces an existing member. A mismatch gets status 412, and the edit stays pending for the next :term:`ingest`. ``If-Match`` compares ETags strongly, so a weak one (``W/``) never matches; an edit to a member whose stored ETag is weak waits, without being sent, until a sync stores a strong one, and :meth:`~calendaring_sync.adapters.base.SyncAdapter.waiting` lists such edits. The ETag comes from the ``DAV:getetag`` property of the server's answer, not from an HTTP header a proxy can weaken, so edits wait only on a server that reports a weak ETag itself.
 
-Creates and updates use python-caldav's ``save()`` with ``raw_data``, which sends the bytes you recorded unchanged, and ``precondition="etag"``, which sends the stored ETag. Updates also pass ``increase_seqno=False``: ``save()`` increments an event's ``SEQUENCE`` revision number by default, and though it leaves the bytes passed as ``raw_data`` alone, the flag says outright that the server receives the edit you recorded. Creates pass ``no_overwrite=True``, which sends ``If-None-Match: *``. Deletes use ``delete(if_match=True, precondition="etag")``. A 404 on an update or a delete means the member is gone; a tested server answers a delete of a gone member with 412 instead, which leaves the delete pending until the next ingest reports the removal.
+Creates and updates use python-caldav's ``save()`` with ``raw_data``, which sends the bytes you recorded unchanged, and ``precondition="etag"``, which sends the stored ETag. Updates also pass ``increase_seqno=False``: ``save()`` increments an event's ``SEQUENCE`` revision number by default, and though it leaves the bytes passed as ``raw_data`` alone, the flag says outright that the server receives the edit you recorded. Creates pass ``no_overwrite=True``, which sends ``If-None-Match: *``. Deletes use ``delete(if_match=True, precondition="etag")``. A 404 or a 410 on an update or a delete means the member is gone (:rfc:`9110#section-15.5.5`, :rfc:`9110#section-15.5.11`); a tested server answers a delete of a gone member with 412 instead, which leaves the delete pending until the next ingest reports the removal.
 
 A create uses an href of unreserved characters only (:rfc:`3986#section-2.3`), a UUID and ``.ics``, so it can't come back spelled differently. A server that already holds the UID answers with an error element, ``CALDAV:no-uid-conflict`` (:rfc:`4791#section-5.3.2.1`) or ``CALDAV:unique-scheduling-object-resource`` (:rfc:`6638#section-3.2.4.1`); python-caldav raises ``UidConflictError`` for either, whatever the status, so the adapter never mistakes it for a plain precondition failure. For a create or an update, that's a :term:`conflict` with the object that holds the UID, named when the error element holds its ``DAV:href`` and that href is in the calendar; otherwise the core finds the holder by its UID.
 
 Unless the client is configured with the server's profile, python-caldav searches the calendar for the UID of a create before sending it, which costs a request per create and refuses a UID that's taken even on a server that would accept it. Configuring the client's ``features`` with the server's name lets the server check instead.
 
 A server MUST NOT return a strong ETag for data it stored differently (:rfc:`4791#section-5.3.4`), but tested servers do, so an applied push doesn't keep the ETag it got back.
 
+Why a write refused with a 4xx is a conflict
+--------------------------------------------
+
+A 4xx status says the client seems to have erred (:rfc:`9110#section-15.5`), so the same write would get the same answer. Tested servers answer that way for a calendar you may only read, with a bare 403 or one naming ``DAV:need-privileges`` (:rfc:`3744#section-7.1.1`), and for content a calendar won't store, with a 400, a 415, or a 403 naming a precondition such as ``CALDAV:supported-calendar-component`` (:rfc:`4791#section-1.3`). So a write refused with a 4xx becomes a :term:`conflict` the server didn't allow, which holds the status, the precondition names and the server's explanation as its reason. A member that's gone and a mismatch (412) aren't refusals, as above, and a few 4xx statuses mean something else and are raised instead, leaving the push :term:`in flight` for the next sync: a login (401, 407), and a request worth repeating later (408, :rfc:`9110#section-15.5.9`; 425, :rfc:`8470#section-5.2`; 429, :rfc:`6585#section-4`). A 507 is a conflict too: a write that got it "MUST NOT be repeated until it is requested by a separate user action" (:rfc:`4918#section-11.5`), and your resolution is that action. Any other 5xx is raised.
+
 JMAP
 ====
 
@@ -187,7 +192,7 @@ Pages are counted by position, so an event destroyed while the listing runs shif
 Why every create carries its own UID
 ------------------------------------
 
-A server generates a ``uid`` only when a create omits it (draft-ietf-jmap-calendars-33 section 5.9). The adapter refuses a create without one, and a change that would leave a create it hasn't sent yet without one, so a create whose result was lost is recognized when it comes back under a server-assigned id. A create the server refuses because another event holds its ``uid`` (``invalidProperties`` naming ``uid``) becomes a :term:`conflict`; the server doesn't name that event, so the core finds it by the logical key. A tested server refuses an instance whose ``uid`` another instance in the same calendar holds by naming ``calendarIds`` instead, as it does for a calendar that doesn't exist, so the adapter counts that refusal as the same conflict only when the scope holds another event with that ``uid`` in one of the event's calendars, and raises it otherwise. Both tested servers keep an event's ``uid`` when an update sends another one, so a change can't take a ``uid`` either.
+A server generates a ``uid`` only when a create omits it (draft-ietf-jmap-calendars-33 section 5.9). The adapter refuses a create without one, and a change that would leave a create it hasn't sent yet without one, so a create whose result was lost is recognized when it comes back under a server-assigned id. A create the server refuses because another event holds its ``uid`` (``invalidProperties`` naming ``uid``) becomes a :term:`conflict`; the server doesn't name that event, so the core finds it by the logical key. A tested server refuses an instance whose ``uid`` another instance in the same calendar holds by naming ``calendarIds`` instead, as it does for a calendar that doesn't exist, so the adapter counts that refusal as the same conflict only when the scope holds another event with that ``uid`` in one of the event's calendars, and otherwise as a write the server won't take. Both tested servers keep an event's ``uid`` when an update sends another one, so a change can't take a ``uid`` either.
 
 Why writes use ifInState
 ------------------------
@@ -196,6 +201,11 @@ A ``/set`` with ``ifInState``, the state the client expects, fails with ``stateM
 
 That state covers the whole account, so a rejection means something changed, not necessarily this object. The rest of the sync's pushes would be rejected too, so they aren't sent: every edit stays pending for the next sync, which ingests the change first. Another client's delete moves the state as well, so a delete of an event someone else deleted is rejected before the server would say it's gone; the next ingest reports the removal. When a server does answer ``notFound``, the push ended gone: a change becomes a :term:`conflict` and a delete is done.
 
+Why some SetErrors are conflicts
+--------------------------------
+
+A ``/set`` refuses one record with a SetError (:rfc:`8620#section-5.3`). Those that say the same write would be refused again make it a :term:`conflict` the server didn't allow, with the error's type and the properties it names as its reason: ``forbidden`` (a write a permissions policy refuses, as a tested server answers for a calendar shared with you to read), ``overQuota``, ``tooLarge``, ``invalidPatch``, ``singleton``, ``invalidProperties`` other than a taken ``uid``, and ``noSupportedScheduleMethods`` (draft-ietf-jmap-calendars-33 section 5.9.2), as does an account that's read-only (``accountReadOnly``, :rfc:`8620#section-3.6.2`). Such a refusal is about the one event, and leaves the state where it was, so the rest of the sync's pushes still go out. Any other error is raised, such as ``rateLimit``, which may pass, and ``serverFail``.
+
 Why the adapter syncs events only
 ---------------------------------
 

docs/explanation/design.rst

@@ -180,7 +180,7 @@ Every :term:`record` is in one of five states at a time: synced, pending, in fli
        in_flight --> synced: applied
        in_flight --> removed: delete done
        in_flight --> pending: rejected
-       in_flight --> conflict: gone, or UID taken
+       in_flight --> conflict: gone, UID taken, or not allowed
        conflict --> synced: resolved: server's copy
        conflict --> pending: resolved: push it
        conflict --> removed: resolved: removal
@@ -228,6 +228,9 @@ The table lists every transition with its conditions, including these, which the
    * - in flight
      - Refused because another object holds its logical key
      - Conflict
+   * - in flight
+     - Not allowed by the server
+     - Conflict
    * - in flight
      - Unknown, or the process stopped
      - Settled by the next complete ingest
@@ -239,7 +242,7 @@ The table lists every transition with its conditions, including these, which the
      - Pending again, unless another object in the scope still holds the key
    * - conflict
      - Resolved
-     - Synced, pending, or removed, depending on the resolution; a create the server refused is deleted from the store when you discard it or push it as an update to the object holding its logical key
+     - Synced, pending, or removed, depending on the resolution; a create the server refused is deleted from the store when you keep the server's copy or delete it, or push it as an update to the object holding its logical key
    * - removed
      - Remote change: the server holds an object at that id again
      - Synced with the new content, as the same record
@@ -281,7 +284,7 @@ The core classifies each change by looking only at the :term:`synced copy`, the
    * - ``CONFLICT``
      - Both sides acted; a conflict record is stored, of one :class:`~calendaring_sync.objects.conflict.ConflictKind`
 
-Later remote changes update the record's :term:`server's copy`. The conflict resolves itself when the server reaches your edit, goes back to the synced copy, or removes an object you deleted too. An edit refused for its logical key goes back to pending once the object holding that key no longer holds it.
+Later remote changes update the record's :term:`server's copy`. The conflict resolves itself when the server reaches your edit, goes back to the synced copy, or removes an object you deleted too. An edit the server refused holds no copy of its own: for a change or a delete, the server's copy is the synced copy, so only another copy, or a removal, is news, and the server reporting the synced copy again leaves the refusal open. An edit refused for its logical key goes back to pending once the object holding that key no longer holds it. One the server didn't allow stays open until you resolve it, since nothing tells the client that the server would take it now, so only you know when sending it again makes sense.
 
 The core stores conflicts instead of raising them: an exception for one object would abort every other change in the sync, and a :term:`resolver` called during a sync would run again on every retry. So you resolve them when you choose, outside any sync, each with a :data:`~calendaring_sync.objects.resolution.Resolution`: one with :meth:`~calendaring_sync.replica.Replica.resolve`, or every open one through your :term:`resolver` with :meth:`~calendaring_sync.replica.Replica.resolve_conflicts`. Anything but the server's copy becomes a pending edit over the server's current copy, so its push is checked against the server's current version. :meth:`~calendaring_sync.replica.Replica.resolve` refuses a record whose conflict changed since you read it, so a decision is never applied to news you haven't seen.
 

docs/how-to/conflicts.rst

@@ -29,3 +29,5 @@ Resolve every conflict with a function
    :end-before: # end: merge
 
 :meth:`~calendaring_sync.replica.Replica.resolve_conflicts` calls your function with each open conflict, and ``None`` leaves one open. It returns a :class:`~calendaring_sync.objects.resolution.ResolvedConflicts` saying which conflicts it resolved, left open, or found changed while your function ran. A :class:`~calendaring_sync.objects.resolution.Merge` needs the fields the adapter reads from content, which :meth:`~calendaring_sync.adapters.base.Adapter.edit_fields` returns. :attr:`Record.conflict <calendaring_sync.objects.record.Record.conflict>` holds the :term:`server's copy`, and :class:`~calendaring_sync.objects.conflict.ConflictKind` says what each side did.
+
+An edit the server won't take as you sent it, such as one to a calendar you may only read, is a conflict of kind :attr:`~calendaring_sync.objects.conflict.ConflictKind.NOT_ALLOWED`. Its :attr:`~calendaring_sync.objects.conflict.Conflict.reason` holds the server's explanation, to show to the person who made the edit. It stays open until you resolve it; :class:`~calendaring_sync.objects.resolution.KeepLocal` sends the edit again, for once the server would take it.

docs/how-to/errors.rst

@@ -15,4 +15,4 @@ Every error calendaring-sync raises on purpose derives from :class:`~calendaring
    :start-after: # start: sync
    :end-before: # end: sync
 
-The client library's own errors, such as a refused login, reach you unchanged. Each adapter's class description names them.
+Errors from the client library, such as a refused login, reach you unchanged, and each adapter's class description names them. A server refusing one edit isn't among them: that edit becomes a :term:`conflict`, as :doc:`conflicts` explains.

docs/how-to/write-an-adapter.rst

@@ -29,7 +29,7 @@ Read changes
 Send edits
 ==========
 
-:meth:`~calendaring_sync.adapters.base.Adapter._send` maps each answer your protocol gives to a :data:`~calendaring_sync.objects.push.PushOutcome`: here a version mismatch is :class:`~calendaring_sync.objects.push.Rejected`, and an object the server doesn't have is :class:`~calendaring_sync.objects.push.Gone`. Any other error propagates.
+:meth:`~calendaring_sync.adapters.base.Adapter._send` maps each answer your protocol gives to a :data:`~calendaring_sync.objects.push.PushOutcome`: here a version mismatch is :class:`~calendaring_sync.objects.push.Rejected`, and an object the server doesn't have is :class:`~calendaring_sync.objects.push.Gone`. Map a refusal about rights or content to :class:`~calendaring_sync.objects.push.NotAllowed`, with the server's explanation as its reason, so the edit becomes a conflict instead of stopping every sync. Any other error propagates.
 
 Test it
 =======

docs/reference/glossary.rst

@@ -29,7 +29,7 @@ These are the sync terms calendaring-sync's pages use. :doc:`../explanation/desi
       To mark a :term:`pending edit` :term:`in flight` before it's pushed, in one step that fails if another process got there first. :ref:`design-states` shows a record's states and what moves it between them.
 
    conflict
-      An object where your :term:`pending edit` and the server disagree: both sides changed it, one side deleted it while the other edited it, or the server refused your create or change because another object holds its :term:`logical key`. calendaring-sync reports it and leaves the decision to you.
+      An object where your :term:`pending edit` and the server disagree: both sides changed it, one side deleted it while the other edited it, the server refused your create or change because another object holds its :term:`logical key`, or the server won't take your edit as you sent it. calendaring-sync reports it and leaves the decision to you.
 
    conflict record
       What the core stores for one :term:`conflict`: the :term:`synced copy`, your :term:`local edit`, the :term:`server's copy`, and the kind of conflict. It stays open until you resolve it or it resolves itself.

docs/reference/guarantees.rst

@@ -33,6 +33,8 @@ Guarantees
      - :ref:`design-server-answer`
    * - An object the server removed while you edited it is always a :term:`conflict`, never resolved for you.
      - :ref:`design-conflicts`
+   * - An edit the server won't take as it was sent becomes a :term:`conflict`, so it never stops a sync, or the edits after it, and isn't sent again until you resolve it.
+     - :ref:`design-conflicts`
    * - A conflict that changed since you read it can't be resolved, so a decision never applies to changes you haven't seen.
      - :ref:`design-conflicts`
    * - A server's copy of your own write isn't reported as a change, however the server rewrote it.