Stores built from a storage, and resolutions that apply themselves

docs/conf.py

@@ -158,3 +158,12 @@
     "members": True,
     "show-inheritance": True,
 }
+
+
+def _public_bases(app, name, obj, options, bases):
+    """Leave private base classes, such as the one every resolution shares, out of "Bases"."""
+    bases[:] = [base for base in bases if not base.__name__.startswith("_")]
+
+
+def setup(app):
+    app.connect("autodoc-process-bases", _public_bases)

docs/contribute.rst

@@ -48,7 +48,7 @@ How the code is laid out
    * - ``replica.py``, ``session.py``, ``objects/``, ``_classify.py``
      - The core: the replica, sync sessions, the classifier, and the values they take and return
    * - ``store/``
-     - The store interface, and the stores that ship
+     - The store and storage interfaces, and the stores that ship
    * - ``adapters/base.py``
      - The adapter base classes
    * - ``adapters/<protocol>.py``

docs/examples/keeper.py

@@ -0,0 +1,44 @@
+# SPDX-FileCopyrightText: 2026 calendaring-sync contributors
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+# start: keeper
+import pickle
+from pathlib import Path
+
+from calendaring_sync import MemoryData, MemoryKeeper, MemoryStore, Replica, StoreError
+
+
+class FileKeeper(MemoryKeeper):
+    """Keeps a memory store's data in one file, replaced whole on each save."""
+
+    def __init__(self, path: Path) -> None:
+        self.path = path
+
+    def load(self) -> MemoryData | None:
+        if not self.path.exists():
+            return None
+        data: MemoryData = pickle.loads(self.path.read_bytes())
+        return data
+
+    def save(self, data: MemoryData) -> None:
+        # Written beside the file and moved over it, so a failure halfway keeps the last save.
+        written = self.path.with_name(self.path.name + ".new")
+        written.write_bytes(pickle.dumps(data))
+        written.replace(self.path)
+
+    def error(self, exc: Exception) -> StoreError | None:
+        return StoreError(str(exc)) if isinstance(exc, OSError) else None
+
+
+replica = Replica(MemoryStore(keeper=FileKeeper(Path("sync.pickle"))))
+# end: keeper
+
+from memory_adapter import MemoryAdapter, MemoryServer
+
+server = MemoryServer()
+server.put("meeting", b"Team meeting", if_version=None)
+MemoryAdapter(replica, server, "work").sync()
+
+# The next process loads what the last one saved.
+reopened = Replica(MemoryStore(keeper=FileKeeper(Path("sync.pickle"))))
+meeting = reopened.find_record("work", "meeting")

docs/explanation/design.rst

@@ -46,7 +46,7 @@ Why the parts around the core can be swapped
 
 The :term:`core` holds every sync rule and meets the rest through narrow interfaces, so each part around it can change without the core or the other parts changing:
 
-- where the state lives: a SQLite file, memory, or storage of your own, behind one store interface, as :doc:`../how-to/store` shows
+- where the state lives: a SQLite file, memory, or a :term:`storage` of your own, behind one interface, as :doc:`../how-to/store` shows
 - where changes come from: an adapter that ships with calendaring-sync, one you write, or your own code handing the core :term:`change sets <change set>`, as :doc:`../how-to/write-an-adapter` and :doc:`../how-to/without-adapter` show
 - what reads and edits the data: the core never reads inside :term:`content`, so your application uses whichever library suits the format
 - how a sync runs: with a synchronous or an asynchronous client, for one :term:`scope` or many across servers and accounts, from one thread or several, as :doc:`../how-to/async` and :doc:`../how-to/parallel` show
@@ -108,7 +108,7 @@ You can also ask for a full listing at any time, and each scope records when its
 Why local edits are recorded, not discovered
 ============================================
 
-You tell calendaring-sync about each :term:`local edit` as you make it: a change, a delete, or a create, which returns the new record's :term:`local key`. Repeated edits to one object collapse into one, an edit back to the :term:`synced content` cancels itself, and you can discard an edit. You can't edit an object with an open :term:`conflict` until you resolve it.
+You tell calendaring-sync about each :term:`local edit` as you make it: a change, a delete, or a create, which returns the new record's :term:`local key`. Repeated edits to one object collapse into one, an edit back to the :term:`synced copy` cancels itself, and you can discard an edit. You can't edit an object with an open :term:`conflict` until you resolve it.
 
 Comparing every object at every sync instead would cost a full comparison each time, and a difference can't tell whether your application edited an object or never loaded its latest version.
 
@@ -183,7 +183,7 @@ Every :term:`record` is in one of five states at a time: synced, pending, in fli
        in_flight --> removed: delete done
        in_flight --> pending: rejected
        in_flight --> conflict: gone, or UID taken
-       conflict --> synced: resolved: server's side
+       conflict --> synced: resolved: server's copy
        conflict --> pending: resolved: push it
        conflict --> removed: resolved: removal
        removed --> [*]: purged
@@ -210,7 +210,7 @@ The table lists every transition with its conditions, including these, which the
      - Local edit recorded
      - Pending
    * - pending
-     - Edit back to the synced content, or discarded
+     - Edit back to the synced copy, or discarded
      - Synced; a create the server never had is deleted from the store instead
    * - pending
      - Remote change or removal ingested
@@ -235,7 +235,7 @@ The table lists every transition with its conditions, including these, which the
      - Settled by the next complete ingest
    * - conflict
      - Remote change ingested
-     - Conflict with the new remote side; synced if it now equals your edit, pending again if the server is back at the synced content, or removed if both sides deleted the object
+     - Conflict with the new server's copy; synced if it now equals your edit, pending again if the server is back at the synced copy, or removed if both sides deleted the object
    * - conflict
      - The object holding the logical key an edit was refused for stops holding it, removed or changed on the server or by your own push
      - Pending again, unless another object in the scope still holds the key
@@ -260,9 +260,9 @@ The table lists every transition with its conditions, including these, which the
 Why conflicts are reported, not resolved
 ========================================
 
-When a :term:`conflict` comes up, calendaring-sync reports it and leaves the decision to your code. Only your application knows whether the local edit, the server's version, or a merge is right. A removal on the server against a local edit is always a conflict, never resolved automatically, because that would delete your edit without asking.
+When a :term:`conflict` comes up, calendaring-sync reports it and leaves the decision to your code. Only your application knows whether the local edit, the server's copy, or a merge is right. A removal on the server against a local edit is always a conflict, never resolved automatically, because that would delete your edit without asking.
 
-The core classifies each change by looking only at the :term:`synced content`, the server's change, and your edit. The result is one of six classes:
+The core classifies each change by looking only at the :term:`synced copy`, the server's change, and your edit. The result is one of six classes:
 
 .. list-table::
    :header-rows: 1
@@ -283,9 +283,9 @@ The core classifies each change by looking only at the :term:`synced content`, t
    * - ``CONFLICT``
      - Both sides acted; a conflict record is stored, of one :class:`~calendaring_sync.objects.conflict.ConflictKind`
 
-Later remote changes update the record's remote side. The conflict resolves itself when the server reaches your edit, goes back to the synced content, or removes an object you deleted too. An edit refused for its logical key goes back to pending once the object holding that key no longer holds it.
+Later remote changes update the record's :term:`server's copy`. The conflict resolves itself when the server reaches your edit, goes back to the synced copy, or removes an object you deleted too. An edit refused for its logical key goes back to pending once the object holding that key no longer holds it.
 
-The core stores conflicts instead of raising them: an exception for one object would abort every other change in the sync, and a :term:`resolver` called during a sync would run again on every retry. So you resolve them when you choose, outside any sync, with a :data:`~calendaring_sync.objects.resolution.Resolution`. Anything but the server's side becomes a pending edit over the server's current copy, so its push is checked against the server's current version. :meth:`~calendaring_sync.replica.Replica.resolve` refuses a record whose conflict changed since you read it, so a decision is never applied to news you haven't seen.
+The core stores conflicts instead of raising them: an exception for one object would abort every other change in the sync, and a :term:`resolver` called during a sync would run again on every retry. So you resolve them when you choose, outside any sync, each with a :data:`~calendaring_sync.objects.resolution.Resolution`: one with :meth:`~calendaring_sync.replica.Replica.resolve`, or every open one through your :term:`resolver` with :meth:`~calendaring_sync.replica.Replica.resolve_conflicts`. Anything but the server's copy becomes a pending edit over the server's current copy, so its push is checked against the server's current version. :meth:`~calendaring_sync.replica.Replica.resolve` refuses a record whose conflict changed since you read it, so a decision is never applied to news you haven't seen.
 
 .. _design-feed:
 
@@ -308,4 +308,4 @@ If a calendar or account disappears from the server, the adapter raises an error
 Why storage is synchronous
 ==========================
 
-Storage is a typed interface, :class:`~calendaring_sync.store.base.StateStore`, with an in-memory implementation for tests and a SQLite one for real use. One :term:`store` holds many scopes. A key-value interface couldn't apply a change set and advance a token in one transaction, and an asynchronous store would push synchronous callers through an event loop. Adapters come in synchronous and asynchronous variants that share their logic. The asynchronous ones run each store call in a worker thread, since ingesting a large full listing is one long transaction and would otherwise block the event loop. Reads from other threads don't wait for that transaction, at most for its commit: until then they see what the store held before it began, so your application can show its records while a sync runs.
+A :term:`store` is a :class:`~calendaring_sync.store.base.StateStore` over a :term:`storage`, a typed interface that a SQLite file, memory or storage of your own implements. The store holds every rule about transactions, threads, checks and errors once, so a storage only reads and writes. One store holds many scopes. A key-value interface couldn't apply a change set and advance a token in one transaction, and an asynchronous store would push synchronous callers through an event loop. Adapters come in synchronous and asynchronous variants that share their logic. The asynchronous ones run each store call in a worker thread, since ingesting a large full listing is one long transaction and would otherwise block the event loop. Reads from other threads don't wait for that transaction, at most for its commit: until then they see what the store held before it began, so your application can show its records while a sync runs.

docs/how-to/conflicts.rst

@@ -28,4 +28,4 @@ Resolve every conflict with a function
    :start-after: # start: merge
    :end-before: # end: merge
 
-:meth:`~calendaring_sync.replica.Replica.resolve_conflicts` calls your function with each open conflict, and ``None`` leaves one open. A :class:`~calendaring_sync.objects.resolution.Merge` needs the fields the adapter reads from content, which :meth:`~calendaring_sync.adapters.base.Adapter.edit_fields` returns. :attr:`Record.conflict <calendaring_sync.objects.record.Record.conflict>` holds the server's side, and :class:`~calendaring_sync.objects.conflict.ConflictKind` says what each side did.
+:meth:`~calendaring_sync.replica.Replica.resolve_conflicts` calls your function with each open conflict, and ``None`` leaves one open. It returns a :class:`~calendaring_sync.objects.resolution.ResolvedConflicts` saying which conflicts it resolved, left open, or found changed while your function ran. A :class:`~calendaring_sync.objects.resolution.Merge` needs the fields the adapter reads from content, which :meth:`~calendaring_sync.adapters.base.Adapter.edit_fields` returns. :attr:`Record.conflict <calendaring_sync.objects.record.Record.conflict>` holds the :term:`server's copy`, and :class:`~calendaring_sync.objects.conflict.ConflictKind` says what each side did.

docs/how-to/mirror.rst

@@ -17,6 +17,6 @@ Your application often keeps its own table of events, in the shape its screens a
 
 The table and the :term:`cursor` live in one database, and each page of the feed updates both in one transaction. So a crash leaves the table matching the cursor, and the next run continues from there: no entry is handled twice, and none is skipped.
 
-``apply`` reads the record each entry is about, since the record holds its latest synced content, however many entries there were since the last run. A record that's removed, or that was :term:`purged <purge>` meanwhile, loses its row, and a dropped scope loses every row. Your own edits write no feed entries until the server has them, so the table holds each record's synced content, the copy both sides last agreed on. Read :attr:`Record.edit <calendaring_sync.objects.record.Record.edit>` as well where your application shows edits that aren't sent yet.
+``apply`` reads the record each entry is about, since the record holds its latest synced copy, however many entries there were since the last run. A record that's removed, or that was :term:`purged <purge>` meanwhile, loses its row, and a dropped scope loses every row. Your own edits write no feed entries until the server has them, so the table holds each record's :term:`synced copy`. Read :attr:`Record.edit <calendaring_sync.objects.record.Record.edit>` as well where your application shows edits that aren't sent yet.
 
 When the feed was trimmed past your cursor, :meth:`~calendaring_sync.replica.Replica.read_feed` raises :class:`~calendaring_sync.error.TrimmedFeedError`: rebuild the table from :meth:`~calendaring_sync.replica.Replica.records`, as :doc:`change-feed` shows.

docs/how-to/store.rst

@@ -49,4 +49,13 @@ Keep sync state in memory
 Keep sync state in your own storage
 ===================================
 
-To keep sync state where your application keeps its other data, subclass :class:`~calendaring_sync.store.memory.MemoryStore` and override how it loads and saves its data, as its description shows with :mod:`pickle`. For a database that should look up records itself, implement :class:`~calendaring_sync.store.base.StateStore`: its description lists the methods a store writes, and the base class adds every check, transaction rule and error a store needs.
+To keep sync state in a file of your own, give :class:`~calendaring_sync.store.memory.MemoryStore` a :class:`~calendaring_sync.store.memory.MemoryKeeper`. The store loads its data from the keeper once, when it opens, and hands it to the keeper each time a write commits:
+
+.. literalinclude:: ../examples/keeper.py
+   :language: python
+   :start-after: # start: keeper
+   :end-before: # end: keeper
+
+Only load a file your application wrote: unpickling a file runs code from it. Every lookup scans the records in memory, so for a large store use SQLite.
+
+For a database that looks records up itself, implement :class:`~calendaring_sync.store.base.Storage` and :class:`~calendaring_sync.store.base.StorageTransaction`, and pass the storage to :class:`~calendaring_sync.store.base.StateStore`, which does everything but the reading and writing.

docs/reference/glossary.rst

@@ -32,7 +32,7 @@ These are the sync terms calendaring-sync's pages use. :doc:`../explanation/desi
       An object where your :term:`pending edit` and the server disagree: both sides changed it, one side deleted it while the other edited it, or the server refused your create or change because another object holds its :term:`logical key`. calendaring-sync reports it and leaves the decision to you.
 
    conflict record
-      What the core stores for one :term:`conflict`: the :term:`synced content`, your edit, the server's side, and the kind of conflict. It stays open until you resolve it or it resolves itself.
+      What the core stores for one :term:`conflict`: the :term:`synced copy`, your :term:`local edit`, the :term:`server's copy`, and the kind of conflict. It stays open until you resolve it or it resolves itself.
 
    container
       A calendar, or another grouping, that an object belongs to. One object can belong to more than one.
@@ -105,15 +105,21 @@ These are the sync terms calendaring-sync's pages use. :doc:`../explanation/desi
    session
       One sync of one :term:`scope`: it :term:`ingests <ingest>` every page first, then :term:`pushes <push>` your pending edits.
 
+   server's copy
+      The server's newer copy of an object you edited, which conflicts with your :term:`local edit`. A record in conflict holds it in ``Record.conflict``, in the fields whose names start with ``remote_``.
+
+   storage
+      What a :term:`store` keeps its data in, such as a SQLite file, behind the :class:`~calendaring_sync.store.base.Storage` interface. Write one to keep sync state somewhere new.
+
    store
       Where the core keeps each scope's sync token, its records, your pending edits, open conflicts, and the :term:`change feed`.
 
    sync token
    token
       A marker the server gives out that stands for one point in a scope's history. The adapter sends it back to ask what changed since then. The core only stores and compares it.
 
-   synced content
-      An object's content as both sides last agreed on it, at the end of the last sync.
+   synced copy
+      An object as both sides last agreed on it, at the end of the last sync. A :term:`record` holds it in its fields from ``version`` to ``logical_key``.
 
    truncated page
    page

docs/reference/guarantees.rst

@@ -41,7 +41,7 @@ Guarantees
      - :ref:`design-feed`
    * - A scope the server no longer has is never emptied for you. Dropping it returns the edits and conflicts it discards.
      - :ref:`design-dropped-scope`
-   * - In the stores that ship, a read in another thread doesn't wait for a sync's write, at most for its commit, and sees the store as it was before the write began. A store of your own does the same where it sets ``_reads_beside_writes``.
+   * - In the stores that ship, a read in another thread doesn't wait for a sync's write, at most for its commit, and sees the store as it was before the write began. A store over a storage of your own does the same where the storage sets :attr:`~calendaring_sync.store.base.Storage.reads_beside_writes`.
      - :ref:`design-storage`
    * - Every store checks arguments and raises errors the same way, since the checks live in the base class.
      - :class:`~calendaring_sync.store.base.StateStore`

docs/reference/store.rst

@@ -5,17 +5,19 @@
 ======================================================
 
 .. meta::
-   :description: StateStore, the storage interface, and its implementations: SQLiteStore for real use and MemoryStore for tests.
+   :description: StateStore, the storage interface it keeps its data in, and the stores that ship: SQLiteStore, and MemoryStore with an optional keeper.
 
 .. automodule:: calendaring_sync.store
    :no-members:
 
 .. automodule:: calendaring_sync.store.base
    :members:
-   :private-members: _begin, _commit, _rollback, _scope_records_in, _store_error, _writing, _reads_beside_writes
 
 .. automodule:: calendaring_sync.store.memory
    :members:
+   :exclude-members: MemoryData
+
+.. autoclass:: calendaring_sync.store.memory.MemoryData()
 
 .. automodule:: calendaring_sync.store.sqlite
    :members: