Skip to content

๐Ÿ—‚๏ธ Curating your archive

Twenty years of photos, videos, mail and documents don't arrive tidy. NoCloud's job is to turn that pile into an ordered archive โ€” and to do it so that nothing is ever lost, everything can be undone, and why something ended up where it did stays on record.

Your originals are never edited

The vocabulary NoCloud uses to change your archive has no "edit file" operation at all. That's not a promise, it's an absence โ€” there is no way to express it:

  • Renaming a file moves it. The bytes are untouched (checked before and after).
  • Dating a photo writes an XMP sidecar next to it, never inside the JPEG.
  • Deduplicating moves the loser to quarantine, it doesn't erase it.
  • Only an explicit, gated delete ever removes bytes, and it's logged.

So the guarantee "your originals stay intact" doesn't depend on every script behaving โ€” it's a property of the vocabulary itself.

Everything is one undo away

Every bulk operation โ€” importing from a phone dump, fixing thousands of dates, filing documents โ€” is recorded as a run in a single append-only journal: what changed, from what to what, and why. One command reverts the whole run, applying each step's inverse in reverse order. The previous state of anything replaced (an old sidecar, say) is kept, addressed by content, so it can always come back.

If a step can't be undone, the whole revert refuses rather than doing half the job.

Operation Its inverse
Copy a file in Remove the copy (your source is untouched)
Move / rename Move it back
Annotate (date, tag, description) Restore the previous sidecar
Record / link a fact Retract it
Delete None โ€” that's why it's gated and logged

Rules you can read, not a black box

Dates, albums and classifications aren't guessed by an opaque model. They come from readable rules over evidence:

  • Extractors read signal without changing anything: the date in a filename, EXIF, the mail a photo arrived in, a visual fingerprint.
  • Rules propose facts from that evidence ("IMG-20200504-WA0007.jpg โ†’ 4 May 2020").
  • Constraints veto impossible ones โ€” a photo can't be dated outside the lifetime of a person who appears in it, a scan's date isn't the photo's date, 1970 is a sentinel and not a date.
  • Resolution settles conflicts by policy: the smallest plausible date usually wins, and a time of 00:00:00 means "approximate".

Nothing is invented: with no reliable signal, a photo goes to undated and the investigation queue, not to a made-up date.

The curation runtime

The rules behind these decisions are ordinary, readable Python. A decorator registers a small, pure function; it does not change a file on its own. The runtime collects proposals, orders them, checks them, then lowers only an accepted decision into a journaled operation.

@extractor observes a signal
      -> @rule and @classifier propose Evidence
      -> @resolution orders competing evidence for each field
      -> @constraint vetoes impossible candidates
      -> accepted fact, or a review-queue item
      -> compile_facts() lowers it through algebra.Run

All decorators receive the same asset, ctx pair. asset is a curation.Asset with its path and filename; ctx is the instance-aware, read-only Context. Context supplies things such as known people, their lifespan, a prebuilt Takeout index, the archive destination, and the Immich connection. A decorator must return Evidence values or an empty list. It must not mutate a file, database, or network service.

from curation import Asset, Context, Evidence

asset = Asset("/archive/Photos/IMG-20240504-WA0001.jpg")
candidate = Evidence(
    field="date",
    value="2024-05-04T00:00:00",
    source="filename",
    confidence="high",
    precision="day",
    meta={"tag": "whatsapp"},
)
Evidence value Meaning
field The fact slot. Built-in lowerings understand date, album, visibility, and category.
value The proposed value. Dates must start with a four-digit ISO year.
source A human-readable origin preserved in the derivation.
confidence high, medium, or low; unrecognised values rank below low.
precision Source precision such as second, day, or year.
meta Optional structured detail: needs_review, review_reason, options, and the producing decorator name.

@extractor: observe, without deciding

An extractor reads raw signal. extract(asset, ctx) runs every registered extractor, but evaluate() deliberately does not call it implicitly: a rule that needs an observation calls its extractor itself. This keeps the dependency visible and prevents a corpus-wide scan from happening while evaluating one file.

import re
import curation

@curation.extractor("receipt_year")
def receipt_year(asset, ctx):
    match = re.search(r"receipt-(20\d{2})", asset.filename, re.I)
    if not match:
        return []
    return [curation.Evidence(
        "date", f"{match.group(1)}-07-01T12:00:00",
        source="receipt filename", confidence="low", precision="year",
    )]

@rule: propose a candidate fact

Use a rule for a conditional fact about one asset. Its output joins the candidate pool; it is never a direct write. The runtime stores the rule name in the fact's derivation.

@curation.rule("receipt_year_date")
def receipt_year_date(asset, ctx):
    return receipt_year(asset, ctx)

Rules may request human confirmation even when their proposal is plausible:

@curation.rule("invoice_category")
def invoice_category(asset, ctx):
    if "invoice" not in asset.filename.lower():
        return []
    return [curation.Evidence(
        "category", "Finance", source="invoice filename", confidence="medium",
        meta={
            "needs_review": True,
            "review_reason": "Filename overlaps with legal documents",
            "options": ["Finance", "Legal", "Other"],
        },
    )]

@classifier: propose membership or category

A classifier has the same return contract as a rule, but communicates that the output is group membership, normally an album or category. It does not have its own conflict mechanism: classifier and rule candidates compete normally when they propose the same field.

@curation.classifier("screenshot_folder")
def screenshot_folder(asset, ctx):
    name = asset.filename.lower()
    if name.endswith(".png") and "screenshot" in name:
        return [curation.Evidence(
            "album", "Screenshots", source="filename pattern", confidence="high"
        )]
    return []

The built-in classifiers are deliberately conservative: obvious_album recognises unambiguous screenshots and memes, visual_triage uses image characteristics, and doc_category scores filename plus extracted text. Medium or low document classifications are queued instead of being moved automatically.

@constraint: veto an impossible fact

Every registered constraint receives every candidate in resolution order. Return True to pass, False to veto, or (False, reason) to veto with an explanation that is retained in the derivation. Constraints do not mutate and do not choose an alternative themselves.

@curation.constraint("no_future_due_dates")
def no_future_due_dates(asset, field, value, ctx):
    if field != "date":
        return True
    if value[:4] > "2026":
        return False, "date is after the configured archive horizon"
    return True

The built-in lifespan constraint rejects a date outside a depicted person's known lifespan. plausible_date rejects the 1970-01-01 epoch sentinel and dates after the current UTC year. If a candidate is vetoed, the runtime tries the next one rather than abandoning the field.

@resolution: order competing evidence

There is one resolver per field. It receives all Evidence candidates for that field and returns them in preferred order. A resolver does not pick an unconditional winner: constraints still get a chance to veto the first item. Fields without a resolver use descending confidence; ties retain registration order.

@curation.resolution("album", name="prefer-high-confidence")
def prefer_high_confidence(evidences):
    scores = {"high": 3, "medium": 2, "low": 1}
    return sorted(evidences, key=lambda ev: scores.get(ev.confidence, 0), reverse=True)

The shipped date resolver is min-date: it sorts ISO dates from earliest to latest, then by confidence. For a photo of somebody born in 1950, the following sequence is intentional:

1940-01-01T00:00:00  -> first by min-date, rejected by lifespan
1970-01-01T00:00:00  -> next, rejected as an epoch sentinel
1984-05-04T00:00:00  -> accepted

The accepted fact records its selected evidence, the discarded alternatives and veto reasons, the resolver name, and each constraint that passed. That is what the product's Why? view reads.

From accepted fact to reversible operation

evaluate(asset, ctx) returns accepted facts. compile_facts() is the only standard lowering path: it sends automatic facts through an algebra.Run, and sends facts marked needs_review to the review queue without changing the archive.

import algebra
import curation

ctx = curation.Context()
asset = curation.Asset("/archive/Inbox/receipt-2026-07.pdf")
facts = curation.evaluate(asset, ctx)

with algebra.Run("curate one asset", tool="example") as run:
    curation.compile_facts(run, asset, facts, ctx)
Accepted field Reversible lowering
date ANNOTATE: write an XMP sidecar through ctx.build_date_xmp().
album LINK in the catalog and, when configured, a compensating Immich album operation.
visibility A compensating Immich visibility operation.
category MOVE to ctx.category_target(asset, value).

The algebra also provides COPY, RECORD, LINK, and a journaled DELETE. A run records the prior state and inverse for every operation. Reverting a run executes those inverses in reverse order; a terminal delete is intentionally not recoverable and must never be mixed into a batch expected to revert.

For the complete implementation contract, including every algebra method, Context capability, review API, and extension checklist, see the curation runtime reference.

The review queue: you have the last word

When a rule's proposal isn't certain enough to apply on its own, it doesn't get applied โ€” it gets queued for you. The review queue is a page that shows each uncertain proposal with its evidence and a preview of what would change, and gives you three verbs, one by one or in bulk:

  • Accept the proposal as-is.
  • Override it with another value โ€” chosen from the alternatives the classifier itself weighed, so reviewing is a real choice, not a free-text field.
  • Reject it โ€” the archive stays untouched.

A pending item is a proposal, not yet a change: nothing is journaled until you resolve it. Once accepted (or overridden), it goes through the same journaled, reversible pipeline as every other operation.

Do you remember?

The archive already knows, precisely, what it doesn't know: a photo dated only to a year, an Immich face never linked to anyone, a person field still holding a bulk-import default instead of a real answer. Do you remember? turns that backlog into a warm ritual instead of an admin chore โ€” hand a tablet to a grandparent and let them answer one huge, simple question at a time: a photo and "about what year was this?", a face and "do you recognize this?", a small fact about someone in the family, each with a handful of big tap targets and an honest "I don't know / Skip". Anyone signed in can answer, not just admins โ€” that's the whole point of the feature. An answer never changes the archive by itself: it becomes a pending item in the same review queue described above, with who answered and when recorded alongside it, so an admin still accepts, overrides or rejects it before anything is written โ€” the same one-word-at-a-time, fully reversible path any other queued fact takes.

Folder drift: when a file disappears outside the app

Deleting a file from your archive by hand (in Finder, not through NoCloud) isn't gated by anything in the app โ€” there's no way to stop it from happening. What NoCloud can do is notice before that deletion quietly reaches your home server and your backups too, and give you a chance to say whether it was on purpose.

  • A background scan on your Mac compares what's actually on disk against what it last knew to be there. A file that's vanished is queued, not deleted from anywhere else yet.
  • Your daily backups (both the local versioned one and the external-disk mirror) check that queue before they run โ€” a pending file is kept out of their own deletion pass, so it survives in your backups until you decide.
  • A banner appears at the top of every page when something is pending, linking straight to the Folder Drift app. There you see each missing file (or, if a whole folder vanished at once, one grouped decision instead of hundreds) and pick:
  • Restore โ€” copies the file back into your archive from the protected backup copy.
  • Delete for good โ€” confirms it; the next backup run finally removes it there too.
  • Nothing happens on its own either way: a pending file just stays protected, indefinitely, until you resolve it.

"Why did this end up here?"

Every accepted fact carries its derivation: the value chosen, the alternatives that were discarded and why, and which constraints it passed. That derivation isn't just recorded, it's readable on demand, journaled alongside the change it explains. That's the difference between a glass box and a black box โ€” an answer you can query is an answer you can correct. Correcting the system means editing a rule, not retraining a model.

Today that record lives in the Provenance app: click View history next to any file โ€” Synced Documents' file browser is one place that link shows up โ€” to open Provenance already searched for that path, or search a path yourself from Provenance directly. It works for any file in your archive, not only synced documents. Each event shows a plain-language summary of what happened and why; expanding Technical evidence on it shows the full derivation underneath โ€” the chosen value, the discarded alternatives, and which constraints it passed. Bringing the same lookup onto a photo's own detail page inside your photo library is the one piece still missing โ€” the derivation is already there and already correct, it just isn't one click away from every photo yet.

Teach it in plain language

A rule doesn't always exist yet for what you know. Coaching turns a sentence โ€” the way you'd explain it to a person โ€” into a new rule, without ever hand-writing code:

  1. In the Rule Coach app, you describe the pattern in your own words ("photos of [a person] before they were born are scans of old prints, not new photos").
  2. The local model on your own machine translates that into a small, readable rule: what it matches, what it proposes, how confident it is. Nothing is applied yet.
  3. You read the rule back โ€” in the same plain block notation, not JSON โ€” right there in the app, before deciding anything.
  4. Only once you approve it does the rule join the catalog and start proposing facts, exactly like every rule shipped with NoCloud. Uncertain proposals still land in your review queue first. Changed your mind before approving? Discard it โ€” nothing was ever applied.

The model never runs your archive through a black box and never writes to it directly โ€” it writes one small, versioned rule that you read before it does anything, and every fact it later proposes still carries its own derivation back to the sentence that taught it.

Is a coached rule the same language as the @rule decorators above? Not literally, and the difference is the whole point. A hand-written rule is arbitrary Python โ€” full expressiveness, but you have to write and trust it. A coached rule can never be that: the model's only output is a fixed, closed vocabulary (which people appear or don't, a filename pattern, a path fragment; a date/album/category to propose) that a small, unchanging piece of NoCloud code turns into a rule function โ€” no Python is generated, and nothing the model outputs is ever executed as code. The block notation you read on screen is a plain-English rendering of that closed vocabulary, not a language of its own. Once built, a coached rule is registered in the exact same catalog as every hand-written one and runs through the exact same call, so from that point on the engine can't tell them apart โ€” the difference is entirely in how each one was made, not in how either one runs.

It shows up where you actually look

A decision is worthless if you open your photo library and see nothing changed, so each kind of fact goes to the sink that surfaces it:

  • Dates, descriptions, tags, GPS โ†’ the XMP sidecar, which Immich (the photo library) re-reads.
  • Album membership and archived/hidden state โ†’ Immich's own API, because those live only in its database. Undoable too, by a compensating action (remove from the album, restore the previous visibility).
  • Edits you make yourself inside Immich โ€” dragging a photo into an album, archiving it โ€” are noticed and recorded in the journal too, so the history of an asset stays complete and a later undo never quietly overwrites something you did by hand.

See also: Key concepts ยท The database ยท Open standards ยท Security & privacy