Receive a Catalog package

Obscura Catalog describes the images; Vault preserves them. To hand a library over to Vault, Catalog writes a Catalog package. Vault verifies it, finds the originals it names, archives it all and can give it back. The two applications share neither database nor code: the package is a plain folder, described by a written contract (docs/technique/contrat-paquet-vault.md).

The Catalog Packages screen is in the Preserve group, under Archives (⌘0).

What a Catalog package is

A package is a folder Paquet Catalog – <collection> – <date>, sometimes carried in a ZIP archive of the same name. It holds:

• package.json: its identity card (identifiers of the package and of the library, collection, date, version of Catalog that wrote it) and the exact list of its files, each with its size and SHA-256 fingerprint;
• backup/: the full backup of the Catalog library, to restore it in Catalog;
• export/: the full export in open formats, to read it without Catalog;
• originals.csv: the list of the originals (the image files), each named by its fingerprint, with where Catalog saw it;
• sometimes originals/: copies of some originals, when Catalog put them there.

Vault keeps the package as one unit: it does not read the catalogue (works, records, publications) and never modifies the package.

Import a package

Importing a package needs a licence, like creating an archive.

1. In Catalog Packages, click Import Catalog Package… (or, in the File menu, Import Catalog Package…).
2. Choose the package’s folder (the one holding package.json) or its ZIP archive. A folder without package.json is flagged at once: it is not a package.
3. The Import a Catalog Package sheet recalls what will happen. Tick Also encrypt the originals (AES-256-GCM) if you wish (the package itself is always encrypted, see below).
4. Click Import.

Vault works in three stages, shown one after the other above the list, with their progress:

1. Verification: for a ZIP archive, Vault checks it before extracting anything; then it verifies that the package holds exactly the files package.json lists, each of its size and fingerprint, and that originals.csv is well formed.
2. Matching: each original is looked up by its fingerprint: first in Vault’s archives, then in the package, then where Catalog saw it (on this Mac or on a plugged-in disk).
3. Archiving: the package is archived as one encrypted archive, then each original not yet in Vault is read, checked against its fingerprint while it is read, and archived. Vault only reads the originals where they are: it never writes there.

The import runs while you look at other screens; the toolbar shows it, like an archiving.

Stop and resume

Stop works at every stage (after two minutes, Vault asks for confirmation):

• during the verification or the matching, nothing is archived;
• during the archiving, what is already archived stays archived. Import the same package again (the same folder or ZIP archive): Vault recognises it, says so (“This package was already received, but the archiving of its originals had stopped: it resumes where it stopped…”) and does not read again what is already archived.

A package already fully archived is not archived again: only its originals not yet in Vault are. That is how to complete an import after plugging in the disk of missing originals.

Refused package

If the package does not conform, Vault refuses it entirely, archiving nothing, and the banner gives the reason and the file in question: “Catalog package refused: “export/…” is missing.”, “… does not have the expected fingerprint”, an unreadable ZIP archive or one with an item that would land outside the extraction folder… The package was changed or damaged since Catalog wrote it: write a new one from Catalog.

A package written by a more recent version of Catalog, in a format this version of Vault cannot read, is refused with Package Too Recent for This Version: choose Obscura Vault › Check for Updates…, then import it again.

The import report

At the end, the report opens; Show Report opens it again while the banner is shown. It counts the package’s originals in four classes, with their size:

Each missing original comes with its reason: its disk is not plugged in, not found at the location given, the package does not say where it is, the file found does not have the expected size or fingerprint (it is not the same), unreadable, or its archive was not found intact at its last check. A missing original never makes the package refused.

Export Missing List… writes a CSV file: fingerprint, size, name and reason of each missing original, with no path (paths reveal how your folders are organised). To complete: plug in the disk, or have Catalog embed the original in the package, then import the package again.

Sensitivity and encryption

Every package is sensitive: backup/ holds the whole Catalog database, confidential records and internal notes included, whatever Confidential says. Vault therefore always archives it encrypted (AES-256-GCM), with this Mac’s archive encryption key, and writes nothing of its content in its logs. Even what stays readable next to an encrypted archive (its manifest) names neither the collection, nor your folders, nor your files: only identifiers, sizes and fingerprints.

The originals follow your choice (Also encrypt the originals (AES-256-GCM)), like a project archive: they are the same images, stored by fingerprint and without a name, and an image already archived from a project is not archived a second time.

Encryption key: if this Mac had no archive encryption key yet, it is created with the first package, and the import sheet says so. The Catalog Packages screen then reminds you to export it (Back Up Keys…), like the Archives screen: without it, the package cannot be given back on any other Mac. See Keys and security.

If the keychain allows neither reading nor creating the key, the package is not archived at all: never in clear.

The list of packages and their record

The list groups the packages by Catalog library (the newest one’s collection, the library’s shortened identifier, the number of packages), newest first. Each new package of a library is added; none is removed (see “Retention” below). A row marked Originals to resume is a package whose originals archiving stopped: import it again.

Record opens the package’s record:

• Collection, Written by Catalog on, Written by (the application and its version), Received on;
• Catalog library and Package identifier, shortened (3F2504E0…3301): hovering shows the full identifier, which can be selected and copied;
• Confidential: “Yes” when Catalog’s export includes confidential records, “No”, or “Unknown” for a package received before Vault kept that value; it changes nothing to how the package is protected;
• Originals, Originals archiving (completed, or to resume) and Package archive (the name of its archive in the archives folder);
• the earlier packages of this library, each with its own record.

Give back

Giving back needs no licence. Nothing is ever written over an existing file: Vault creates a new folder in the folder chosen (-2, -3… if the name is taken); the archives folder is refused. A project folder is flagged before anything is written, as for a restore (Restore Anyway). The progress is shown with Stop; at the end, Show in Finder opens the folder given back.

• Give Back Package… (on the row, Give Back menu, or in the record): the package is given back byte for byte, as Catalog wrote it, embedded copies included, in a folder Paquet Catalog – <collection> – <date>. It is verified again, as at import, before it appears: otherwise nothing is given back. To get the library back, open it in Catalog from its backup folder.
• Give Back Originals…: every original of the package that Vault archived, each stored under its fingerprint (ab/abcdef…, as in a package), each verified. An original in no archive (it was missing at import) or whose archive is not available is reported, without stopping the others.
• Give Back Originals by Fingerprint… (⋯ icon at the top of the screen): paste SHA-256 fingerprints, or Read File… to take them from a file (for example the exported list of missing originals, or a list written by Catalog), then Give Back…. Anything that is not a fingerprint is ignored.

After Stop, a package is never given back halfway; the originals already given back stay, each whole and verified.

Storage medium or offsite: giving back reads the archives folder. An archive that is now only on a storage medium or offsite must first be put back there, under its name: copy its folder from the medium, or download it (Offsite › Download…), then put it in the archives folder. Otherwise Vault says “The archive … that holds the package is not in the Archives folder”, or counts the originals concerned as unavailable.

Retention: why some deletions are refused

Vault keeps every package it received, with no setting, and never deletes one by itself. It therefore refuses any deletion that would prevent giving a package back byte for byte, or lose an original the package designates as archived: the package’s archive, the archive of the originals it took, or a project’s archive that alone holds one of its originals intact. The message names the revision, the place and the package concerned (its collection and its date). A copy on a storage medium or offsite can be deleted as long as the archive stays intact in the archives folder, which giving back reads, or somewhere else; if it is the last intact copy, put it back in the archives folder first. A project archive whose originals are also held intact by another revision stays free.