The .obf format

An .obf archive is made to outlive the app that wrote it. It is not a closed file: it is a folder holding files in open formats and a readable inventory. This chapter describes what is inside, and what you can do with it without Obscura Vault.

Compatibility with Obscura Flow

The .obf format is that of Obscura Flow 2.0.8, unchanged. Vault reads, verifies and restores archives written by Flow (tar, tar.gz, zip, zstd, encrypted or not; an incremental revision is read, verified and restored from its chain, and only its files can be extracted, see Restore an incremental revision), and writes its own archives in this same format. Archives written by Flow are kept as proof of compatibility, and each version of Vault checks that it can still read them, verify their signature, decrypt them and restore them identically.

To verify and restore Flow’s archives, Vault needs Flow’s keys: see Keys and security.

What an archive contains

An archive is a folder named after the project and the revision:

Mariage Dupont-v2/
├── Mariage Dupont-v2.tar.zst    the content: the whole project folder, compressed (and encrypted if requested)
├── manifest.json                the inventory, readable
├── signature.ed25519            the signature of the inventory
├── inventory.json               the inventory annex: the project's complete state (Vault only)
└── inventory.ed25519            the signature of the annex (Vault only)

The first three files are Obscura Flow’s format. The last two are added by Vault to every archive it creates: see The inventory annex.

The content

The content file carries the extension of its format:

Format chosen	Content file
zstd	`….tar.zst`
tar.gz	`….tar.gz`
tar (uncompressed)	`….tar`
zip	`….zip`

It holds the whole project folder, subfolders and empty folders included, with their structure: exactly the files the manifest lists, nothing else (no symbolic link, no link target). An incremental revision (written by Vault or by Obscura Flow) holds only the files changed since the revision it builds on: it is never restored alone, only with its whole chain of revisions.

The manifest

manifest.json is the archive’s inventory, in JSON. It contains in particular:

archiveId, projectName, archiveName, revision (and baseRevision for an incremental revision);
engine (the format), createdAt, createdBy (“Obscura Vault” or Obscura Flow);
payloadFile and payloadChecksum: the content file’s name and its SHA-256 fingerprint, as it is on disk (encrypted, if it is);
files: the list of all files, with their path, size and SHA-256 fingerprint;
fileCount, totalSourceSize;
signatureAlgorithm (ed25519) and publicKeyFingerprint: the SHA-256 fingerprint of the public key that signed;
isEncrypted and encryptionAlgorithm (aes-256-gcm).

The signature

signature.ed25519 is an Ed25519 signature of the exact bytes of manifest.json. Changing a single character of the manifest makes the signature invalid. Since the manifest contains the content’s fingerprint, the signature also protects the content: checking the signature and then the content’s fingerprint is enough to prove that the archive has not changed since it was created.

The inventory annex

inventory.json describes the project’s complete state at this revision: all its files, with their path, size and SHA-256 fingerprint, sorted by path. For a full archive, it is the same list as the manifest’s. It serves mostly incremental revisions, whose manifest lists only the changed files: the annex then also tells which files did not change, and which were deleted.

It contains inventoryVersion (1), archiveId, revision (and baseRevision for an incremental revision), manifestSha256, fileCount, totalSourceSize, files and createdBy. manifestSha256 is the SHA-256 fingerprint of the bytes of manifest.json: the annex belongs to that package, and cannot be copied onto another one.

inventory.ed25519 is an Ed25519 signature of the exact bytes of inventory.json, made with the same key as the manifest.

When Vault verifies an archive (Verify, integrity audit, restore, offsite send), it also checks the annex when there is one: its signature, its link to the manifest, and its agreement with it (same files, same fingerprints, exact totals). An annex that does not match makes the archive failed: “The inventory annex (inventory.json) does not match this archive: the archive is altered.” An archive without an annex (written by Obscura Flow or by Vault 1.0) is verified as before.

The annex changes nothing in the three files of the format: the manifest and its signature stay identical, byte for byte, to what Obscura Flow would write, and Flow reads these archives ignoring the annex. It is not sent with offsite copies yet.

Encryption

An encrypted archive is encrypted with AES-256-GCM, in 4 MiB blocks each authenticated separately: a truncated or modified block does not open. Only the content is encrypted; the manifest and the signature stay readable, which makes it possible to verify an encrypted archive without its key. To restore it, you need the encryption key.

Without Obscura Vault

An unencrypted archive remains usable with standard tools:

Read the inventory: open manifest.json (or inventory.json) in any text editor.
Check the annex’s link: shasum -a 256 manifest.json must give the value of manifestSha256 in inventory.json.
Check the content: in Terminal, shasum -a 256 "Mariage Dupont-v2.tar.zst" must give the value of payloadChecksum.
Extract the files: the content is an ordinary tar, tar.gz or zip archive. The tar shipped with macOS can read a .tar.zst archive.
Check an extracted file: shasum -a 256 path/to/file must give the fingerprint that files lists for it.

Verifying the signature by hand requires a tool that can verify an Ed25519 signature with the archive’s public key. An encrypted archive requires its encryption key and a program that knows the block layout described above.