From 8b00e239d24a3c14f9c109d50a58703c97d57953 Mon Sep 17 00:00:00 2001 From: veg Date: Sun, 31 May 2026 14:23:15 +0000 Subject: [PATCH] Trim README to admin altitude, reframe as export-first Drop the backup-and-destroy headline framing (the script only ever exports; removal is the caller's opt-in) and cut the ELI5 hand-holding aimed at someone who isn't the admin of this box. Collapse exit codes to a table, demote removal to a short exit-code-gated snippet. No code change. --- README.md | 138 ++++++++++++++++++------------------------------------ 1 file changed, 46 insertions(+), 92 deletions(-) diff --git a/README.md b/README.md index b723f91..57ec992 100644 --- a/README.md +++ b/README.md @@ -1,32 +1,21 @@ # mm3-hk-export -Export a private **HyperKitty** (Mailman 3) list archive to an `mbox` file from -the command line: the scriptable equivalent of clicking the web UI's -**Download** button, so it fits a backup-and-destroy loop without manual -browser steps. +Export a private **HyperKitty** (Mailman 3) list archive to mbox from the command +line — the CLI equivalent of the web UI's Download button. -## Why this exists +## Why -HyperKitty has **no `hyperkitty_export` management command**; the only built-in -export is the web "Download" view (gated by the `HYPERKITTY_MBOX_EXPORT` -setting). This script reproduces that view's exact output by calling the same -`Email.as_bytes()` code path on each archived message, driven through Debian's -`mailman-web` wrapper: no web login, no direct database reconstruction. - -The archives live in the **mailman-web / HyperKitty** database, which is -separate from Mailman **core**. That is why the export must go through -`mailman-web` (not `mailman` / `mailman-wrapper`, whose `shell` only sees core). +HyperKitty ships no `hyperkitty_export` command; the only built-in export is the +web Download view. This drives the same `Email.as_bytes()` path through the +`mailman-web` wrapper, so the output is byte-for-byte identical — no web login, no +DB reconstruction. It has to be `mailman-web`, not `mailman-wrapper`: the archive +lives in the HyperKitty DB, not core. ## Requirements -- Debian-style Mailman 3 with the `mailman-web` wrapper on `PATH` - (Django site at `/usr/share/mailman3-web/`, HyperKitty installed). -- Run as **root**: `mailman-web` drops privileges to the web user to read the - archive, and the script (as root) moves the finished file into place. -- Optional, only to enrich the "not archived" notice: the `mailman-wrapper` - core CLI (for a best-effort `archive_policy` lookup via `mailman shell`, - reading core's DB directly, no REST, no credentials). Missing it just falls - back to a generic message. +Run as **root** — `mailman-web` drops to the web user to read the archive, then +the script publishes the result as root. Uses `mailman-wrapper` to report a +list's `archive_policy` when there's nothing to export. ## Usage @@ -34,90 +23,55 @@ separate from Mailman **core**. That is why the export must go through mm3-hk-export [-z|--gzip] [output] ``` -- `output` defaults to `./.mbox` (`./.mbox.gz` with `-z`). -- `-z` / `--gzip` gzip-compresses the stream, matching the web download's - `.mbox.gz` byte-for-byte (deterministic header, `mtime=0`); `.gz` is appended - to the output name if absent. - -Examples: +`output` defaults to `./.mbox` (`.mbox.gz` with `-z`). `-z` gzips the +stream to match the web `.mbox.gz` byte-for-byte (deterministic header, `mtime=0`). ```sh mm3-hk-export announce@lists.example.com /root/mm3/announce.mbox mm3-hk-export --gzip announce@lists.example.com /root/mm3/announce.mbox.gz ``` -## Outcomes and exit codes +## Exit codes -The script never writes a partial or empty file. The **exit code** tells a -caller exactly what happened, most importantly whether a backup was produced: +Never writes a partial or empty file. The exit code reports whether a backup was +produced: -| Situation | HyperKitty state | Result | Exit | -|---|---|---|:--:| -| Archive has messages | row with N emails | writes the mbox, prints `wrote N messages to …` | `0` | -| Archived but empty | row, 0 emails | no file, prints an "empty archive" notice | `3` | -| No archive at all | no `MailingList` row | no file; consults core `archive_policy` to say whether the list is `never`-archived or simply has nothing archived yet | `3` | -| Usage / bad address | n/a | usage message | `2` | -| Genuine error (e.g. `mailman-web` failing) | n/a | underlying output on stderr | `1` | +| Exit | Meaning | +|:--:|---| +| `0` | archive written (`wrote N messages to …`) | +| `3` | nothing to archive — empty list, or no archive at all (reports `archive_policy`: `never` vs nothing-yet) | +| `2` | usage / bad address | +| `1` | error (e.g. `mailman-web` failed) | -Exit **`3`** ("nothing to archive") is deliberately distinct from `0` ("archive -written"). A genuinely empty or never-archived list is a normal, expected -outcome, but it is not a successful backup, so a backup-and-destroy caller -can refuse to delete a list whose archive was never captured (see below). +`3` is distinct from `0` so a caller can tell "backed up" from "nothing to back +up" and refuse to destroy an uncaptured list. -## In a backup-and-destroy loop +## Gating a removal on the backup + +The script only exports. To remove a list after a verified backup, gate it on the +exit code yourself: ```sh -mkdir -p /root/mm3 -for list in list1 list2; do - addr="$list@domain.tld" - - mm3-hk-export "$addr" "/root/mm3/$list.mbox" - case $? in - 0) ;; # archive written; safe to proceed - 3) # nothing to archive (empty / no archive): - # no backup file exists, so by default do - # not destroy. Override only if you mean it. - if [ "${ALLOW_NO_ARCHIVE:-0}" != 1 ]; then - echo "skip: $addr has no archive to back up; not removing" >&2 - continue - fi ;; - *) echo "skip: $addr export failed; not removing" >&2; continue ;; - esac - - mailman-wrapper members "$addr" > "/root/mm3/$list.members" \ - || { echo "skip: $addr member dump failed; not removing" >&2; continue; } - - mailman-wrapper remove "$addr" -done +mm3-hk-export "$addr" "/root/mm3/$list.mbox" \ + || { echo "no backup; keeping $addr" >&2; exit 1; } +mailman-wrapper members "$addr" > "/root/mm3/$list.members" +mailman-wrapper remove "$addr" ``` -The loop destroys a list only when its archive was actually written -(exit `0`) and its member roster was dumped. A list with nothing to archive -(exit `3`) is left intact unless you opt in with `ALLOW_NO_ARCHIVE=1`, so a -transient or unexpected "no archive" can never silently authorise a delete. - -Export before `remove`, and never `remove` on a failed backup. (HyperKitty's -archive lives in a separate database and is not deleted by core's `remove`, but -a failed export must still halt the destroy step.) +Export before `remove`, never `remove` on a non-zero exit. (Core's `remove` +leaves the HyperKitty archive untouched anyway — separate DB — but a failed +export must still halt the destroy.) ## Caveats -- **Reconstruction, not the raw original.** HyperKitty rebuilds each message - from parsed fields: `From`, `To`, `Subject`, `Date`, `Message-ID`, - `In-Reply-To`, body, and attachments. Headers like `Cc`, `References`, - `List-*`, and custom `X-*` are not preserved. This is inherent to - HyperKitty; the manual web download has the identical limitation. -- **Memory.** Messages are streamed with a DB iterator and written - incrementally, avoiding the web export's known blow-up on large lists. -- **Re-import.** The output is valid mbox (proper `From ` separators, - `From`-mangled bodies) and can be re-imported with `hyperkitty_import`. -- **Handoff integrity.** The archive is written by the privilege-dropped web - user into `/var/tmp`, then published by root. Root opens that file with - `O_NOFOLLOW`, copies it while computing a SHA-256, and refuses to publish - unless the digest matches the one the exporter computed over the bytes it - wrote, so a same-uid process that swaps, truncates, or corrupts the temp - file between export and publish is caught (the run fails, nothing is - published, and the destroy loop will not remove the list). A fully - compromised mailman-web process is out of scope: it builds the archive from - a database it already controls, so no file-level check downstream can vouch - for the content it produced. +- **Reconstruction, not the original bytes.** HyperKitty rebuilds each message + from parsed fields (`From`, `To`, `Subject`, `Date`, `Message-ID`, + `In-Reply-To`, body, attachments); `Cc`, `References`, `List-*`, and `X-*` are + lost. Inherent to HyperKitty — the web download is identical. +- **Large lists.** Streamed via a DB iterator, so no memory blow-up like the web + export. +- **Re-import.** Valid mbox; re-importable with `hyperkitty_import`. +- **Handoff.** The web user builds the archive in `/var/tmp`; root verifies it by + SHA-256 and publishes atomically, so a swap, truncation, or corruption before + publish is caught and nothing is published. A compromised `mailman-web` is out + of scope.