Trim README to admin altitude, reframe as export-first
Drop the backup-and-destroy headline framing (the script only ever exports; removal is the caller's opt-in) and cut the ELI5 hand-holding aimed at someone who isn't the admin of this box. Collapse exit codes to a table, demote removal to a short exit-code-gated snippet. No code change.
This commit is contained in:
parent
4069f26769
commit
8b00e239d2
1 changed files with 46 additions and 92 deletions
138
README.md
138
README.md
|
|
@ -1,32 +1,21 @@
|
|||
# mm3-hk-export
|
||||
|
||||
Export a private **HyperKitty** (Mailman 3) list archive to an `mbox` file from
|
||||
the command line: the scriptable equivalent of clicking the web UI's
|
||||
**Download** button, so it fits a backup-and-destroy loop without manual
|
||||
browser steps.
|
||||
Export a private **HyperKitty** (Mailman 3) list archive to mbox from the command
|
||||
line — the CLI equivalent of the web UI's Download button.
|
||||
|
||||
## Why this exists
|
||||
## Why
|
||||
|
||||
HyperKitty has **no `hyperkitty_export` management command**; the only built-in
|
||||
export is the web "Download" view (gated by the `HYPERKITTY_MBOX_EXPORT`
|
||||
setting). This script reproduces that view's exact output by calling the same
|
||||
`Email.as_bytes()` code path on each archived message, driven through Debian's
|
||||
`mailman-web` wrapper: no web login, no direct database reconstruction.
|
||||
|
||||
The archives live in the **mailman-web / HyperKitty** database, which is
|
||||
separate from Mailman **core**. That is why the export must go through
|
||||
`mailman-web` (not `mailman` / `mailman-wrapper`, whose `shell` only sees core).
|
||||
HyperKitty ships no `hyperkitty_export` command; the only built-in export is the
|
||||
web Download view. This drives the same `Email.as_bytes()` path through the
|
||||
`mailman-web` wrapper, so the output is byte-for-byte identical — no web login, no
|
||||
DB reconstruction. It has to be `mailman-web`, not `mailman-wrapper`: the archive
|
||||
lives in the HyperKitty DB, not core.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Debian-style Mailman 3 with the `mailman-web` wrapper on `PATH`
|
||||
(Django site at `/usr/share/mailman3-web/`, HyperKitty installed).
|
||||
- Run as **root**: `mailman-web` drops privileges to the web user to read the
|
||||
archive, and the script (as root) moves the finished file into place.
|
||||
- Optional, only to enrich the "not archived" notice: the `mailman-wrapper`
|
||||
core CLI (for a best-effort `archive_policy` lookup via `mailman shell`,
|
||||
reading core's DB directly, no REST, no credentials). Missing it just falls
|
||||
back to a generic message.
|
||||
Run as **root** — `mailman-web` drops to the web user to read the archive, then
|
||||
the script publishes the result as root. Uses `mailman-wrapper` to report a
|
||||
list's `archive_policy` when there's nothing to export.
|
||||
|
||||
## Usage
|
||||
|
||||
|
|
@ -34,90 +23,55 @@ separate from Mailman **core**. That is why the export must go through
|
|||
mm3-hk-export [-z|--gzip] <list@domain> [output]
|
||||
```
|
||||
|
||||
- `output` defaults to `./<localpart>.mbox` (`./<localpart>.mbox.gz` with `-z`).
|
||||
- `-z` / `--gzip` gzip-compresses the stream, matching the web download's
|
||||
`.mbox.gz` byte-for-byte (deterministic header, `mtime=0`); `.gz` is appended
|
||||
to the output name if absent.
|
||||
|
||||
Examples:
|
||||
`output` defaults to `./<localpart>.mbox` (`.mbox.gz` with `-z`). `-z` gzips the
|
||||
stream to match the web `.mbox.gz` byte-for-byte (deterministic header, `mtime=0`).
|
||||
|
||||
```sh
|
||||
mm3-hk-export announce@lists.example.com /root/mm3/announce.mbox
|
||||
mm3-hk-export --gzip announce@lists.example.com /root/mm3/announce.mbox.gz
|
||||
```
|
||||
|
||||
## Outcomes and exit codes
|
||||
## Exit codes
|
||||
|
||||
The script never writes a partial or empty file. The **exit code** tells a
|
||||
caller exactly what happened, most importantly whether a backup was produced:
|
||||
Never writes a partial or empty file. The exit code reports whether a backup was
|
||||
produced:
|
||||
|
||||
| Situation | HyperKitty state | Result | Exit |
|
||||
|---|---|---|:--:|
|
||||
| Archive has messages | row with N emails | writes the mbox, prints `wrote N messages to …` | `0` |
|
||||
| Archived but empty | row, 0 emails | no file, prints an "empty archive" notice | `3` |
|
||||
| No archive at all | no `MailingList` row | no file; consults core `archive_policy` to say whether the list is `never`-archived or simply has nothing archived yet | `3` |
|
||||
| Usage / bad address | n/a | usage message | `2` |
|
||||
| Genuine error (e.g. `mailman-web` failing) | n/a | underlying output on stderr | `1` |
|
||||
| Exit | Meaning |
|
||||
|:--:|---|
|
||||
| `0` | archive written (`wrote N messages to …`) |
|
||||
| `3` | nothing to archive — empty list, or no archive at all (reports `archive_policy`: `never` vs nothing-yet) |
|
||||
| `2` | usage / bad address |
|
||||
| `1` | error (e.g. `mailman-web` failed) |
|
||||
|
||||
Exit **`3`** ("nothing to archive") is deliberately distinct from `0` ("archive
|
||||
written"). A genuinely empty or never-archived list is a normal, expected
|
||||
outcome, but it is not a successful backup, so a backup-and-destroy caller
|
||||
can refuse to delete a list whose archive was never captured (see below).
|
||||
`3` is distinct from `0` so a caller can tell "backed up" from "nothing to back
|
||||
up" and refuse to destroy an uncaptured list.
|
||||
|
||||
## In a backup-and-destroy loop
|
||||
## Gating a removal on the backup
|
||||
|
||||
The script only exports. To remove a list after a verified backup, gate it on the
|
||||
exit code yourself:
|
||||
|
||||
```sh
|
||||
mkdir -p /root/mm3
|
||||
for list in list1 list2; do
|
||||
addr="$list@domain.tld"
|
||||
|
||||
mm3-hk-export "$addr" "/root/mm3/$list.mbox"
|
||||
case $? in
|
||||
0) ;; # archive written; safe to proceed
|
||||
3) # nothing to archive (empty / no archive):
|
||||
# no backup file exists, so by default do
|
||||
# not destroy. Override only if you mean it.
|
||||
if [ "${ALLOW_NO_ARCHIVE:-0}" != 1 ]; then
|
||||
echo "skip: $addr has no archive to back up; not removing" >&2
|
||||
continue
|
||||
fi ;;
|
||||
*) echo "skip: $addr export failed; not removing" >&2; continue ;;
|
||||
esac
|
||||
|
||||
mailman-wrapper members "$addr" > "/root/mm3/$list.members" \
|
||||
|| { echo "skip: $addr member dump failed; not removing" >&2; continue; }
|
||||
|
||||
mailman-wrapper remove "$addr"
|
||||
done
|
||||
mm3-hk-export "$addr" "/root/mm3/$list.mbox" \
|
||||
|| { echo "no backup; keeping $addr" >&2; exit 1; }
|
||||
mailman-wrapper members "$addr" > "/root/mm3/$list.members"
|
||||
mailman-wrapper remove "$addr"
|
||||
```
|
||||
|
||||
The loop destroys a list only when its archive was actually written
|
||||
(exit `0`) and its member roster was dumped. A list with nothing to archive
|
||||
(exit `3`) is left intact unless you opt in with `ALLOW_NO_ARCHIVE=1`, so a
|
||||
transient or unexpected "no archive" can never silently authorise a delete.
|
||||
|
||||
Export before `remove`, and never `remove` on a failed backup. (HyperKitty's
|
||||
archive lives in a separate database and is not deleted by core's `remove`, but
|
||||
a failed export must still halt the destroy step.)
|
||||
Export before `remove`, never `remove` on a non-zero exit. (Core's `remove`
|
||||
leaves the HyperKitty archive untouched anyway — separate DB — but a failed
|
||||
export must still halt the destroy.)
|
||||
|
||||
## Caveats
|
||||
|
||||
- **Reconstruction, not the raw original.** HyperKitty rebuilds each message
|
||||
from parsed fields: `From`, `To`, `Subject`, `Date`, `Message-ID`,
|
||||
`In-Reply-To`, body, and attachments. Headers like `Cc`, `References`,
|
||||
`List-*`, and custom `X-*` are not preserved. This is inherent to
|
||||
HyperKitty; the manual web download has the identical limitation.
|
||||
- **Memory.** Messages are streamed with a DB iterator and written
|
||||
incrementally, avoiding the web export's known blow-up on large lists.
|
||||
- **Re-import.** The output is valid mbox (proper `From ` separators,
|
||||
`From`-mangled bodies) and can be re-imported with `hyperkitty_import`.
|
||||
- **Handoff integrity.** The archive is written by the privilege-dropped web
|
||||
user into `/var/tmp`, then published by root. Root opens that file with
|
||||
`O_NOFOLLOW`, copies it while computing a SHA-256, and refuses to publish
|
||||
unless the digest matches the one the exporter computed over the bytes it
|
||||
wrote, so a same-uid process that swaps, truncates, or corrupts the temp
|
||||
file between export and publish is caught (the run fails, nothing is
|
||||
published, and the destroy loop will not remove the list). A fully
|
||||
compromised mailman-web process is out of scope: it builds the archive from
|
||||
a database it already controls, so no file-level check downstream can vouch
|
||||
for the content it produced.
|
||||
- **Reconstruction, not the original bytes.** HyperKitty rebuilds each message
|
||||
from parsed fields (`From`, `To`, `Subject`, `Date`, `Message-ID`,
|
||||
`In-Reply-To`, body, attachments); `Cc`, `References`, `List-*`, and `X-*` are
|
||||
lost. Inherent to HyperKitty — the web download is identical.
|
||||
- **Large lists.** Streamed via a DB iterator, so no memory blow-up like the web
|
||||
export.
|
||||
- **Re-import.** Valid mbox; re-importable with `hyperkitty_import`.
|
||||
- **Handoff.** The web user builds the archive in `/var/tmp`; root verifies it by
|
||||
SHA-256 and publishes atomically, so a swap, truncation, or corruption before
|
||||
publish is caught and nothing is published. A compromised `mailman-web` is out
|
||||
of scope.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue