Add mm3-hk-export: HyperKitty archive -> mbox exporter

Scriptable equivalent of HyperKitty's web Download button for
backup-and-destroy loops. Runs the export through Debian's mailman-web
wrapper (HyperKitty lives in the web DB, separate from core), reproducing
the web view's Email.as_bytes() output.

- Handles the three archive states without writing partial/empty files:
  populated (writes mbox), row-but-empty, and no row at all.
- For a missing row, best-effort core REST lookup distinguishes
  archive_policy=never from 'nothing archived yet'.
- -z/--gzip matches the web .mbox.gz (deterministic header, mtime=0).
- Privilege-safe: web user writes a 0600 mkstemp file, root moves it.
- Address validated before interpolation into the embedded Python.
This commit is contained in:
veg 2026-05-30 23:30:07 +00:00
commit f7a9714e93
2 changed files with 228 additions and 0 deletions

86
README.md Normal file
View file

@ -0,0 +1,86 @@
# mm3-hk-export
Export a private **HyperKitty** (Mailman 3) list archive to an `mbox` file from
the command line: the scriptable equivalent of clicking the web UI's
**Download** button, so it fits a backup-and-destroy loop without manual
browser steps.
## Why this exists
HyperKitty has **no `hyperkitty_export` management command**; the only built-in
export is the web "Download" view (gated by the `HYPERKITTY_MBOX_EXPORT`
setting). This script reproduces that view's exact output by calling the same
`Email.as_bytes()` code path on each archived message, driven through Debian's
`mailman-web` wrapper: no web login, no direct database reconstruction.
The archives live in the **mailman-web / HyperKitty** database, which is
separate from Mailman **core**. That is why the export must go through
`mailman-web` (not `mailman` / `mailman-wrapper`, whose `shell` only sees core).
## Requirements
- Debian-style Mailman 3 with the `mailman-web` wrapper on `PATH`
(Django site at `/usr/share/mailman3-web/`, HyperKitty installed).
- Run as **root**: `mailman-web` drops privileges to the web user to read the
archive, and the script (as root) moves the finished file into place.
- Optional, only to enrich the "not archived" notice: `curl` and the
`mailman-wrapper` core CLI (for a best-effort `archive_policy` lookup over the
core REST API). Missing either just falls back to a generic message.
## Usage
```
mm3-hk-export [-z|--gzip] <list@domain> [output]
```
- `output` defaults to `./<localpart>.mbox` (`./<localpart>.mbox.gz` with `-z`).
- `-z` / `--gzip` gzip-compresses the stream, matching the web download's
`.mbox.gz` byte-for-byte (deterministic header, `mtime=0`); `.gz` is appended
to the output name if absent.
Examples:
```sh
mm3-hk-export announce@lists.example.com /root/mm3/announce.mbox
mm3-hk-export --gzip announce@lists.example.com /root/mm3/announce.mbox.gz
```
## Outcomes
The script never writes a partial or empty file, and exits `0` on all three:
| Situation | HyperKitty state | Result |
|---|---|---|
| Archive has messages | row with N emails | writes the mbox, prints `wrote N messages to …` |
| Archived but empty | row, 0 emails | no file, prints an "empty archive" notice |
| No archive at all | no `MailingList` row | no file; consults core `archive_policy` to say whether the list is `never`-archived or simply has nothing archived yet |
A genuine error (e.g. `mailman-web` failing) exits non-zero with the underlying
output on stderr.
## In a backup-and-destroy loop
```sh
mkdir -p /root/mm3
for list in list1 list2; do
addr="$list@domain.tld"
mm3-hk-export "$addr" "/root/mm3/$list.mbox" # skips cleanly if no archive
mailman-wrapper members "$addr" > "/root/mm3/$list"
mailman-wrapper remove "$addr"
done
```
Export before `remove`. (HyperKitty's archive lives in a separate database and
is not deleted by core's `remove`, but exporting first keeps the intent clear.)
## Caveats
- **Reconstruction, not the raw original.** HyperKitty rebuilds each message
from parsed fields: `From`, `To`, `Subject`, `Date`, `Message-ID`,
`In-Reply-To`, body, and attachments. Headers like `Cc`, `References`,
`List-*`, and custom `X-*` are not preserved. This is inherent to
HyperKitty; the manual web download has the identical limitation.
- **Memory.** Messages are streamed with a DB iterator and written
incrementally, avoiding the web export's known blow-up on large lists.
- **Re-import.** The output is valid mbox (proper `From ` separators,
`From`-mangled bodies) and can be re-imported with `hyperkitty_import`.