veg f7a9714e93 Add mm3-hk-export: HyperKitty archive -> mbox exporter

Scriptable equivalent of HyperKitty's web Download button for
backup-and-destroy loops. Runs the export through Debian's mailman-web
wrapper (HyperKitty lives in the web DB, separate from core), reproducing
the web view's Email.as_bytes() output.

- Handles the three archive states without writing partial/empty files:
  populated (writes mbox), row-but-empty, and no row at all.
- For a missing row, best-effort core REST lookup distinguishes
  archive_policy=never from 'nothing archived yet'.
- -z/--gzip matches the web .mbox.gz (deterministic header, mtime=0).
- Privilege-safe: web user writes a 0600 mkstemp file, root moves it.
- Address validated before interpolation into the embedded Python.

2026-05-31 12:24:58 +00:00

3.5 KiB

Raw Blame History

mm3-hk-export

Export a private HyperKitty (Mailman 3) list archive to an mbox file from the command line: the scriptable equivalent of clicking the web UI's Download button, so it fits a backup-and-destroy loop without manual browser steps.

Why this exists

HyperKitty has no hyperkitty_export management command; the only built-in export is the web "Download" view (gated by the HYPERKITTY_MBOX_EXPORT setting). This script reproduces that view's exact output by calling the same Email.as_bytes() code path on each archived message, driven through Debian's mailman-web wrapper: no web login, no direct database reconstruction.

The archives live in the mailman-web / HyperKitty database, which is separate from Mailman core. That is why the export must go through mailman-web (not mailman / mailman-wrapper, whose shell only sees core).

Requirements

Debian-style Mailman 3 with the mailman-web wrapper on PATH (Django site at /usr/share/mailman3-web/, HyperKitty installed).
Run as root: mailman-web drops privileges to the web user to read the archive, and the script (as root) moves the finished file into place.
Optional, only to enrich the "not archived" notice: curl and the mailman-wrapper core CLI (for a best-effort archive_policy lookup over the core REST API). Missing either just falls back to a generic message.

Usage

mm3-hk-export [-z|--gzip] <list@domain> [output]

output defaults to ./<localpart>.mbox (./<localpart>.mbox.gz with -z).
-z / --gzip gzip-compresses the stream, matching the web download's .mbox.gz byte-for-byte (deterministic header, mtime=0); .gz is appended to the output name if absent.

Examples:

mm3-hk-export announce@lists.example.com /root/mm3/announce.mbox
mm3-hk-export --gzip announce@lists.example.com /root/mm3/announce.mbox.gz

Outcomes

The script never writes a partial or empty file, and exits 0 on all three:

Situation	HyperKitty state	Result
Archive has messages	row with N emails	writes the mbox, prints `wrote N messages to …`
Archived but empty	row, 0 emails	no file, prints an "empty archive" notice
No archive at all	no `MailingList` row	no file; consults core `archive_policy` to say whether the list is `never`-archived or simply has nothing archived yet

A genuine error (e.g. mailman-web failing) exits non-zero with the underlying output on stderr.

In a backup-and-destroy loop

mkdir -p /root/mm3
for list in list1 list2; do
    addr="$list@domain.tld"
    mm3-hk-export "$addr" "/root/mm3/$list.mbox"   # skips cleanly if no archive
    mailman-wrapper members "$addr" > "/root/mm3/$list"
    mailman-wrapper remove "$addr"
done

Export before remove. (HyperKitty's archive lives in a separate database and is not deleted by core's remove, but exporting first keeps the intent clear.)

Caveats

Reconstruction, not the raw original. HyperKitty rebuilds each message from parsed fields: From, To, Subject, Date, Message-ID, In-Reply-To, body, and attachments. Headers like Cc, References, List-*, and custom X-* are not preserved. This is inherent to HyperKitty; the manual web download has the identical limitation.
Memory. Messages are streamed with a DB iterator and written incrementally, avoiding the web export's known blow-up on large lists.
Re-import. The output is valid mbox (proper From separators, From-mangled bodies) and can be re-imported with hyperkitty_import.

3.5 KiB Raw Blame History