mm3-hk-export

Export a private HyperKitty (Mailman 3) list archive to an mbox file from the command line: the scriptable equivalent of clicking the web UI's Download button, so it fits a backup-and-destroy loop without manual browser steps.

Why this exists

HyperKitty has no hyperkitty_export management command; the only built-in export is the web "Download" view (gated by the HYPERKITTY_MBOX_EXPORT setting). This script reproduces that view's exact output by calling the same Email.as_bytes() code path on each archived message, driven through Debian's mailman-web wrapper: no web login, no direct database reconstruction.

The archives live in the mailman-web / HyperKitty database, which is separate from Mailman core. That is why the export must go through mailman-web (not mailman / mailman-wrapper, whose shell only sees core).

Requirements

• Debian-style Mailman 3 with the mailman-web wrapper on PATH (Django site at /usr/share/mailman3-web/, HyperKitty installed).
• Run as root: mailman-web drops privileges to the web user to read the archive, and the script (as root) moves the finished file into place.
• Optional, only to enrich the "not archived" notice: the mailman-wrapper core CLI (for a best-effort archive_policy lookup via mailman shell, reading core's DB directly, no REST, no credentials). Missing it just falls back to a generic message.

Usage

mm3-hk-export [-z|--gzip] <list@domain> [output]

• output defaults to ./<localpart>.mbox (./<localpart>.mbox.gz with -z).
• -z / --gzip gzip-compresses the stream, matching the web download's .mbox.gz byte-for-byte (deterministic header, mtime=0); .gz is appended to the output name if absent.

Examples:

mm3-hk-export announce@lists.example.com /root/mm3/announce.mbox
mm3-hk-export --gzip announce@lists.example.com /root/mm3/announce.mbox.gz

Outcomes and exit codes

The script never writes a partial or empty file. The exit code tells a caller exactly what happened, most importantly whether a backup was produced:

Situation	HyperKitty state	Result	Exit
Archive has messages	row with N emails	writes the mbox, prints wrote N messages to …	0
Archived but empty	row, 0 emails	no file, prints an "empty archive" notice	3
No archive at all	no MailingList row	no file; consults core archive_policy to say whether the list is never-archived or simply has nothing archived yet	3
Usage / bad address	n/a	usage message	2
Genuine error (e.g. mailman-web failing)	n/a	underlying output on stderr	1

Exit 3 ("nothing to archive") is deliberately distinct from 0 ("archive written"). A genuinely empty or never-archived list is a normal, expected outcome, but it is not a successful backup, so a backup-and-destroy caller can refuse to delete a list whose archive was never captured (see below).

In a backup-and-destroy loop

mkdir -p /root/mm3
for list in list1 list2; do
    addr="$list@domain.tld"

    mm3-hk-export "$addr" "/root/mm3/$list.mbox"
    case $? in
        0) ;;                       # archive written; safe to proceed
        3)                          # nothing to archive (empty / no archive):
                                    # no backup file exists, so by default do
                                    # not destroy. Override only if you mean it.
            if [ "${ALLOW_NO_ARCHIVE:-0}" != 1 ]; then
                echo "skip: $addr has no archive to back up; not removing" >&2
                continue
            fi ;;
        *)  echo "skip: $addr export failed; not removing" >&2; continue ;;
    esac

    mailman-wrapper members "$addr" > "/root/mm3/$list.members" \
        || { echo "skip: $addr member dump failed; not removing" >&2; continue; }

    mailman-wrapper remove "$addr"
done

The loop destroys a list only when its archive was actually written (exit 0) and its member roster was dumped. A list with nothing to archive (exit 3) is left intact unless you opt in with ALLOW_NO_ARCHIVE=1, so a transient or unexpected "no archive" can never silently authorise a delete.

Export before remove, and never remove on a failed backup. (HyperKitty's archive lives in a separate database and is not deleted by core's remove, but a failed export must still halt the destroy step.)

Caveats

• Reconstruction, not the raw original. HyperKitty rebuilds each message from parsed fields: From, To, Subject, Date, Message-ID, In-Reply-To, body, and attachments. Headers like Cc, References, List-*, and custom X-* are not preserved. This is inherent to HyperKitty; the manual web download has the identical limitation.
• Memory. Messages are streamed with a DB iterator and written incrementally, avoiding the web export's known blow-up on large lists.
• Re-import. The output is valid mbox (proper From separators, From-mangled bodies) and can be re-imported with hyperkitty_import.
• Handoff integrity. The archive is written by the privilege-dropped web user into /var/tmp, then published by root. Root opens that file with O_NOFOLLOW, copies it while computing a SHA-256, and refuses to publish unless the digest matches the one the exporter computed over the bytes it wrote, so a same-uid process that swaps, truncates, or corrupts the temp file between export and publish is caught (the run fails, nothing is published, and the destroy loop will not remove the list). A fully compromised mailman-web process is out of scope: it builds the archive from a database it already controls, so no file-level check downstream can vouch for the content it produced.