# mm3-hk-export

Export a private **HyperKitty** (Mailman 3) list archive to an `mbox` file from
the command line: the scriptable equivalent of clicking the web UI's
**Download** button, so it fits a backup-and-destroy loop without manual
browser steps.

## Why this exists

HyperKitty has **no `hyperkitty_export` management command**; the only built-in
export is the web "Download" view (gated by the `HYPERKITTY_MBOX_EXPORT`
setting). This script reproduces that view's exact output by calling the same
`Email.as_bytes()` code path on each archived message, driven through Debian's
`mailman-web` wrapper: no web login, no direct database reconstruction.

The archives live in the **mailman-web / HyperKitty** database, which is
separate from Mailman **core**. That is why the export must go through
`mailman-web` (not `mailman` / `mailman-wrapper`, whose `shell` only sees core).

## Requirements

- Debian-style Mailman 3 with the `mailman-web` wrapper on `PATH`
  (Django site at `/usr/share/mailman3-web/`, HyperKitty installed).
- Run as **root**: `mailman-web` drops privileges to the web user to read the
  archive, and the script (as root) moves the finished file into place.
- Optional, only to enrich the "not archived" notice: the `mailman-wrapper`
  core CLI (for a best-effort `archive_policy` lookup via `mailman shell`,
  reading core's DB directly, no REST, no credentials). Missing it just falls
  back to a generic message.

## Usage

```
mm3-hk-export [-z|--gzip] <list@domain> [output]
```

- `output` defaults to `./<localpart>.mbox` (`./<localpart>.mbox.gz` with `-z`).
- `-z` / `--gzip` gzip-compresses the stream, matching the web download's
  `.mbox.gz` byte-for-byte (deterministic header, `mtime=0`); `.gz` is appended
  to the output name if absent.

Examples:

```sh
mm3-hk-export announce@lists.example.com /root/mm3/announce.mbox
mm3-hk-export --gzip announce@lists.example.com /root/mm3/announce.mbox.gz
```

## Outcomes and exit codes

The script never writes a partial or empty file. The **exit code** tells a
caller exactly what happened, most importantly whether a backup was produced:

| Situation | HyperKitty state | Result | Exit |
|---|---|---|:--:|
| Archive has messages | row with N emails | writes the mbox, prints `wrote N messages to …` | `0` |
| Archived but empty | row, 0 emails | no file, prints an "empty archive" notice | `3` |
| No archive at all | no `MailingList` row | no file; consults core `archive_policy` to say whether the list is `never`-archived or simply has nothing archived yet | `3` |
| Usage / bad address | n/a | usage message | `2` |
| Genuine error (e.g. `mailman-web` failing) | n/a | underlying output on stderr | `1` |

Exit **`3`** ("nothing to archive") is deliberately distinct from `0` ("archive
written"). A genuinely empty or never-archived list is a normal, expected
outcome, but it is not a successful backup, so a backup-and-destroy caller
can refuse to delete a list whose archive was never captured (see below).

## In a backup-and-destroy loop

```sh
mkdir -p /root/mm3
for list in list1 list2; do
    addr="$list@domain.tld"

    mm3-hk-export "$addr" "/root/mm3/$list.mbox"
    case $? in
        0) ;;                       # archive written; safe to proceed
        3)                          # nothing to archive (empty / no archive):
                                    # no backup file exists, so by default do
                                    # not destroy. Override only if you mean it.
            if [ "${ALLOW_NO_ARCHIVE:-0}" != 1 ]; then
                echo "skip: $addr has no archive to back up; not removing" >&2
                continue
            fi ;;
        *)  echo "skip: $addr export failed; not removing" >&2; continue ;;
    esac

    mailman-wrapper members "$addr" > "/root/mm3/$list.members" \
        || { echo "skip: $addr member dump failed; not removing" >&2; continue; }

    mailman-wrapper remove "$addr"
done
```

The loop destroys a list only when its archive was actually written
(exit `0`) and its member roster was dumped. A list with nothing to archive
(exit `3`) is left intact unless you opt in with `ALLOW_NO_ARCHIVE=1`, so a
transient or unexpected "no archive" can never silently authorise a delete.

Export before `remove`, and never `remove` on a failed backup. (HyperKitty's
archive lives in a separate database and is not deleted by core's `remove`, but
a failed export must still halt the destroy step.)

## Caveats

- **Reconstruction, not the raw original.** HyperKitty rebuilds each message
  from parsed fields: `From`, `To`, `Subject`, `Date`, `Message-ID`,
  `In-Reply-To`, body, and attachments. Headers like `Cc`, `References`,
  `List-*`, and custom `X-*` are not preserved. This is inherent to
  HyperKitty; the manual web download has the identical limitation.
- **Memory.** Messages are streamed with a DB iterator and written
  incrementally, avoiding the web export's known blow-up on large lists.
- **Re-import.** The output is valid mbox (proper `From ` separators,
  `From`-mangled bodies) and can be re-imported with `hyperkitty_import`.
- **Handoff integrity.** The archive is written by the privilege-dropped web
  user into `/var/tmp`, then published by root. Root opens that file with
  `O_NOFOLLOW`, copies it while computing a SHA-256, and refuses to publish
  unless the digest matches the one the exporter computed over the bytes it
  wrote, so a same-uid process that swaps, truncates, or corrupts the temp
  file between export and publish is caught (the run fails, nothing is
  published, and the destroy loop will not remove the list). A fully
  compromised mailman-web process is out of scope: it builds the archive from
  a database it already controls, so no file-level check downstream can vouch
  for the content it produced.