# mm3-hk-export

Export a private **HyperKitty** (Mailman 3) list archive to an `mbox` file from
the command line: the scriptable equivalent of clicking the web UI's
**Download** button, so it fits a backup-and-destroy loop without manual
browser steps.

## Why this exists

HyperKitty has **no `hyperkitty_export` management command**; the only built-in
export is the web "Download" view (gated by the `HYPERKITTY_MBOX_EXPORT`
setting). This script reproduces that view's exact output by calling the same
`Email.as_bytes()` code path on each archived message, driven through Debian's
`mailman-web` wrapper: no web login, no direct database reconstruction.

The archives live in the **mailman-web / HyperKitty** database, which is
separate from Mailman **core**. That is why the export must go through
`mailman-web` (not `mailman` / `mailman-wrapper`, whose `shell` only sees core).

## Requirements

- Debian-style Mailman 3 with the `mailman-web` wrapper on `PATH`
  (Django site at `/usr/share/mailman3-web/`, HyperKitty installed).
- Run as **root**: `mailman-web` drops privileges to the web user to read the
  archive, and the script (as root) moves the finished file into place.
- Optional, only to enrich the "not archived" notice: the `mailman-wrapper`
  core CLI (for a best-effort `archive_policy` lookup via `mailman shell`,
  reading core's DB directly, no REST, no credentials). Missing it just falls
  back to a generic message.

## Usage

```
mm3-hk-export [-z|--gzip] <list@domain> [output]
```

- `output` defaults to `./<localpart>.mbox` (`./<localpart>.mbox.gz` with `-z`).
- `-z` / `--gzip` gzip-compresses the stream, matching the web download's
  `.mbox.gz` byte-for-byte (deterministic header, `mtime=0`); `.gz` is appended
  to the output name if absent.

Examples:

```sh
mm3-hk-export announce@lists.example.com /root/mm3/announce.mbox
mm3-hk-export --gzip announce@lists.example.com /root/mm3/announce.mbox.gz
```

## Outcomes and exit codes

The script never writes a partial or empty file. The **exit code** tells a
caller exactly what happened, most importantly whether a backup was produced:

| Situation | HyperKitty state | Result | Exit |
|---|---|---|:--:|
| Archive has messages | row with N emails | writes the mbox, prints `wrote N messages to …` | `0` |
| Archived but empty | row, 0 emails | no file, prints an "empty archive" notice | `3` |
| No archive at all | no `MailingList` row | no file; consults core `archive_policy` to say whether the list is `never`-archived or simply has nothing archived yet | `3` |
| Usage / bad address | n/a | usage message | `2` |
| Genuine error (e.g. `mailman-web` failing) | n/a | underlying output on stderr | `1` |

Exit **`3`** ("nothing to archive") is deliberately distinct from `0` ("archive
written"). A genuinely empty or never-archived list is a normal, expected
outcome, but it is not a successful backup, so a backup-and-destroy caller
can refuse to delete a list whose archive was never captured (see below).

## In a backup-and-destroy loop

```sh
mkdir -p /root/mm3
for list in list1 list2; do
    addr="$list@domain.tld"

    mm3-hk-export "$addr" "/root/mm3/$list.mbox"
    case $? in
        0) ;;                       # archive written; safe to proceed
        3)                          # nothing to archive (empty / no archive):
                                    # no backup file exists, so by default do
                                    # not destroy. Override only if you mean it.
            if [ "${ALLOW_NO_ARCHIVE:-0}" != 1 ]; then
                echo "skip: $addr has no archive to back up; not removing" >&2
                continue
            fi ;;
        *)  echo "skip: $addr export failed; not removing" >&2; continue ;;
    esac

    mailman-wrapper members "$addr" > "/root/mm3/$list.members" \
        || { echo "skip: $addr member dump failed; not removing" >&2; continue; }

    mailman-wrapper remove "$addr"
done
```

The loop destroys a list only when its archive was actually written
(exit `0`) and its member roster was dumped. A list with nothing to archive
(exit `3`) is left intact unless you opt in with `ALLOW_NO_ARCHIVE=1`, so a
transient or unexpected "no archive" can never silently authorise a delete.

Export before `remove`, and never `remove` on a failed backup. (HyperKitty's
archive lives in a separate database and is not deleted by core's `remove`, but
a failed export must still halt the destroy step.)

## Caveats

- **Reconstruction, not the raw original.** HyperKitty rebuilds each message
  from parsed fields: `From`, `To`, `Subject`, `Date`, `Message-ID`,
  `In-Reply-To`, body, and attachments. Headers like `Cc`, `References`,
  `List-*`, and custom `X-*` are not preserved. This is inherent to
  HyperKitty; the manual web download has the identical limitation.
- **Memory.** Messages are streamed with a DB iterator and written
  incrementally, avoiding the web export's known blow-up on large lists.
- **Re-import.** The output is valid mbox (proper `From ` separators,
  `From`-mangled bodies) and can be re-imported with `hyperkitty_import`.