Verify export integrity with SHA-256 before publishing

The O_NOFOLLOW + S_ISREG handoff blocks a
symlink swap but still accepts a *regular-file* swap — a process holding
the web uid can replace/truncate the /var/tmp temp with different content
before root copies it, and root would publish it and exit 0, which the
destroy loop reads as authorization to remove the list.

The exporter now hashes the bytes it wrote (read back through its own
held fd — the original inode, immune to any name swap) and returns the
SHA-256 on the FILE status line. Root copies the source while hashing and
refuses to publish unless the digest matches; a swap/truncation/partial
write yields a mismatch, a non-zero exit, and no removal.

This does NOT defend a fully compromised mailman-web process (it controls
the source DB the archive is built from) — that is inherent and now
documented as out of scope. It closes same-uid swaps, partial writes, and
accidental corruption while staying in the existing architecture.

Verified: held-fd hash equals an independent by-name read (gzip + plain);
correct digest publishes (0600, content intact, source unlinked); wrong
digest, post-hash swap, and empty digest are all rejected with nothing
published; end-to-end NONE/EMPTY still exit 3, bad digest exits 1.

README.md

@@ -111,3 +111,13 @@ a failed export must still halt the destroy step.)
   incrementally, avoiding the web export's known blow-up on large lists.
 - **Re-import.** The output is valid mbox (proper `From ` separators,
   `From`-mangled bodies) and can be re-imported with `hyperkitty_import`.
+- **Handoff integrity.** The archive is written by the privilege-dropped web
+  user into `/var/tmp`, then published by root. Root opens that file with
+  `O_NOFOLLOW`, copies it while computing a SHA-256, and refuses to publish
+  unless the digest matches the one the exporter computed over the bytes it
+  wrote, so a same-uid process that swaps, truncates, or corrupts the temp
+  file between export and publish is caught (the run fails, nothing is
+  published, and the destroy loop will not remove the list). A fully
+  compromised mailman-web process is out of scope: it builds the archive from
+  a database it already controls, so no file-level check downstream can vouch
+  for the content it produced.

mm3-hk-export

@@ -73,7 +73,7 @@ EOF
 }
 
 result=$(mailman-web shell <<EOF
-import sys, os, tempfile, gzip
+import sys, os, tempfile, gzip, hashlib
 from hyperkitty.models import MailingList
 addr = "$addr"
 GZIP = $gzip_py
@@ -90,7 +90,9 @@ if n == 0:
     print("MM3HK|EMPTY|0")
     sys.exit(0)
 fd, path = tempfile.mkstemp(prefix="mm3hk-", suffix=".mbox", dir="/var/tmp")
-raw = os.fdopen(fd, "wb")
+# w+b so we can read the bytes back through this same fd (the original inode,
+# immune to any /var/tmp name-swap) to hash exactly what we wrote.
+raw = os.fdopen(fd, "w+b")
 # mtime=0 keeps the gzip header deterministic (no timestamp), matching the
 # web export, which uses a bare zlib gzip stream.
 fh = gzip.GzipFile(fileobj=raw, mode="wb", mtime=0) if GZIP else raw
@@ -100,9 +102,18 @@ try:
         fh.write(b"\n\n")
 finally:
     if GZIP:
-        fh.close()
-    raw.close()
-print("MM3HK|FILE|%d|%s" % (n, path))
+        fh.close()  # flush the gzip trailer into raw (does not close raw)
+    raw.flush()
+# Hash the on-disk bytes via the held fd, then hand root the digest so it can
+# verify the file wasn't swapped/truncated by another same-uid process before
+# it copies it. (A fully-compromised exporter is out of scope, since it controls
+# the source DB, but this closes same-uid swaps, partial writes, and corruption.)
+h = hashlib.sha256()
+raw.seek(0)
+for chunk in iter(lambda: raw.read(1 << 16), b""):
+    h.update(chunk)
+raw.close()
+print("MM3HK|FILE|%d|%s|%s" % (n, path, h.hexdigest()))
 EOF
 )
 
@@ -141,6 +152,7 @@ case "$status" in
         ;;
     FILE)
         path=$(printf '%s' "$line" | cut -d'|' -f4)
+        sha=$(printf '%s' "$line" | cut -d'|' -f5)
         mkdir -p "$(dirname "$out")"
         # Publish the web-user temp file as root without re-resolving its path.
         # The export lives in /var/tmp owned by the web user, so any process
@@ -149,11 +161,14 @@ case "$status" in
         # root-readable file. So: open the source ONCE with O_NOFOLLOW (a swapped
         # symlink makes the open fail outright), confirm it's a regular file via
         # the open fd, copy from that fd into a fresh root-owned temp in the
-        # destination dir, fsync, and atomically rename into place. Paths are
-        # passed as argv (heredoc is single-quoted) so nothing is interpolated.
-        if ! python3 - "$path" "$out" <<'PY'
-import os, sys, stat, shutil, tempfile
-src, out = sys.argv[1], sys.argv[2]
+        # destination dir while hashing, verify the SHA-256 matches what the
+        # exporter reported (rejects a regular-file swap, truncation, or partial
+        # write by another same-uid process), fsync, and atomically rename into
+        # place. Paths + digest are passed as argv (heredoc is single-quoted) so
+        # nothing is interpolated.
+        if ! python3 - "$path" "$out" "$sha" <<'PY'
+import os, sys, stat, tempfile, hashlib
+src, out, expected = sys.argv[1], sys.argv[2], sys.argv[3]
 try:
     fd = os.open(src, os.O_RDONLY | os.O_NOFOLLOW)
 except OSError as e:
@@ -163,11 +178,17 @@ try:
     if not stat.S_ISREG(os.fstat(fd).st_mode):
         sys.exit("mm3-hk-export: export temp is not a regular file")
     dfd, dtmp = tempfile.mkstemp(prefix=".mm3hk.", dir=os.path.dirname(out) or ".")
+    h = hashlib.sha256()
     r = os.fdopen(fd, "rb"); fd = -1      # fd now owned by r
     with r, os.fdopen(dfd, "wb") as w:    # dfd now owned by w
-        shutil.copyfileobj(r, w)
+        for chunk in iter(lambda: r.read(1 << 16), b""):
+            h.update(chunk)
+            w.write(chunk)
         w.flush()
         os.fsync(w.fileno())
+    if not expected or h.hexdigest() != expected:
+        os.unlink(dtmp); dtmp = None
+        sys.exit("mm3-hk-export: export integrity check failed (file changed under us); not published")
     os.replace(dtmp, out)                 # atomic same-dir rename, root-owned, 0600
     dtmp = None                           # consumed by the rename
     # fsync the containing directory so the new directory entry is crash-durable