gomail/gomail-action-plan-v4.md

GoMail — Action Plan v4 (post-Phase 9)

Status: Phases 1–9 complete and verified. Phases 10–15 remain.

This supersedes v3 for anything marked complete below. Where v3's design held up under actual implementation, it's summarized, not repeated — see v3 for full rationale on decisions that didn't change (encryption scheme, POP3 off-by-default, OAuth self-registration, time-based linked-account caching, etc.).

---

What's built (Phases 1–9)

Phase	Package(s)	What it does	Status
1	config, crypto, db	YAML config + env secrets, HKDF-per-record encryption, SQLite schema/migrations, bootstrap	✅ verified
2	smtp, mailstore, tlsutil	Inbound MTA (:25/:587/:465), STARTTLS, SASL PLAIN/LOGIN, encrypted Maildir delivery	✅ verified
3	dkim, queue	DKIM sign/verify (stdlib crypto only), outbound retry queue, MX delivery via net/smtp, bounce DSN	✅ verified
4	pipeline	SPF/DKIM/DMARC/header/URL checks, quarantine storage, verdict scoring	✅ verified against live DNS
5	imap, pop3, auth	Full IMAP core command set, POP3 (off by default), shared credential verification	✅ verified
6	accounts, imapclient	MailProvider interface, local + generic-IMAP backends, hand-rolled IMAP client	✅ verified (dogfoods own server)
7	vcard, ical, dav	CalDAV/CardDAV server (PROPFIND/REPORT/PUT/GET/DELETE), encrypted contacts/events	✅ verified
8	webtoken, webmail	Hand-rolled JWT, REST API + dark Tailwind SPA (folders/messages/compose/quarantine)	✅ verified
9	jmap	JMAP Core/Mail subset (session, Core/echo, Mailbox/get, Email/query, Email/get)	✅ verified
9.5	sieve, managesieve	Sieve interpreter (RFC 5228 subset) + ManageSieve server (RFC 5804), wired into inbound delivery	✅ verified
10	oauth2, extended accounts/imapclient/webmail	Hand-rolled OAuth2 (auth code grant + refresh), XOAUTH2 IMAP login, webmail account-linking endpoints (CSRF-protected)	✅ verified against fake OAuth2 server + fake XOAUTH2 IMAP stub — see note below
11	admin	Admin portal: RBAC-gated REST API (global_admin/tenant_admin) + dark Tailwind SPA — tenants, domains (+DKIM keygen/rotation), users, list rules, outbound queue, global quarantine, dashboard stats	✅ verified — full RBAC enforcement, real DKIM key generation/rotation checked cryptographically
13	acme, tlsutil (ACMEManager)	Real ACME v2 client (RFC 8555): JWS/ES256 signing, HTTP-01 challenge, SNI-aware cert serving, encrypted storage, auto-renewal loop. Pulled forward from phase order — see rationale below.	✅ verified against a from-scratch fake ACME server with real JWS/thumbprint verification — see note below
12	totp, extended webmail/db	Auth hardening: TOTP MFA (RFC 6238) with two-step login, backup codes, app-password self-service UI, recovery-email password reset. Completed after Phase 13 since 13 was pulled forward — see Phase 13's entry above.	✅ verified — TOTP checked against 5 real RFC 6238 published test vectors, full two-step login/backup-code/app-password/reset flows over real HTTP
14	extended pipeline (ClamAV/Rspamd/LLM stages)	Optional external security services: real clamd INSTREAM protocol, real rspamd /checkv2 HTTP API, OpenAI-compatible LLM classification (llama.cpp server) — all off by default, only added to the pipeline when configured	✅ verified — genuine EICAR malware detection (byte-level, not canned), real rspamd score pass-through, LLM score capping, graceful negative-path handling for unreachable services
15	ratelimit, fuzz tests, install.sh/gomail.service/README.md	Hardening + deploy: per-IP token-bucket rate limiting on every listener, fuzz tests on the 4 riskiest hand-written parsers, go vet pass, systemd install script + hardened unit, full README with DNS setup guide	✅ verified — rate limiting proven under the race detector and over real sockets/HTTP, ~1.5M fuzz executions with zero panics, systemd unit verified with systemd-analyze verify

~14,400 lines of Go, one third-party runtime dependency beyond mattn/go-sqlite3/golang.org/x/crypto/google/uuid/gopkg.in/yaml.v3 — everything else (SMTP, IMAP, POP3, JMAP, CalDAV/CardDAV, JWT, DKIM, SPF, DMARC) is hand-rolled on the Go standard library, per the original dependency-minimal requirement.

Deferred within completed phases (honest gaps, not hidden)

• DAV: REPORT filter/time-range queries return the full collection, not a filtered subset. MKCALENDAR/MKCOL not implemented — default collection auto-created per user instead.
• IMAP: no IDLE, CONDSTORE/QRESYNC, SORT/THREAD, or mailbox CREATE/DELETE/RENAME. Core command set only (see Phase 5 code comments).
• JMAP: Email/set (flags/delete via JMAP), Email/import (send via JMAP), push (EventSource), and Mailbox/set are not implemented — the webmail SPA still uses Phase 8's direct REST API for all mutations. ManageSieve (RFC 5804) was paired with JMAP in the original plan and was not started this phase at all — full gap, see Phase 9.5 below.
• accounts.IMAPProvider.Move: returns "not implemented" — needs IMAP APPEND, which isn't in the imapclient package yet.
• TLS: self-signed cert generation only (tlsutil), no ACME. Every linked IMAP/SMTP connection uses InsecureSkipVerify: true as a known, commented gap — real cert trust (pinning or ACME) is Phase 13.

---

Remaining phases

Phase 9.5 — ManageSieve + Sieve interpreter — ✅ COMPLETE

internal/sieve: hand-written recursive-descent interpreter covering if/elsif/else, header :contains/:is tests, fileinto/discard/ keep/stop actions. Verified: correct short-circuit on stop, correct RFC 5228 §4.4 "explicit keep after discard still delivers" semantics, malformed scripts rejected at parse time.

internal/managesieve: RFC 5804 server on :4190 — CAPABILITY, STARTTLS (auth refused before TLS, verified), AUTHENTICATE PLAIN, PUTSCRIPT (rejects syntactically invalid scripts at upload time via sieve.Parse), GETSCRIPT, LISTSCRIPTS, SETACTIVE (enforces exactly one active script per user), DELETESCRIPT, LOGOUT.

Wired into internal/smtp/session.go's local-delivery path: after the security pipeline clears a message (clean/flagged), the recipient's active Sieve script (if any) runs against the message headers and determines the destination folder or discard — verified end-to-end with a real SMTP delivery routed by an uploaded rule.

Deferred within this phase: only header tests are supported (no address, envelope, size, exists tests); no anyof/allof boolean combinators (only single-condition if/elsif); no Sieve extensions (vacation, reject, notify, imap4flags); webmail visual rule builder not built — raw script upload via the ManageSieve protocol is the only interface (testable with sieve-connect or similar real clients).

Phase 10 — Multi-account webmail: OAuth2 + Gmail/M365 — ✅ PARTIALLY COMPLETE

What's done and verified:

• internal/oauth2: hand-rolled OAuth2 authorization-code grant + refresh (RFC 6749 §4.1, §6), ~150 lines, stdlib net/http+encoding/json only. WellKnownEndpoints("google"|"microsoft") returns the real fixed endpoint URLs — not operator-configurable, matching how real integrations work (only Client ID/Secret are operator-supplied).
• imapclient.LoginXOAUTH2: SASL XOAUTH2 mechanism for connecting to Gmail/M365 with a bearer token instead of a password.
• accounts.IMAPProvider extended: branches on AuthType — password accounts unchanged, OAuth2 accounts decrypt a stored token, transparently refresh it if expired (persisting the new token to avoid re-refreshing on every call), and authenticate via XOAUTH2.
• accounts.LinkOAuth2Account / wellKnownIMAPHost: stores encrypted OAuth2 tokens, points Gmail/M365 accounts at their real documented IMAP hosts (imap.gmail.com:993, outlook.office365.com:993).
• webmail API: /api/accounts (list), /api/accounts/{id} (unlink), /api/accounts/oauth/{provider}/start (builds auth URL, stores CSRF state server-side keyed to the authenticated user), /api/accounts/oauth/{provider}/callback (validates state — rejects unknown AND replayed state, verified — exchanges code, links account).
• Verified with a from-scratch fake OAuth2 authorization server (/authorize + /token, both grant types) and a fake XOAUTH2-accepting IMAP stub — full round trip: start → redirect → callback → encrypted token storage → real XOAUTH2 IMAP login → forced token expiry → transparent refresh → persisted new token. This is real protocol-correctness verification; it does not touch real Google/Microsoft infrastructure, since no real app credentials exist in the dev environment this was built in. The next session should register a real Google Cloud OAuth app (or ask the person for one) and do one live end-to-end test against actual Gmail before considering this phase fully closed — the protocol layer is proven, but "does real Gmail's token endpoint actually behave the way RFC 6749 says it should" has not been checked against the real service.

Deferred within this phase:

• Native Gmail API / Microsoft Graph API providers (GmailProvider, M365Provider as distinct from the IMAP+OAuth2 path) — per the locked decision, mail goes through IMAPProvider with OAuth2 credentials for now; native API push (Gmail watch+Pub/Sub, Graph delta webhooks) is Phase 14 per the original plan.
• Calendar/contacts via native Graph API / Gmail Calendar+People API — not started. No IMAP fallback exists for these, so this is still fully greenfield work reusing the OAuth2 token store just built.
• Webmail UI: account switcher, unified "All Inboxes" view, per-account "send as" in compose, federated search — the backend (/api/accounts) exists but the SPA has no UI for any of this yet.
• Real userinfo/profile API call to learn the linked account's actual email address at callback time — currently accepts an ?email= query param as a stand-in (noted in code); a real implementation calls Google's userinfo endpoint or Microsoft Graph's /me with the fresh access token.
• IMAPProvider.Move still returns "not implemented" (needs IMAP APPEND, unchanged from Phase 6).

Phase 11 — Admin portal — ✅ COMPLETE (core CRUD)

What's done and verified:

• internal/admin: REST API + embedded dark Tailwind SPA, JWT sessions (same webtoken scheme as webmail), role checked at auth time AND re-checked against the live DB row on every request (a demoted admin's existing token stops working immediately, not just after expiry).
• Tenants: list/create, global_admin only (verified: tenant_admin gets 403).
• Domains: list/create/delete, real DKIM key generation on create (verified: stored key decrypts and parses as a genuine RSA private key, not just "a create call succeeded"), DKIM rotation (verified: key material genuinely changes, not a no-op). global_admin defaults to their own tenant if none specified; tenant_admin forced to their own tenant regardless of what they send.
• Users: list/create/suspend/activate/reset-password/delete. tenant_admin scoped to their own tenant and blocked from creating or modifying admin-role accounts (verified: 403 on a privilege-escalation attempt).
• List rules: list/create/delete, tenant-scoped.
• Outbound queue: list, retry-now (verified: next_attempt_at genuinely rescheduled to immediate, not just a 200 response), cancel.
• Global quarantine: list, discard (verified: status genuinely transitions to deleted in the DB). Per-user release (as opposed to discard) already existed in webmail from Phase 8 — this is the admin-wide review/cleanup view, not a duplicate of that.
• Dashboard: user/domain counts, 24h message count, queue depth, quarantine-held count.

Two real bugs caught by testing, both fixed:

1. global_admin has no forced tenant (unlike tenant_admin), but every domain still needs one — domain creation 400'd with "tenant_id is required" until a fallback to the admin's own tenant was added.
2. User creation requires a valid domain_id (a mailbox must belong to a domain) — neither the test nor the admin SPA's "Add User" form supplied one at first. Fixed in both: the SPA now has a domain ID field, and the Domains page shows each domain's ID (click to copy) so there's an actual way to get that value into the form.

Deferred within this phase:

• Aliases CRUD (table exists, no admin UI yet).
• Pipeline settings UI (per-tenant score thresholds / check toggles) — the tenants.settings_json column exists but nothing reads or writes it yet; pipeline behavior is still entirely driven by the global config.Pipeline values.
• TLS cert status view, DMARC/TLS-RPT report viewer — both depend on Phase 13 (ACME/DANE/MTA-STS) existing first.
• Live log viewer (SSE) — webmail's sseEvents pattern from Phase 8 is directly reusable here, just not wired up yet.
• Admin IP allowlist (admin_ip_allowlist in config) is not enforced at the HTTP layer — the admin server binds to 127.0.0.1 by default, which covers the common case, but the config value itself is currently inert.

Phase 13 — TLS + ACME + DANE/MTA-STS — ✅ CORE COMPLETE (pulled forward)

Pulled ahead of strict phase order per the reasoning that closing the InsecureSkipVerify gap improves the security posture of everything already built more than most net-new features would — see the handover doc for the fuller rationale.

What's done and verified:

• internal/acme: hand-rolled ACME v2 client (RFC 8555) — no third-party ACME/JOSE library. jws.go implements JWS signing (ES256, flattened JSON serialization per RFC 7515) and RFC 7638 JWK thumbprint computation from scratch. client.go implements the full protocol: directory fetch, nonce management, account registration, order creation, authorization polling, HTTP-01 challenge response, CSR building, finalize, and certificate download. challenge.go is the HTTP-01 responder (/.well-known/acme-challenge/{token}). obtain.go is the high-level orchestration function tying it together.
• internal/tlsutil/acme_manager.go: ACMEManager provides SNI-aware certificate serving (tls.Config.GetCertificate), encrypted-at-rest storage (same HKDF-per-record scheme as messages/contacts/DKIM keys — new tls_certs table, migration 0010), in-memory caching, and a background renewal loop (checks daily by default, renews within 30 days of expiry).
• Wired into cmd/gomail/main.go: when tls.mode: acme and tls.acme_domains are configured, a real ACMEManager drives cert serving instead of the self-signed fallback, and a plain :80 listener serves the HTTP-01 challenge responder. Falls back to self-signed with a now-accurate warning message when ACME mode is set but no domains are configured, or when mode is off.
• Verified with a from-scratch fake ACME server (cmd/e2etest13, deleted after passing per the established pattern) that does real protocol-level verification, not a rubber stamp: ES256 JWS signature verification against the account's actual public key, RFC 7638 thumbprint recomputation from that key, and a genuine HTTP callback to the challenge responder to check the key authorization matches — exactly what a real CA does. Confirmed: full obtain flow succeeds; the issued certificate works in an actual tls.Dial handshake with the correct CommonName; the certificate is encrypted at rest (checked the raw DB column, then decrypted and confirmed it's a real PEM cert); a second call hits the cache instead of re-issuing; a tampered JWS signature is correctly rejected; and — importantly — finalizing an order without completing domain validation is correctly refused with 403, proving the fake CA (and by extension, the real protocol logic our client drives) won't issue a certificate without genuine authorization.
• Two stale/inaccurate log messages caught and fixed while wiring this in: the self-signed fallback's warning previously claimed "ACME is not yet implemented (Phase 13)" — no longer true, so the message is now context-aware (distinguishes "acme mode but no domains configured" from "mode is off entirely"). A comment in accounts.IMAPProvider about InsecureSkipVerify was similarly updated to reflect that the gap is now narrower (GoMail's own server can get a real cert; a linked external account's server is still outside our control regardless).

Deferred within this phase (genuinely not started, not just untested):

• DANE (TLSA record lookup + verification for outbound delivery) — not implemented.
• MTA-STS (fetch/cache .well-known/mta-sts.txt, enforce TLS to declared MX hosts) — not implemented.
• TLS-RPT (aggregate report generation/sending) — not implemented.
• ACME DNS-01 challenge type (only HTTP-01 is implemented) — DNS-01 is necessary for wildcard certs, which GoMail doesn't currently need, but worth naming as a real gap rather than assuming HTTP-01 is sufficient forever.
• No live test against a real CA (Let's Encrypt staging or production) — same category of limitation as Phase 10's OAuth2: no real public domain/DNS control in this sandbox to satisfy HTTP-01 validation from the actual internet. The protocol implementation is proven correct against a genuinely-verifying fake server; a live run against Let's Encrypt staging is the natural first action for whoever continues this, same recommendation as Phase 10's OAuth caveat.
• Every InsecureSkipVerify: true in the codebase (linked IMAP accounts, the ACME manager's own... actually the ACME manager doesn't skip verification, only the generic IMAP/OAuth2 linked account paths in internal/accounts still do, since those connect to servers outside GoMail's control) is not yet an argument for removal — search the codebase for that string to find remaining call sites.

Phase 12 — Auth hardening — ✅ CORE COMPLETE (TOTP + app passwords + reset)

Completed after Phase 13 in this session's actual order (13 was pulled forward for its security-posture impact; this closed the gap in numeric order afterward).

What's done and verified:

• internal/totp: hand-rolled RFC 6238 TOTP (HOTP RFC 4226 underneath) — no third-party OTP library. Verified against 5 of RFC 6238's own published Appendix B test vectors (not just internal self-consistency): our 6-digit output exactly matches the last 6 digits of each published 8-digit reference vector, which is mathematically guaranteed to hold only if the HMAC-SHA1 counter derivation and truncation are byte-for-byte correct. Also verified: ±30s clock-skew tolerance works and correctly has a boundary (120s does NOT validate), and the otpauth:// provisioning URI is well-formed.
• Two-step login: POST /api/auth/login returns {mfa_required: true, mfa_token: <5-min pre-auth token>} instead of a session when user.MFAEnabled — POST /api/auth/mfa-verify redeems that token plus either a valid TOTP code or an unused backup code for a real session. The pre-auth token is a genuinely distinct, narrowly-scoped token type (webtoken.Claims.Purpose = "mfa_pending"), not just a normal session token handed out early.
• MFA setup/confirm/disable: POST /api/me/mfa/setup generates a secret and stores it encrypted but not yet enabled — POST /api/me/mfa/confirm requires one valid code before MFA actually takes effect and backup codes are issued, so an abandoned setup never locks anyone out. 8 backup codes (SHA-256 hashed, one-time use, verified) are shown to the user exactly once, matching the app-password pattern. POST /api/me/mfa/disable re-requires the password (a session token alone isn't enough to turn MFA off).
• App passwords: full self-service CRUD (GET/POST /api/me/app-passwords, DELETE /api/me/app-passwords/{id}) — the server-side scope-checking machinery already existed from Phase 5, this phase added the UI-facing API. Verified: the raw token is returned exactly once at creation and never re-appears in the listing response.
• Password reset: POST /api/auth/forgot-password + POST /api/auth/reset-password, gated on a user-configured recovery_email (new users.recovery_email column) rather than the account's own mailbox — deliberately avoids the chicken-and-egg problem of emailing a reset link to a mailbox the person is locked out of. Verified: a genuine outbound-queue entry is created addressed to the recovery email (real delivery infrastructure, not a stub), and the endpoint returns the identical response whether or not the account exists (no account-existence leak via response differences).

Two real bugs found and fixed while testing, both worth remembering:

1. db.GetUser was built in Phase 11 for admin's minimal display needs (id/email/role/active) and never extended when Phase 12 code started depending on it for MFA fields — mfa_confirm failed with "no pending MFA setup" even though setup had just succeeded, because the fetch silently dropped totp_secret_enc. Fixed by making GetUser the one canonical full-row fetch rather than fragmenting into admin-flavored/auth-flavored variants.
2. A genuine SQLite connection-pool deadlock in ConsumeBackupCode: it ran a db.Query (find the matching backup code) followed by a db.Exec (mark it used) in the same function, but this database is capped to a single open connection (SetMaxOpenConns(1), see internal/db/db.go) — calling Exec while the Query's rows was still open (only deferred-closed, not yet executed) caused the request to hang forever waiting for a connection that could only free up after the function returned. Fixed by closing rows explicitly before the Exec. The codebase was then scanned for the same pattern (any function mixing Query and Exec) and this was confirmed isolated — worth doing that scan again after any future change that mixes both in one function, given SQLite's single-connection constraint makes this an easy trap to fall into.

Deferred within this phase:

• Passkey/WebAuthn — not started at all, and deliberately so: it requires CBOR decoding for attestation/assertion objects, which is not in Go's standard library and would be a meaningfully sized sub-project of its own (on the order of the JWS/ACME work in Phase 13), with little shared surface with anything else in this codebase. users. passkey_credentials_json exists in the schema as a placeholder for when this is picked up.
• QR code rendering for TOTP setup is not implemented — totp. ProvisioningURI returns the otpauth:// URI as plain text, which every mainstream authenticator app accepts as manual entry; actual QR rendering (Reed-Solomon error correction, matrix placement) is real standalone work, noted in totp.go's doc comment rather than silently skipped.
• No webmail SPA pages for any of this yet — MFA setup/confirm, app password management, and recovery-email configuration are all functional, tested REST endpoints with no frontend UI. The existing internal/webmail/static/index.html pattern is directly reusable, just not built out for these yet.

Phase 14 — Optional external services — ✅ CORE COMPLETE (ClamAV/Rspamd/LLM)

What's done and verified:

• internal/pipeline/stage_clamav.go: ClamAVStage speaks clamd's real documented INSTREAM binary protocol by hand — no third-party clamd client library. Send "zINSTREAM\0", then the message in 4-byte- big-endian-length-prefixed chunks terminated by a zero-length chunk, then read the single-line response. Accepts unix: or tcp: prefixed addresses. A detected match is always a hard block (score 100) rather than a scored contribution, since malware detection isn't a "maybe." Verified with a genuine EICAR test-string detection — the fake clamd test server actually reassembles the streamed chunks and inspects the bytes for the real industry-standard EICAR signature, the same way real clamd does, rather than returning a canned response; a clean message correctly passes, and an unreachable clamd is handled as CheckError rather than crashing or hanging.
• internal/pipeline/stage_rspamd.go: RspamdStage POSTs the raw RFC 5322 message to rspamd's real documented /checkv2 HTTP endpoint and maps its JSON response (score, action, symbols) onto this pipeline's result model — reject → CheckFail, add header/ rewrite subject/greylist → CheckWarn, anything else → CheckPass. rspamd's own score is passed through directly rather than being rescaled, since both scales are already meant to be compared against configurable thresholds the same way. Verified: score pass-through is exact, action-to-result mapping is correct for all three cases tested.
• internal/pipeline/stage_llm.go: LLMStage calls the OpenAI-compatible /v1/chat/completions endpoint that llama.cpp's server (and most other local-inference servers) expose — no llama.cpp-specific protocol needed. The model is asked for a single 0-100 integer; parsing extracts the first run of digits rather than requiring an exact match, since local models don't always follow format instructions exactly (verified with a deliberately messy response — extra whitespace and trailing commentary — that still parses correctly). LLM output is capped at 30 points of total score contribution regardless of what the model returns, since non-deterministic model output shouldn't singlehandedly quarantine mail the way a deterministic SPF/DKIM/DMARC failure can — verified the cap applies correctly (raw score 95/100 → capped contribution 28.5, not 95).
• pipeline.StagesFromConfig: builds DefaultStages() (the always-on deterministic set) plus any of the three optional stages whose config field (clamav_socket / rspamd_url / llm_url) is non-empty — an unconfigured optional service is genuinely absent from the pipeline, not merely present-but-disabled, so a misconfigured or unreachable service that was never meant to be used can't accidentally affect delivery. Verified: 5 stages with nothing configured, 8 with all three configured. Wired into cmd/gomail/main.go in place of the old DefaultStages() call.

One real bug found and fixed while testing — in the test's fake clamd server, not the product code: the fake server's initial read of the "zINSTREAM\0" command used a generic buffered conn.Read() call with a 32-byte buffer, which could over-read into the next protocol bytes (TCP doesn't preserve write-call boundaries, so a client's separate Write() calls can arrive coalesced in one read). That silently desynced the fake server's chunk-length parsing loop, which then hung waiting for bytes that had already been consumed — manifesting as an i/o timeout on the client side, initially indistinguishable from a real product bug. Fixed by switching the fake server to a bufio.Reader with exact-length io.ReadFull reads for both the command and every subsequent chunk. Worth remembering as a general pattern for any future protocol work over raw TCP: never assume one Read() call aligns with the sender's Write() call boundaries.

Deferred within this phase:

• Gmail API push (watch+Pub/Sub) and Microsoft Graph delta webhooks, to replace Phase 10's IMAP polling for linked accounts — not started.
• No live test against real ClamAV, rspamd, or a real llama.cpp server — same category of limitation as Phase 10's OAuth2 and Phase 13's ACME: the protocol implementations are proven correct against genuinely- behaving fake servers, but never against the real services. If the operator has any of these installed, pointing clamav_socket / rspamd_url / llm_url at them and sending one real test message (the EICAR string is safe and appropriate for this) is the natural first live-test.
• No admin UI toggle for these — they're config-file only (pipeline.clamav_socket, pipeline.rspamd_url, pipeline.llm_url/ llm_model in config.yaml), consistent with how they were already scaffolded as plain fields back in Phase 1.

Phase 15 — Hardening + deploy — ✅ CORE COMPLETE (last phase in the plan)

What's done and verified:

• internal/ratelimit: hand-rolled per-key token-bucket limiter (no third-party rate-limiting library) — a token bucket rather than a fixed window deliberately, to avoid the classic "burst allowed at both edges of a reset boundary" flaw a naive counter has. rate=0 cleanly disables limiting for a given listener rather than blocking everything, which is how an unset config value opts out. Verified under Go's race detector with 500 concurrent goroutines against a rate=100 limiter — exactly 100 allowed, zero drift, proving the mutex genuinely serializes access rather than merely looking correct single-threaded. Then proven over real sockets and real HTTP: a real TCP listener genuinely rejecting connections past the limit, a real httptest server returning genuine 429s past the limit, and a specific check that X-Forwarded-For trust correctly keys the limiter on the forwarded IP rather than the raw socket peer (only trusted when server.real_ip_header is explicitly configured — untrusted otherwise, since blindly trusting it would let any client spoof their rate-limit identity).
• Wired into every listener: SMTP (per-IP connection limit at accept time, plus a separate limiter on AUTH attempts specifically, checked before any credential parsing happens), IMAP (per-IP connection limit at accept time), and a single shared limiter across DAV, webmail, admin, and external JMAP — deliberately one combined per-IP budget across all of them, not one each, since a client hitting its limit on one HTTP surface shouldn't get a fresh budget by switching to another.
• Fuzz tests (go test -fuzz, real *_fuzz_test.go files — the first standard Go test files in this repo; everything before this used disposable cmd/e2etestN directories) on the four hand-written parsers most exposed to untrusted network input: vcard.Parse, ical.Parse, sieve.Parse (explicitly flagged as the highest-stakes target, since every ManageSieve PUTSCRIPT runs through it), and IMAP's tokenize (runs on every line a connected client sends, before authentication necessarily succeeds). Each actually run for 15s, not just written: ~400K–590K executions per target, ~1.5M executions total, zero panics.
• go vet ./... — clean across the entire codebase. staticcheck was attempted but is not runnable in this sandbox — it requires a newer Go toolchain than this network-restricted environment can fetch (same class of limitation documented elsewhere in this plan for the real Go 1.25.6 toolchain). This is a real, disclosed gap: go vet provides real but narrower coverage than staticcheck would. Running staticcheck ./... on a real machine with normal internet access is a legitimate quick win for whoever continues this.
• install.sh: creates a dedicated system user (no login shell), sets up /etc/gomail, /var/lib/gomail, /var/log/gomail with correct ownership, builds and installs the binary, generates GOMAIL_MASTER_KEY/GOMAIL_JWT_SECRET into a mode-600 env file if none exist, installs and enables (but does not start) the systemd unit. Syntax-checked with bash -n (clean); shellcheck isn't available in this sandbox either, so deeper static analysis of the script itself is another real machine follow-up.
• gomail.service: systemd unit with genuine hardening — capability-based privileged-port binding (AmbientCapabilities=CAP_NET_BIND_SERVICE) instead of running as root, plus ProtectSystem=strict, ProtectHome, PrivateTmp, PrivateDevices, kernel/clock/hostname protection, namespace/SUID/realtime restrictions, MemoryDenyWriteExecute (with an inline comment flagging it as the first thing to try removing if the CGO-based binary fails to start under it — mattn/go-sqlite3 requires CGO), and a syscall filter. Actually verified with systemd-analyze verify (available in this sandbox, unlike staticcheck/shellcheck) — returned clean against a dummy executable standing in for the real binary.
• README.md: quick start, TLS/ACME guidance (including the staging-before-production reminder from Phase 13's own caveat), and a full DNS setup section — MX, SPF, DKIM (with the exact record shape GoMail logs at bootstrap), DMARC, all with real example values and guidance on starting permissive (~all, p=quarantine) before tightening. MTA-STS/TLS-RPT sections are present but honestly marked not-yet-implemented rather than filled with placeholder content.

Deferred within this phase:

• staticcheck and shellcheck — both blocked by sandbox network/ toolchain restrictions, not by any decision; run both on a real machine.
• No fuzz corpus regression files exist yet (expected — Go only persists a corpus entry to testdata/fuzz/ on an actual crash, and none of the 15s runs found one; longer fuzzing runs on a real machine, especially with -fuzztime measured in hours rather than seconds, stand a better chance of finding something a 15-second burst didn't).
• No fuzz targets for the SMTP command parser or JMAP JSON handling specifically — the four targets chosen were judged highest-value for the time available; SMTP's command parsing is somewhat covered indirectly by the SMTP E2E tests from earlier phases but not fuzzed directly, and JMAP's JSON handling rides on encoding/json, which is already extensively fuzzed upstream in the Go project itself.
• No log rotation configuration, no monitoring/metrics endpoint, no backup tooling for the SQLite database or Maildir tree — genuinely unstarted, not scoped in the original Phase 15 description either.

---

Project status: all 15 phases from the original plan have now been

touched, 13 of them completed and verified

Phase 10 (native Gmail/Graph API, webmail account-switcher UI) and Phase 11 (aliases CRUD, per-tenant pipeline settings UI) remain "partially" or "core" complete with named remaining pieces — see their entries above. Every other phase reached a genuinely complete, tested state. This does not mean the project is finished in an absolute sense — every phase's entry above lists real deferred items, and the live-service verification caveats (OAuth2 against real Google, ACME against real Let's Encrypt staging) are still open — but the plan as originally scoped has been worked through in full, not just partially sampled.

---

Architecture notes that matter for continuation

Dependency management in a network-restricted sandbox

The Anthropic sandbox this was built in blocks proxy.golang.org and sum.golang.org — only github.com (plus npm/pypi, irrelevant here) is reachable. Every session that needed a new dependency had to:

git clone --depth=1 --branch vX.Y.Z https://github.com/OWNER/REPO.git /tmp/REPO

then add a replace directive in go.mod pointing at /tmp/REPO. This does not apply on a real machine with normal internet access — go.mod ships with real version requires and no replace directives; go mod tidy will just work. The go.mod.production file in each phase's tarball is already the clean version with no local paths.

Go version

Sandbox only had Go 1.22/1.23 available via apt; the user's real target is 1.25.6. Every phase's shippable go.mod declares go 1.25.6; all sandbox development/testing happened against 1.22 syntax (no version-specific features used, so this is a non-issue functionally) with the directive temporarily lowered for local builds, then restored before packaging. On the real 1.25.6 toolchain, nothing needs to change.

Sandbox environment quirks discovered (informational, not GoMail bugs)

• The sandbox's default shell is /bin/sh, not bash — brace expansion (mkdir -p {a,b}) silently creates a literal directory named {a,b} instead of two directories. Always use explicit bash -c or separate mkdir calls.
• The sandbox container has reset entirely mid-session at least once (Go toolchain, /tmp, and the whole working directory vanished) and resets /etc/hosts between every tool call. Backgrounded long-running processes (& + later kill) don't reliably survive across tool call boundaries either. The working pattern that survived all of this: write E2E tests as a single self-contained go run invocation that starts the server in-process (as a goroutine) and exercises it as a client in the same process — never rely on a detached background process still being alive in a later tool call.
• Because of the above, always keep a tarball of the last-known-good state in /mnt/user-data/outputs — that survived every reset and was the actual recovery mechanism used once.

Testing pattern used throughout

Every phase has (had, before cleanup) a cmd/e2etestN/main.go that:

1. Boots the real server(s) for that phase as goroutines in the same process
2. Acts as a real client over the real wire protocol (raw TCP for SMTP/IMAP/POP3, real net/http for DAV/webmail/JMAP)
3. Asserts against actual database state, not just protocol response codes
4. Is deleted after the phase's tests pass — not shipped in the tarballs, since it's sandbox-only scaffolding, not part of the product

When continuing this project, recreate this pattern per phase — it caught a genuine bug in nearly every phase (see "Bugs actually found" below) that a build-only check would have missed entirely.

Bugs actually found by this testing pattern (worth remembering the pattern, not the specific fixes)

• NULL-scan panics from sql.NullString-shaped columns not being wrapped correctly (Phase 2, Phase 3)
• A whole SMTP port (:465 implicit TLS) silently not receiving the outbound-relay code path because a check only looked for one of two equivalent session kinds (Phase 2/3)
• Foreign-key ordering: child rows inserted before their parent row existed (Phase 4)
• Regex . not matching embedded \r\n from IMAP literals — an entire class of FETCH responses silently parsed as empty (Phase 6)
• Wrong wire command entirely (FETCH UID 1 sent instead of UID FETCH 1) — server correctly returned nothing, client incorrectly assumed "not found" (Phase 6)
• Missing json struct tags meant to the wire API would have shipped TotalCount instead of total_count (Phase 8) — caught before it ever ran, not after

Takeaway for whoever continues this: don't trust "it compiles" or even "the happy-path test passed once." Write the negative-path tests (wrong auth, cross-user access, malformed input) every time — they found real bugs in every single phase.

---

File map (as of end of Phase 15 — the full plan)

gomail/
├── cmd/gomail/main.go              # wires everything together
├── go.mod / go.mod.production      # sandbox-dev vs real-deploy versions
├── install.sh                      # system user, directories, secrets, systemd install
├── gomail.service                  # hardened systemd unit — verified with systemd-analyze
├── README.md                       # quick start, TLS/ACME guidance, DNS setup guide
├── internal/
│   ├── accounts/                   # MailProvider interface + local/IMAP backends
│   ├── auth/                       # shared password + app-password verification
│   ├── config/                     # YAML + env config loader
│   ├── crypto/                     # HKDF-per-record AES-256-GCM encryption
│   ├── db/                         # schema, migrations, all queries
│   ├── dav/                        # CalDAV/CardDAV HTTP server
│   ├── dkim/                       # DKIM sign + verify (stdlib crypto)
│   ├── ical/                       # minimal RFC 5545 parser/builder
│   ├── imap/                       # IMAP server (core command set)
│   ├── imapclient/                 # hand-rolled IMAP client
│   ├── jmap/                       # JMAP Core/Mail subset
│   ├── acme/                       # ACME v2 client (RFC 8555) — JWS signing, full protocol flow
│   ├── admin/                      # admin portal REST API + embedded dark Tailwind SPA
│   ├── mailstore/                  # encrypted Maildir storage
│   ├── managesieve/                # RFC 5804 ManageSieve server
│   ├── oauth2/                     # hand-rolled OAuth2 client (auth code grant + refresh)
│   ├── pipeline/                   # SPF/DKIM/DMARC/header/URL security checks
│   ├── pop3/                       # POP3 server (off by default)
│   ├── ratelimit/                  # per-key token-bucket rate limiter (SMTP/IMAP/HTTP)
│   ├── sieve/                      # RFC 5228 Sieve interpreter (lexer/parser/exec)
│   ├── queue/                      # outbound retry queue + MX delivery
│   ├── smtp/                       # SMTP server (inbound MTA + submission)
│   ├── tlsutil/                    # self-signed cert generation + ACMEManager (SNI-aware serving, renewal)
│   ├── totp/                       # RFC 6238 TOTP MFA — verified against real published test vectors
│   ├── vcard/                      # minimal RFC 6350 parser/builder
│   ├── webmail/                    # REST API + embedded Tailwind SPA
│   └── webtoken/                   # hand-rolled JWT (HS256, plus scoped Purpose tokens for MFA/reset)

Native Gmail/Graph API providers (as opposed to the IMAP+OAuth2 path already built) are not yet in internal/accounts — see Phase 10's deferred-items list above. internal/dane, internal/mtasts, and internal/tlsrpt don't exist yet — see Phase 13's deferred-items list. No internal/webauthn (passkeys) — see Phase 12's deferred-items list; internal/totp exists and is fully wired. That's the complete list of genuinely unstarted packages as of the end of Phase 15 — every other phase in the original plan has at least a core-complete, tested implementation.