GoMail — Handover Document

For: continuing this project in Claude Code / Claude CLI on a real machine As of: end of Phase 15 (all 15 phases in the plan now touched — see the action plan for exact per-phase status)

If you're a fresh Claude instance reading this with no other context: this document plus gomail-action-plan-v4.md (in the same delivery) are everything you need. Read both before writing any code.

What this project is

GoMail — a complete self-hosted email server in a single Go binary, replacing postfix + dovecot + roundcube + spamassassin + radicale. Built from scratch across 9 completed phases, dependency-minimal (stdlib-first — almost everything is hand-rolled: SMTP, IMAP, POP3, JMAP, CalDAV/CardDAV, JWT, DKIM, SPF, DMARC parsers/servers/clients, all written against Go's standard library rather than third-party protocol libraries).

The person you're working with (their GitHub handle: ghostersk) wants a production-grade, secure, self-hosted mail server they fully understand and control — that's why everything is hand-rolled instead of using emersion/go-imap or similar: fewer dependencies, fewer supply-chain trust points, full auditability.

Immediate first steps on a real machine

# 1. Extract whichever phase tarball is newest (gomail-phase15.tar.gz as of
#    this handover — check /mnt/user-data/outputs or wherever it was saved)
tar xzf gomail-phase15.tar.gz
cd gomail

# 2. Use the CLEAN go.mod (no local replace directives — those were only
#    needed in the sandbox's network-restricted environment)
# go.mod.production is already correct; if a go.mod with /tmp/ replace
# directives exists instead, replace it:
cp go.mod.production go.mod  # only if go.mod still has replace directives

# 3. Normal Go workflow from here — real internet access means this just works
go mod tidy
go build ./cmd/gomail/

# 4. Confirm the Go version matches what's declared
go version   # should be able to satisfy "go 1.25.6" in go.mod

If go mod tidy fails on anything, it's almost certainly because the go.mod still has a stray replace (...) block from sandbox development — delete that block entirely and re-run go mod tidy.

For a real system install (not just a local build), use install.sh instead of running the binary by hand — it creates a dedicated system user, sets up /etc/gomail//var/lib/gomail//var/log/gomail with correct permissions, generates the required secrets, and installs gomail.service (a systemd unit with real hardening directives — verified with systemd-analyze verify in the sandbox this was built in). README.md has the full quick-start plus a DNS setup section (MX, SPF, DKIM, DMARC record examples) for whoever's actually standing this up.

How to verify the current state actually works

Don't trust that phase N's code is correct just because it's here — the sandbox proved things via disposable E2E test programs that get deleted after passing (see "Testing pattern" in the action plan doc). Recreate that habit:

# Generate a master key and JWT secret for local testing
go run ./cmd/gomail/ -gen-master-key   # prints GOMAIL_MASTER_KEY=...

export GOMAIL_MASTER_KEY=<paste from above>
export GOMAIL_JWT_SECRET=$(openssl rand -hex 32)
export GOMAIL_ADMIN_INIT_PASSWORD="ChangeMe123!"

# First run auto-generates gomail.yaml — edit server.hostname to something
# real before testing against actual mail flow, or leave as-is for
# loopback-only smoke testing.
go run ./cmd/gomail/ -config /tmp/test-gomail.yaml -debug

Then write a throwaway cmd/e2etestX/main.go that dials the relevant port(s) as a real client and asserts against real server responses and real database state — this is the pattern that caught a genuine bug in every single phase so far (see the bug list in the action plan). Delete the test program once it passes; it's not meant to ship.

What NOT to do

Don't add third-party protocol libraries (no emersion/go-imap, no emersion/go-smtp, no JWT libraries, no CalDAV/CardDAV libraries) unless explicitly asked. The entire point of this project, per the person's explicit instruction at the start, is minimal dependencies and full control. The only allowed exceptions so far: mattn/go-sqlite3 (CGO SQLite driver — no way around needing some driver), golang.org/x/crypto (bcrypt + hkdf — no stdlib equivalent), google/uuid, gopkg.in/yaml.v3.
Don't skip the negative-path tests. "It compiles" and "the happy path returned 200" are not sufficient — every phase's real bug was caught by testing auth failures, cross-user access attempts, malformed input, or literal/encoding edge cases.
Don't silently narrow scope. If something from the plan turns out to be too big for one pass (this happened repeatedly — DAV filtering, IMAP IDLE, JMAP Email/set), say so explicitly in code comments and in your response, the way every deferred item in the action plan is flagged. The person values honesty about gaps over an illusion of completeness.
Don't regenerate work that already exists and passed its tests. Read the action plan's "what's built" table first. Phases 1–9 are done; start at Phase 9.5 (ManageSieve, currently a full gap) or Phase 10 (OAuth2/Gmail/M365) per the person's stated priority.

Design decisions you should NOT re-litigate

These were explicitly decided across a long planning conversation before any code was written — don't re-ask or second-guess them:

SQLite via mattn/go-sqlite3 (CGO), not the pure-Go alternative — chosen for performance/maturity. Postgres/MySQL support planned via build tags (-tags postgres), not yet implemented.
Message storage: Maildir++ on disk, root path configurable, every message encrypted at rest (AES-256-GCM, HKDF-derived per-record key from a single master key — see internal/crypto). This applies to messages, contacts, and calendar events uniformly.
Multi-tenancy: all tenants share the same ports; tenants/domains are managed via the (not-yet-built) admin portal, not separate listeners.
Auth: strong password + MFA/passkey are the target for human login; app passwords (multiple, named, optional expiry) are for mail clients. Already fully working server-side (app_passwords table + auth.Authenticate's scope checking) — just needs a management UI.
Addressbooks/calendars: both per-user AND per-tenant (shared) scopes supported — see db.OwnerType (user | tenant).
CalDAV/CardDAV: in scope, built (Phase 7), core operations only.
Webmail: pure Go html/JS served via net/http + Tailwind CDN, no frontend build toolchain, no JS framework. Reference inspiration was github.com/ghostersk/gowebmail (the person's own earlier project) for UI patterns — dark theme, folder sidebar, message list/view, compose.
Push notifications: SSE only when the browser tab is open, no WebPush. Already implemented in Phase 8 (webmail.sseEvents).
POP3: included, off by default, admin-toggled in config (pop3.enabled: false). Built in Phase 5.
OAuth: self-hosted operators register their own Google/Microsoft apps (Cloud Console / Azure AD), paste Client ID/Secret into config — no shared GoMail-branded OAuth app. Config keys already exist (oauth.google/oauth.microsoft in config.go), unused until Phase 10.
JMAP external exposure: off by default (jmap.external_enabled: false), full spec compliance is still built either way since the webmail's internal use needs it eventually. Currently JMAP is mounted on the webmail's own address always (Phase 9); a second standalone listener binds when the config toggle is on.
Linked-account message caching (for Gmail/M365 accounts once Phase 10 lands): time-based retention, default 90 days, user-adjustable (days/months/years), operator ceiling via max_cache_retention in config. Schema (linked_accounts.cache_retention_days) already exists.
Go version: real target is 1.25.6. The sandbox this was built in could only get 1.22/1.23 via apt (no network path to the real Go toolchain distribution) — this is a sandbox limitation, not a project decision. On a real machine, just use 1.25.6 directly, no workaround needed.

Where the two-tier plan documents are

gomail-action-plan-v4.md — the authoritative current-state doc: phase-by-phase status, deferred items within completed phases, and full detail on Phases 9.5–15 (not yet started). Read this before writing any code — it tells you exactly what exists, what's deliberately incomplete, and why.
This document (HANDOVER.md) — onboarding for a fresh Claude instance: environment setup, testing philosophy, decisions not to re-litigate.

Earlier plan versions (v1 through v3, if you find them referenced anywhere) are superseded — v4 is authoritative for current state; v3 is still useful for the reasoning behind decisions that haven't changed (full Part A/B/C/D/E breakdown of the original Stalwart-inspired design), but don't treat its phase-completion claims as current.

Suggested next session's scope

All 15 phases from the original plan have now been worked through — 13 reached a fully complete, tested state; Phases 10 and 11 are "partially" or "core" complete with named remaining pieces. There is no phase left that hasn't been touched. What's left is: verifying the protocol-correct implementations against real external services, and finishing the named remaining pieces within otherwise-complete phases.

First thing next session — verify against real external services. Three separate pieces of this project were built and proven correct against genuinely-behaving fake servers (not rubber-stamp mocks — real protocol verification: real JWS signatures, real RFC 7638 thumbprints, real EICAR byte-level detection, etc.) but never against the real thing, because this sandbox has no public domain, no real OAuth app credentials, and no real ClamAV/rspamd/llama.cpp installed:

OAuth2 (Phase 10): register a real Google Cloud OAuth app (or ask the person for one) and link a real Gmail account end-to-end.
ACME (Phase 13): run one live issuance against Let's Encrypt staging (https://acme-staging-v02.api.letsencrypt.org/directory — set via tls.acme_directory_url; never test against production first) with a real domain you control.
ClamAV/Rspamd/LLM (Phase 14): if the operator has any of these installed, point pipeline.clamav_socket / pipeline.rspamd_url / pipeline.llm_url at them and send one real EICAR-string test message.

Remaining named pieces within already-complete phases (full detail in the action plan):

Phase 10: native Gmail/Graph API for calendar/contacts, webmail account-switcher/unified-inbox UI, real userinfo lookup instead of the ?email= stand-in at OAuth callback time.
Phase 11: aliases CRUD, per-tenant pipeline settings UI, live log viewer, and actually enforcing admin_ip_allowlist at the HTTP layer (currently inert).
Phase 13: DANE, MTA-STS, TLS-RPT, and DNS-01 challenge support are all fully unstarted, not just untested.
Phase 12: passkeys/WebAuthn are fully unstarted (needs CBOR decoding, not in Go's stdlib — a sub-project on the scale of Phase 13's JWS/ACME work), QR rendering for TOTP setup isn't built, no webmail SPA page yet.
Phase 14: Gmail API push / Microsoft Graph delta webhooks to replace IMAP polling — not started. No admin UI toggle for ClamAV/Rspamd/LLM — config-file only.
Phase 15: staticcheck and shellcheck couldn't run in this sandbox (toolchain/network restrictions, not a decision) — run both on a real machine, it's a quick win. No log rotation, metrics endpoint, or backup tooling for the SQLite database / Maildir tree.

Two bugs worth remembering as general patterns, both caught by actually testing rather than assuming code was correct:

A protocol test's fake server hung its client with what looked exactly like a real product bug — an i/o timeout. The actual cause: the fake server read an initial command with a generic buffered Read() into an oversized buffer, which silently over-read into the next protocol bytes (TCP doesn't preserve write-call boundaries), desyncing the whole parser downstream. Fixed with bufio.Reader + exact-length io.ReadFull. Applies to any future raw-TCP protocol work, product or test code alike.
A function that called db.Query then db.Exec in the same call genuinely deadlocked in production code (not a test), because this database is capped to a single open SQLite connection (SetMaxOpenConns(1)) and the Query's rows weren't closed before the Exec tried to grab that same connection. If you add a new function mixing Query and Exec, close rows explicitly first — don't rely on a deferred close.

Don't feel obligated to do any of this in the order listed if the person asks for something else first — this is a suggestion based on stated priority, not a hard requirement.

One more thing

This was built through genuinely iterative, tested development — not "write code, assume it works." Every phase had real bugs caught by real testing, documented honestly rather than glossed over. If you find yourself tempted to skip writing a test because "this part is simple" — don't. The simplest-looking parts of this codebase (a regex missing a DOTALL flag, a struct missing JSON tags, a boolean check covering only one of two equivalent cases) are exactly where the real bugs were.

14 KiB Raw Permalink Blame History Unescape Escape