mailgosend/PLAN.md

# mailgosend — Full-Stack Go Email Server: Project Plan

> Go 1.26.3 · Single binary · Dark TailwindCSS UI · Security-first

---

## 1. Goal

Self-hosted email server + webmail client in one executable.
Combines and supersedes `gomta` (outbound MTA) and `gowebmail` (IMAP webclient).

### What it runs

| Service | Default port | Protocol |
|---|---|---|
| SMTP inbound (MTA) | 25 | SMTP plain + STARTTLS |
| SMTP submission | 587 | SMTP + mandatory STARTTLS |
| SMTPS submission | 465 | SMTP implicit TLS |
| IMAP | 143 | IMAP + STARTTLS |
| IMAPS | 993 | IMAP implicit TLS |
| Web client | 8080 | HTTP (or TLS) |
| Web admin | 8081 | HTTP (or TLS) |
| CalDAV / CardDAV | 5232 | HTTP (or TLS) |

All ports, bind IPs, and TLS settings configurable in `app_config.conf`.

---

## 2. Design Constraints

- **Go stdlib only** — except packages listed in §4.
- **Zero CGO** — all deps must be pure Go.
- **Single executable** — all web assets embedded with `go:embed`.
- **Config auto-gen** — `app_config.conf` created on first run with secure random secrets.
- **Encryption at rest** — all email bodies, attachments, contacts, calendar events encrypted AES-256-GCM.
- **Input = malicious** — validate/sanitize every boundary.
- **No global mutable state**.
- **`context.Context` timeouts** on all I/O.
- **No frameworks** — `net/http` only for HTTP.
- **TailwindCSS via CDN**, vanilla JS, dark theme.

---

## 3. Directory Structure

```
mailgosend/
├── cmd/
│   └── mailgosend/
│       └── main.go               # Entrypoint — starts all servers, graceful shutdown
├── internal/
│   ├── config/
│   │   └── config.go             # app_config.conf load/autogen/validate
│   ├── db/
│   │   ├── db.go                 # database/sql wrapper, multi-driver support
│   │   ├── migrate.go            # Schema migrations (sequential versioned)
│   │   └── queries/              # Named SQL queries per domain
│   ├── crypto/
│   │   └── crypto.go             # AES-256-GCM encrypt/decrypt, key derivation, bcrypt
│   ├── auth/
│   │   ├── session.go            # Session store (DB-backed, signed tokens)
│   │   ├── brute.go              # Rate limiting + IP ban
│   │   └── totp.go               # TOTP / MFA (stdlib crypto)
│   ├── tls/
│   │   ├── tls.go                # TLS config builder, cert reload (SIGHUP), SNI routing
│   │   ├── acme.go               # ACME client: HTTP-01 + DNS-01 (lego), cert renewal loop
│   │   └── providers/            # DNS provider configs (Cloudflare, Route53, etc.)
│   ├── smtp/
│   │   ├── inbound.go            # Port 25 — receive mail for local domains
│   │   ├── submission.go         # Port 587/465 — authenticated send
│   │   ├── session.go            # SMTP session FSM (shared)
│   │   └── queue.go              # Delivery queue + retry scheduler
│   ├── imap/
│   │   ├── server.go             # IMAP server (go-imap/v2 backend interface)
│   │   └── backend.go            # Backend: maps IMAP ops to DB/storage layer
│   ├── delivery/
│   │   └── delivery.go           # Outbound MTA: DNS MX lookup, TLS dial, queue retry
│   ├── dkim/
│   │   └── dkim.go               # DKIM sign (outbound) + verify (inbound) — stdlib crypto
│   ├── spf/
│   │   └── spf.go                # SPF DNS lookup + policy check (stdlib net)
│   ├── dmarc/
│   │   └── dmarc.go              # DMARC policy fetch + alignment check
│   ├── spam/
│   │   └── spam.go               # Spam scorer: DNSBL, header heuristics, score threshold
│   ├── storage/
│   │   └── storage.go            # Message storage: DB metadata + encrypted body (file or DB blob)
│   ├── caldav/
│   │   ├── server.go             # CalDAV HTTP handler (go-webdav backend)
│   │   └── backend.go            # iCalendar storage backend
│   ├── carddav/
│   │   ├── server.go             # CardDAV HTTP handler
│   │   └── backend.go            # vCard storage backend
│   ├── models/
│   │   └── models.go             # Shared structs: User, Domain, Message, Folder, etc.
│   ├── middleware/
│   │   └── middleware.go         # Auth check, CSRF, rate limit, host check, logging
│   ├── webadmin/
│   │   ├── handlers.go           # Admin panel route handlers
│   │   └── api.go                # Admin JSON API
│   └── webclient/
│       ├── handlers.go           # Webmail route handlers
│       ├── compose.go            # Compose + send
│       └── api.go                # Client JSON API (messages, folders, contacts, calendar)
├── web/
│   ├── admin/
│   │   ├── templates/            # Admin HTML templates
│   │   └── static/               # Admin CSS/JS
│   └── client/
│       ├── templates/            # Webmail HTML templates
│       └── static/               # Client CSS/JS
├── webfs.go                      # go:embed declarations
├── go.mod
├── go.sum
├── app_config.conf               # Auto-generated, never committed
├── .gitignore
└── PLAN.md                       # This file
```

---

## 4. Third-Party Dependencies

Only where stdlib has no equivalent. All actively maintained.

| Package | Why | Stars / Maintainer |
|---|---|---|
| `modernc.org/sqlite` | Pure-Go SQLite, no CGO | ~2k / cznic |
| `golang.org/x/crypto` | bcrypt (passwords), low-level ACME protocol | golang.org |
| `golang.org/x/text` | Charset decoding (email MIME) | golang.org |
| `golang.org/x/oauth2` | Google + Microsoft OAuth2 for external accounts | golang.org |
| `github.com/go-acme/lego/v4` | ACME client: HTTP-01 **and** DNS-01 challenges, wildcard certs, 90+ DNS providers (Cloudflare, Route53, etc.) | ~8k / go-acme |
| `github.com/emersion/go-imap/v2` | IMAP server + client protocol (RFC 3501 is 300+ pages) | ~2k / Simon Ser |
| `github.com/emersion/go-smtp` | SMTP protocol library (RFC 5321 state machine) | ~800 / Simon Ser |
| `github.com/emersion/go-message` | Full MIME email parsing (RFC 2045-2049) | ~400 / Simon Ser |
| `github.com/emersion/go-sasl` | SASL auth mechanisms (PLAIN, LOGIN, XOAUTH2) | transitive |
| `github.com/emersion/go-webdav` | CalDAV + CardDAV server framework (RFC 4791/6352) | ~600 / Simon Ser |

**Optional DB drivers** (user selects in config, only linked if needed via build tags):

| Driver | Package |
|---|---|
| PostgreSQL | `github.com/lib/pq` |
| MySQL/MariaDB | `github.com/go-sql-driver/mysql` |
| MSSQL | `github.com/microsoft/go-mssqldb` |

> Note: `mattn/go-sqlite3` (used in old projects) requires CGO — replaced by `modernc.org/sqlite`.
> `github.com/gorilla/mux` (used in gowebmail) dropped — `net/http` `ServeMux` (Go 1.22+) supports method+path routing.

---

## 5. Configuration (`app_config.conf`)

Auto-generated on first run if missing. INI-style `KEY = value` with comments.
All secrets (encryption key, session secret) are random hex on first gen — never default insecure values.

### Config sections

```ini
# --- Server Identity ---
HOSTNAME         = mail.example.com   # FQDN for SMTP HELO/EHLO, TLS SNI, URLs
DOMAIN_DEFAULT   = example.com        # Primary mail domain

# --- Network Ports & Interfaces ---
SMTP_IFACE       = 0.0.0.0
SMTP_PORT        = 25
SUBMISSION_IFACE = 0.0.0.0
SUBMISSION_PORT  = 587
SMTPS_PORT       = 465
IMAP_IFACE       = 0.0.0.0
IMAP_PORT        = 143
IMAPS_PORT       = 993
WEBCLIENT_IFACE  = 0.0.0.0
WEBCLIENT_PORT   = 8080
WEBADMIN_IFACE   = 127.0.0.1         # Admin on loopback by default
WEBADMIN_PORT    = 8081
CALDAV_IFACE     = 0.0.0.0
CALDAV_PORT      = 5232

# --- TLS ---
# TLS_MODE options:
#   dns01     = Let's Encrypt via DNS-01 challenge (no open ports, supports wildcards)
#               Requires ACME_DNS_PROVIDER + provider credentials below.
#               Works behind NAT/firewall. RECOMMENDED for self-hosted.
#   http01    = Let's Encrypt via HTTP-01 challenge (requires port 80 reachable from internet)
#               No DNS API needed. No wildcard support.
#   manual    = Provide TLS_CERT + TLS_KEY paths (certbot, acme.sh, self-signed, Vault, etc.)
#               SIGHUP reloads certs without restart.
#   off       = No TLS. Use ONLY behind a TLS-terminating reverse proxy.
TLS_MODE         = dns01

# ACME (Let's Encrypt) — used for dns01 and http01 modes
ACME_EMAIL       = admin@example.com  # Required for Let's Encrypt ToS
ACME_CACHE_DIR   = ./acme-cache       # Cache dir for certs, keys, account
ACME_STAGING     = false              # true = use Let's Encrypt staging (for testing)
ACME_DOMAINS     = example.com,*.example.com   # Domains to certify; blank = HOSTNAME only
                                               # Include wildcard for *.example.com coverage

# DNS-01 provider (required when TLS_MODE=dns01)
# Supported providers: cloudflare | route53 | digitalocean | dnsimple | godaddy |
#   namecheap | linode | hetzner | ovh | gandi | porkbun | desec | acmedns | + 80 more
#   Full list: https://go-acme.github.io/lego/dns/
ACME_DNS_PROVIDER = cloudflare

# Cloudflare credentials (ACME_DNS_PROVIDER=cloudflare)
# Option A — API Token (recommended, least privilege: Zone.DNS edit on target zones)
CF_DNS_API_TOKEN  =
# Option B — Global API Key
CF_API_KEY        =
CF_API_EMAIL      =

# AWS Route53 credentials (ACME_DNS_PROVIDER=route53)
AWS_REGION        =
AWS_ACCESS_KEY_ID =
AWS_SECRET_ACCESS_KEY =
AWS_HOSTED_ZONE_ID =      # Optional: skip auto-detection

# DigitalOcean (ACME_DNS_PROVIDER=digitalocean)
DO_AUTH_TOKEN     =

# Generic: any provider using environment variables — lego reads standard env vars
# for each provider. Set them here or export before starting mailgosend.
# See https://go-acme.github.io/lego/dns/<provider>/ for required vars.

# Manual cert paths (TLS_MODE=manual, or per-service override below)
TLS_CERT          = ./certs/cert.pem
TLS_KEY           = ./certs/key.pem

# Per-service cert override — leave blank to inherit global TLS_MODE
# Useful when using wildcard autocert globally but a specific cert for admin
SMTP_TLS_CERT     =
SMTP_TLS_KEY      =
IMAP_TLS_CERT     =
IMAP_TLS_KEY      =
WEB_TLS_CERT      =
WEB_TLS_KEY       =

# --- Secrets (auto-generated, BACK THESE UP) ---
ENCRYPTION_KEY   = <64 hex chars>     # AES-256 master key — email content at rest
SESSION_SECRET   = <64 hex chars>     # Session cookie signing
DKIM_SELECTOR    = mail               # Default DKIM selector

# --- Database ---
DB_DRIVER        = sqlite             # sqlite | postgres | mysql | mssql
DB_PATH          = ./data/mail.db     # SQLite path
DB_DSN           =                    # PostgreSQL/MySQL/MSSQL connection string

# --- Security ---
MAX_MESSAGE_SIZE = 52428800           # 50 MB
SESSION_MAX_AGE  = 604800             # 7 days
BRUTE_MAX_TRIES  = 5
BRUTE_WINDOW_MIN = 30
BRUTE_BAN_HOURS  = 24
TRUSTED_PROXIES  =                    # CIDR list for X-Forwarded-For
SECURE_COOKIE    = false              # true when behind TLS

# --- Spam ---
SPAM_THRESHOLD   = 10                 # Messages scoring >= this → Spam folder
SPAM_DNSBL       = zen.spamhaus.org,bl.spamcop.net
SPAM_CHECK_SPF   = true
SPAM_CHECK_DKIM  = true
SPAM_CHECK_DMARC = true

# --- Delivery ---
QUEUE_MAX_AGE_H  = 72                 # Bounce after 72h
QUEUE_RETRY_MIN  = 5,15,60,240,480   # Retry backoff in minutes
DNS_PRIMARY      = 1.1.1.1
DNS_SECONDARY    = 8.8.8.8

# --- Debug ---
DEBUG            = false
LOG_FILE         = ./logs/mail.log
LOG_LEVEL        = info               # debug | info | warn | error
```

---

## 6. Database Schema

### Tables

```sql
-- System
config           (key, value, updated_at)
migrations       (version, applied_at)

-- Domains & Users
domains          (id, name, enabled, dkim_private_enc, dkim_public, dkim_selector,
                  spf_policy, dmarc_policy, created_at)
users            (id, domain_id, username, email, password_hash, display_name,
                  quota_bytes, enabled, admin, mfa_secret_enc, created_at, last_login)
user_sessions    (id, user_id, token_hash, ip, user_agent, created_at, expires_at)
user_aliases     (id, user_id, alias_email)

-- Mail storage
mailboxes        (id, user_id, name, type, parent_id, uid_validity, uid_next,
                  subscribed, created_at)
-- type: inbox|sent|drafts|trash|spam|archive|custom

messages         (id, mailbox_id, uid, remote_uid, message_id, subject,
                  from_email, from_name, to_list, cc_list, bcc_list, reply_to,
                  date, body_text_enc, body_html_enc, raw_enc,
                  size_bytes, has_attachment, is_read, is_starred, is_draft,
                  flags, spam_score, received_at, deleted_at)
-- All *_enc columns are AES-256-GCM encrypted blobs

attachments      (id, message_id, filename, content_type, size_bytes, data_enc,
                  content_id, inline)

-- Delivery queue
queue            (id, domain_id, from_addr, to_addr, raw_enc, message_id,
                  status, attempts, last_attempt, next_attempt, error_log,
                  created_at, expires_at)
-- status: pending|sending|failed|bounced|delivered

delivery_log     (id, queue_id, from_addr, to_addr, status, smtp_code,
                  smtp_message, mx_host, client_ip, created_at)

-- Security
ip_bans          (id, ip, reason, banned_at, expires_at, released_by)
login_attempts   (id, ip, user_email, success, created_at)
security_events  (id, type, ip, user_id, detail, created_at)

-- Contacts (CardDAV)
addressbooks     (id, user_id, name, description, color, sync_token)
contacts         (id, addressbook_id, uid, vcard_enc, etag, created_at, updated_at)

-- Calendar (CalDAV)
calendars        (id, user_id, name, description, color, timezone, sync_token)
calendar_events  (id, calendar_id, uid, ical_enc, etag, dtstart, dtend,
                  summary, recurring, created_at, updated_at)
```

---

## 7. Core Modules — Detail

### 7.1 Config (`internal/config/`)

- Read `app_config.conf` (INI format, `#` comments)
- Missing file → generate with random secrets + safe defaults
- Missing keys → append to existing file (non-destructive update)
- Env var `MAILGO_<KEY>` overrides file value
- Startup validation: required fields, port ranges, key lengths

### 7.2 Database (`internal/db/`)

- `database/sql` interface — driver loaded by config
- Build tags select driver: `sqlite`, `postgres`, `mysql`, `mssql`
- Sequential migrations (integer version in `migrations` table)
- Prepared statements everywhere — no `fmt.Sprintf` into SQL
- Connection pool config (max open, idle, lifetime)
- `context.Context` on all queries with configurable timeout

### 7.3 Crypto (`internal/crypto/`)

- Master key: 32-byte AES-256 from config (hex)
- Per-user key: HKDF-SHA256(master\_key, user\_id+salt)
- Encrypt: AES-256-GCM, random nonce prepended to ciphertext
- Decrypt: split nonce, decrypt, authenticate
- Password hash: bcrypt cost 12 minimum
- Session tokens: `crypto/rand` 32 bytes, stored as SHA-256 hash in DB
- DKIM keys: RSA-2048 or Ed25519 (configurable)

### 7.4 SMTP Inbound (`internal/smtp/inbound.go`)

Port 25 — receives mail from the internet for local domains.

**Flow:**
1. Accept TCP connection
2. Greet: `220 hostname ESMTP mailgosend`
3. EHLO → advertise: `STARTTLS`, `SIZE <max>`, `8BITMIME`, `SMTPUTF8`
4. STARTTLS (optional but offered)
5. `MAIL FROM` — validate format, check SPF
6. `RCPT TO` — validate recipient exists in local domain
7. `DATA` — accept message
8. Post-receive pipeline:
   - Parse headers + MIME
   - Verify DKIM signature
   - Check DMARC alignment
   - Score spam (DNSBL + heuristics)
   - Encrypt body + attachments
   - Store in recipient's mailbox
   - Emit IMAP push notification (IDLE clients)
9. `250 OK` or appropriate error code

**Rejected on arrival:** open relay (no local recipient), size exceeded, malformed envelope.

### 7.5 SMTP Submission (`internal/smtp/submission.go`)

Ports 587 (STARTTLS mandatory) and 465 (implicit TLS).

**Differences from inbound:**
- STARTTLS mandatory before AUTH on 587 (reject AUTH without TLS)
- AUTH PLAIN + AUTH LOGIN + AUTH XOAUTH2
- Authenticated user must match `MAIL FROM` domain
- No relay to arbitrary domains unless relay rules configured
- DKIM-sign outbound messages
- Add `Received:` header
- Queue for async delivery

### 7.6 SMTP Session FSM (`internal/smtp/session.go`)

Shared state machine (reused by both inbound and submission):

```
States: Greeting → Helo → (Auth) → Mail → Rcpt → Data → Done
```

- Command timeout: 5 min per command (configurable)
- Max recipients: 100 (configurable)
- Pipeline support (ESMTP PIPELINING extension)
- Idle timeout: 10 min

### 7.7 Delivery Queue (`internal/smtp/queue.go` + `internal/delivery/`)

**Queue flow:**
1. Message inserted into `queue` table (status=pending, raw body encrypted)
2. Background goroutine polls queue every 30s
3. For each pending/retry-eligible row:
   - DNS MX lookup for recipient domain
   - Attempt SMTP delivery with TLS (STARTTLS preferred, fallback plain)
   - On success: status=delivered, log entry
   - On soft failure (4xx): increment attempts, set next_attempt (backoff)
   - On hard failure (5xx): status=failed, send bounce to sender
   - On timeout (>queue_max_age): bounce
4. Bounce messages: generated RFC 3464 DSN, stored in sender's mailbox

### 7.8 IMAP Server (`internal/imap/`)

Uses `github.com/emersion/go-imap/v2` server package.
Implement the `backend.Backend` interface backed by our DB/storage layer.

**Supported capabilities:**
- `IMAP4rev1` (RFC 3501)
- `UIDPLUS`
- `IDLE` (push notification when new mail arrives)
- `NAMESPACE`
- `QUOTA`
- `MOVE`
- `SPECIAL-USE`
- `AUTH=PLAIN` + `AUTH=LOGIN`
- `STARTTLS` (port 143), implicit TLS (port 993)

**Backend operations map to:**
- `Login` → verify user credentials against DB
- `ListMailboxes` → query `mailboxes` table
- `SelectMailbox` → load mailbox metadata, update `uid_validity`
- `FetchMessages` → query + decrypt messages
- `StoreFlags` → update `is_read`, `is_starred`, `flags`
- `AppendMessage` → encrypt + insert into `messages`
- `CopyMessages` / `MoveMessages` → DB operations
- `ExpungeMessages` → soft-delete (set `deleted_at`)

### 7.9 DKIM (`internal/dkim/`)

Ported + improved from `gomta`:
- Sign outbound: `rsa-sha256`, `relaxed/relaxed` canonicalization
- Verify inbound: fetch public key via DNS TXT, validate signature
- Per-domain keys stored encrypted in `domains.dkim_private_enc`
- Key generation: RSA-2048 minimum, Ed25519 optional
- DNS record helper: prints ready-to-paste TXT record

### 7.10 SPF (`internal/spf/`)

From scratch using `net` stdlib:
- Parse `SPF1` TXT records from sender domain DNS
- Walk mechanisms: `include`, `a`, `mx`, `ip4`, `ip6`, `all`
- Return `pass`, `fail`, `softfail`, `neutral`, `permerror`, `temperror`
- DNS lookup limit: 10 (per RFC 7208)
- Result fed into spam scorer

### 7.11 DMARC (`internal/dmarc/`)

From scratch:
- Fetch `_dmarc.<domain>` TXT record
- Parse policy (`p=none|quarantine|reject`)
- Check DKIM + SPF alignment (strict/relaxed)
- Apply policy to inbound messages
- Log aggregate data (future: reporting)

### 7.12 Spam Scorer (`internal/spam/`)

Rule-based, no ML dependencies:

| Check | Max points |
|---|---|
| DNSBL hit (zen.spamhaus.org etc.) | +5 per list |
| SPF fail | +4 |
| SPF softfail | +2 |
| DKIM fail / missing | +3 |
| DMARC reject policy violated | +5 |
| Missing `Date` header | +1 |
| Missing `Message-ID` | +1 |
| Suspicious `From` (forged display name) | +2 |
| HTML only (no text/plain) | +1 |
| Excessive recipients | +2 |
| All-caps subject | +1 |

Score >= `SPAM_THRESHOLD` (default 10) → deliver to Spam folder + set flag.

### 7.13 Storage (`internal/storage/`)

- Message raw RFC822 + parsed parts stored encrypted in DB (BLOB)
- Attachments stored as separate encrypted blobs (in DB or filesystem, configurable)
- Encryption: AES-256-GCM with per-user derived key
- Quota enforcement: check `users.quota_bytes` before storing

### 7.14 CalDAV (`internal/caldav/`)

Uses `github.com/emersion/go-webdav` caldav package.
Implement `caldav.Backend` interface:
- `GetCalendar` → query `calendars` table
- `ListCalendars` → all user calendars
- `GetCalendarObject` → decrypt + return iCal
- `PutCalendarObject` → encrypt + upsert event
- `DeleteCalendarObject` → delete event
- Sync token: monotonic counter per calendar (for `calendar-sync` report)

Exposed at `/dav/calendars/<user>/` on CalDAV port.
Supports iOS/Android/Thunderbird/Evolution native calendar sync.

### 7.15 CardDAV (`internal/carddav/`)

Uses `github.com/emersion/go-webdav` carddav package.
- `GetAddressBook` / `ListAddressBooks`
- `GetAddressObject` → decrypt + return vCard
- `PutAddressObject` → encrypt + upsert contact
- Supports iOS/Android contacts sync

Exposed at `/dav/contacts/<user>/` on same CalDAV port.

### 7.16 Web Admin (`internal/webadmin/`)

Accessible on `WEBADMIN_PORT` (default 8081, loopback only by default).

**Pages / features:**
- Dashboard: server stats, queue depth, delivery rate, active connections
- Domains: add/remove/enable, view DKIM DNS records (copy-paste ready), SPF guidance
- Users: create/edit/delete/quota, reset password, enable MFA
- Queue: view pending/failed messages, retry manually, cancel
- Delivery logs: searchable, filterable by status/domain/date
- Security: IP ban list, login attempt log, security events
- Settings: edit config keys (except secrets), reload config without restart
- TLS: cert upload or trigger ACME renewal
- System: log viewer (tail), version info

### 7.17 Web Client (`internal/webclient/`)

Accessible on `WEBCLIENT_PORT` (default 8080).

**Pages / features:**
- Login (+ MFA if enabled)
- Multi-account unified inbox (all local accounts a user has access to, plus external IMAP)
- Folder tree sidebar (all mailboxes, unread counts)
- Message list (virtual scroll, infinite load, search)
- Message view (HTML with sandbox iframe, plain text fallback, inline images, attachment download)
- Compose (rich editor, attachments, To/CC/BCC autocomplete from contacts)
- Contacts view (CardDAV-backed)
- Calendar view (month/week/day, CalDAV-backed)
- Account settings: password change, MFA setup, aliases, display name
- External IMAP accounts: add Gmail/Outlook/custom IMAP servers (like gowebmail)
- Dark theme, responsive, TailwindCSS CDN

---

## 8. Security Architecture

### Authentication

- Passwords: bcrypt cost 12, never stored plain
- Sessions: random 32-byte token → SHA-256 stored in DB
- Session cookie: HttpOnly, SameSite=Strict, Secure (when TLS)
- MFA: TOTP (RFC 6238), QR code at setup, recovery codes
- SMTP auth: PLAIN + LOGIN (both over TLS only)
- Admin panel: separate session store, optional IP allowlist

### Transport Security

- SMTP port 25: STARTTLS offered, not mandatory (internet compatibility)
- SMTP port 587: STARTTLS mandatory before AUTH
- SMTP port 465: implicit TLS
- IMAP port 143: STARTTLS mandatory before AUTH
- IMAP port 993: implicit TLS
- Web ports: TLS via cert file or Let's Encrypt autocert
- Min TLS: 1.2, prefer 1.3

### Input Validation

- Every HTTP handler validates: Content-Type, body size, all fields
- Email addresses: RFC 5321 parse (`net/mail`)
- SQL: prepared statements only
- HTML output: `html/template` auto-escaping
- File uploads: MIME type + magic bytes check, size limit, path traversal prevention
- CSRF: double-submit cookie pattern on all state-changing endpoints
- Host header check: reject unexpected `Host:` values

### Rate Limiting

- SMTP: per-IP connection limit, per-session command rate
- IMAP: per-IP login rate
- HTTP login: brute force counter (configurable window/threshold/ban)
- HTTP API: per-user request rate (sliding window, in-memory + DB fallback)
- Geo blocking: optional deny/allow country lists

### Data Security

- Email bodies: AES-256-GCM, per-user derived key (HKDF)
- DKIM private keys: encrypted in DB
- OAuth tokens (external accounts): encrypted in DB
- MFA secrets: encrypted in DB
- Logs: never log passwords, tokens, or email content
- Bounce messages: no sensitive headers in DSN body

---

## 9. Build System

### Single binary

```bash
go build -o mailgosend ./cmd/mailgosend/
```

### Embedded assets

`webfs.go`:
```go
//go:embed web/admin web/client
var webFS embed.FS
```

### Build tags for DB drivers

```bash
# SQLite only (default)
go build -tags sqlite ./cmd/mailgosend/

# With PostgreSQL support
go build -tags sqlite,postgres ./cmd/mailgosend/

# All drivers
go build -tags sqlite,postgres,mysql,mssql ./cmd/mailgosend/
```

### Dockerfile (optional)

```dockerfile
FROM golang:1.26-alpine AS builder
WORKDIR /app
COPY . .
RUN go build -tags sqlite -o mailgosend ./cmd/mailgosend/

FROM alpine:latest
RUN apk add --no-cache ca-certificates tzdata
COPY --from=builder /app/mailgosend /usr/local/bin/
EXPOSE 25 465 587 143 993 8080 8081 5232
CMD ["mailgosend"]
```

---

## 10. Implementation Phases

### Phase 1 — Foundation (Week 1)
- [ ] `go.mod` with all deps
- [ ] `internal/config/` — config file load/autogen/validate
- [ ] `internal/db/` — DB wrapper, SQLite driver, migrations
- [ ] `internal/crypto/` — AES-256-GCM, bcrypt, HKDF
- [ ] `internal/models/` — all shared structs
- [ ] `internal/auth/session.go` — session store
- [ ] `internal/tls/` — TLS config builder
- [ ] `cmd/mailgosend/main.go` — startup scaffold, graceful shutdown

### Phase 2 — SMTP + Delivery (Week 2)
- [ ] `internal/smtp/session.go` — FSM + command parser
- [ ] `internal/smtp/inbound.go` — port 25 server
- [ ] `internal/smtp/submission.go` — ports 587/465
- [ ] `internal/smtp/queue.go` — queue table + worker
- [ ] `internal/delivery/delivery.go` — MX lookup + outbound SMTP
- [ ] `internal/dkim/dkim.go` — sign + verify
- [ ] `internal/spf/spf.go` — SPF check
- [ ] `internal/dmarc/dmarc.go` — DMARC check
- [ ] `internal/spam/spam.go` — scoring engine

### Phase 3 — IMAP Server (Week 3)
- [ ] `internal/imap/backend.go` — go-imap/v2 backend
- [ ] `internal/imap/server.go` — IMAP + IMAPS listeners
- [ ] `internal/storage/storage.go` — encrypted message storage
- [ ] IMAP IDLE support (push)
- [ ] Quota enforcement

### Phase 4 — Web Admin (Week 4)
- [ ] `internal/webadmin/handlers.go`
- [ ] `internal/auth/brute.go` — brute force protection
- [ ] `internal/middleware/middleware.go` — auth, CSRF, logging
- [ ] All admin templates + TailwindCSS dark theme
- [ ] `web/admin/static/js/admin.js`

### Phase 5 — Web Client (Week 5)
- [ ] `internal/webclient/handlers.go` — message list, view, compose
- [ ] `internal/webclient/api.go` — JSON endpoints
- [ ] Multi-account support (local + external IMAP)
- [ ] All client templates + TailwindCSS dark theme
- [ ] `web/client/static/js/client.js` — virtual scroll, compose editor

### Phase 6 — CalDAV / CardDAV (Week 6)
- [ ] `internal/caldav/` — calendar CRUD + sync
- [ ] `internal/carddav/` — contacts CRUD + sync
- [ ] Calendar UI in webclient (month view)
- [ ] Contacts UI in webclient

### Phase 7 — Polish & Hardening (Week 7)
- [ ] Let's Encrypt autocert integration
- [ ] MFA (TOTP) for web login
- [ ] Delivery log + admin queue management
- [ ] Bounce message generation (RFC 3464 DSN)
- [ ] Import: migrate from gowebmail external accounts
- [ ] End-to-end testing (SMTP → IMAP → webmail round-trip)
- [ ] Security audit pass

---

## 11. Key Differences from Existing Projects

| Feature | gomta | gowebmail | mailgosend |
|---|---|---|---|
| SMTP inbound | No | No | Yes |
| SMTP outbound | Yes | Relay only | Yes (full MTA) |
| IMAP server | No | No | Yes |
| IMAP client | No | Yes | Yes (external accounts) |
| Web client | No | Yes | Yes (improved) |
| Admin panel | Basic | Basic | Full |
| CalDAV | No | Basic | Full RFC |
| CardDAV | No | Basic | Full RFC |
| DKIM verify | No | No | Yes |
| SPF check | No | No | Yes |
| DMARC | No | No | Yes |
| Spam scoring | No | No | Yes |
| Encryption at rest | No | Yes (tokens only) | Yes (all content) |
| Pure Go (no CGO) | No (go-sqlite3) | No (go-sqlite3) | Yes (modernc) |
| Multi-DB support | No | No | Yes |
| Single binary | No | No | Yes |
| Let's Encrypt | No | No | Yes |
| MFA | No | No | Yes |

---

## 12. Design Decisions (Resolved)

| # | Question | Decision |
|---|---|---|
| 1 | External IMAP accounts | **Yes** — local accounts + external Gmail/Outlook/custom IMAP (OAuth2 + credentials). Reuse gowebmail approach. |
| 2 | Storage backend | **Configurable** — DB blob default, filesystem optional. Config switch `STORAGE_BACKEND = db \| fs`. |
| 3 | Admin model | **Global admin + per-domain admin** — domain admin can manage users/settings for their domain only. |
| 4 | Spam filter | **Rules + Bayesian** — static scoring (SPF/DKIM/DMARC/DNSBL) + per-user Bayesian token training from "mark as spam/not spam". |
| 5 | TLS | **DNS-01 default** (lego) — wildcard certs, no ports exposed, Cloudflare/Route53/90+ providers. `http01` for simple setups. `manual` for self-managed certs. Per-service cert override. SIGHUP hot-reload. |
| 6 | DB drivers | **Build tags** — default binary SQLite only. `-tags postgres/mysql/mssql` to add drivers. |
| 7 | DKIM keys | **RSA-2048 default + Ed25519 option** — configurable per domain (`DKIM_ALGO = rsa2048 \| ed25519`). |
| 8 | Queue | **DB-backed** — survives crash. Rows in `queue` table with status/retry/backoff tracking. |

### Additional dependency: OAuth2

External account support requires:
- `golang.org/x/oauth2` — Google + Microsoft OAuth2 flows (same as gowebmail)
- Google: Gmail scope `https://mail.google.com/`
- Microsoft: IMAP.AccessAsUser.All, SMTP.Send, offline_access

---

## 13. TLS Architecture Detail

### Challenge modes

```
TLS_MODE = dns01   →  github.com/go-acme/lego/v4
                        DNS-01 ACME challenge via DNS provider API
                        ✓ No port 80/443 needed — works behind NAT/firewall
                        ✓ Wildcard certificates (*.example.com)
                        ✓ Server can be completely non-public (LAN-only)
                        ✓ 90+ DNS providers (Cloudflare, Route53, DO, etc.)
                        Renewal: automatic background goroutine, 30 days before expiry

TLS_MODE = http01  →  github.com/go-acme/lego/v4
                        HTTP-01 challenge: lego spins up ephemeral listener on port 80
                        ✓ No DNS API credentials needed
                        ✗ Port 80 must be reachable from internet
                        ✗ No wildcard support
                        Renewal: same automatic background goroutine

TLS_MODE = manual  →  Load TLS_CERT + TLS_KEY at startup
                        SIGHUP signal triggers cert reload without restart
                        Compatible with: certbot, acme.sh, Vault PKI, self-signed
                        Use when: internal CA, corporate cert, or existing automation
```

### Wildcard flow (recommended, Cloudflare example)

```
startup → lego checks ACME_CACHE_DIR for cached cert
        → if missing or expiring: start DNS-01 challenge
            → lego calls Cloudflare API: add TXT _acme-challenge.example.com
            → Let's Encrypt verifies TXT record
            → lego calls Cloudflare API: remove TXT record
            → cert issued for: example.com + *.example.com
            → cached in ACME_CACHE_DIR/certificates/
        → build *tls.Config from cert
        → all listeners (SMTP/IMAP/web) share same *tls.Config (or per-service)
renewal → background goroutine checks cert expiry every 12h
        → renews 30 days before expiry, same DNS-01 flow
        → atomic swap of *tls.Config — zero downtime, no restart
```

### Per-service cert override

```
Priority (highest to lowest):
  SMTP_TLS_CERT/KEY  →  used for SMTP + SMTPS
  IMAP_TLS_CERT/KEY  →  used for IMAP + IMAPS
  WEB_TLS_CERT/KEY   →  used for webclient + admin + CalDAV ports
  TLS_CERT/KEY       →  global fallback (TLS_MODE=manual)
  ACME cert          →  global fallback (TLS_MODE=dns01 or http01)
  off                →  no TLS on that service
```

### Minimum security

- TLS 1.2 minimum on all listeners
- TLS 1.3 preferred
- Cipher suites: ECDHE + AES-256-GCM, CHACHA20-POLY1305 only
- HSTS header on all HTTP responses (when TLS active)
- SMTP: `220` banner does not reveal software version

---

*Plan created: 2026-05-21*
*Author: nahakubuilder*
*Last updated: 2026-05-21 — all design decisions resolved*