gomail/.claude/iterative-build-discipline-SKILL.md

name, description

name

description

iterative-build-discipline

Practices for building large, multi-phase software projects (servers, protocol implementations, hand-rolled clients, multi-session builds) where correctness and honest progress tracking matter more than speed. Use this whenever a project spans multiple sessions or phases, involves implementing a network protocol or parser from scratch, needs to prove correctness without access to the real external service it talks to, or risks losing state between sessions (sandbox resets, context limits, long-running builds). Also use when maintaining living planning/handover documents across many edits, or when deciding whether to test something before believing it works. Trigger this proactively for phased builds even if the user doesn't ask for "best practices" — the value is in not needing to be asked.

Iterative Build Discipline

Distilled from building GoMail, a ~14,000-line self-hosted email server implemented phase-by-phase across many sessions (SMTP/IMAP/JMAP/CalDAV/ ManageSieve/OAuth2/ACME, all hand-rolled on stdlib). The patterns below caught real bugs and prevented real losses, repeatedly, across nearly every phase. This isn't theoretical advice — every rule here has a "here's the actual bug this caught" behind it.

Core principle

"It compiles" and "the happy path returned 200" are not evidence of correctness. Every phase of GoMail had at least one real bug that a build-only check would have missed entirely and shipped silently. The single highest-leverage habit is: after writing a chunk of new functionality, write a real test that exercises it as a real client would, including the paths where it should fail. Do this before declaring the phase done, not as an afterthought.

Testing philosophy

Write disposable, real end-to-end tests — not unit-test theater

For each meaningful chunk of work, write a throwaway test program that:

1. Boots the real server(s) as goroutines/processes in the same test run
2. Acts as a real client over the real wire protocol — raw TCP for custom protocols, real net/http for HTTP, not an in-process function call that skips the actual serialization/parsing
3. Asserts against real downstream state (a database row, a file on disk), not just a status code
4. Covers negative paths deliberately: wrong auth, malformed input, cross-user/cross-tenant access attempts, replay of a one-time token, unreachable dependency
5. Gets deleted once it passes — it was scaffolding, not the deliverable

This pattern found a real bug in nearly every phase of a 15-phase project: a NULL-scan panic from a naively-wrapped nullable column, a boolean check that covered only one of two equivalent session kinds (silently dropping an entire code path for one server mode), a regex missing the (?s) DOTALL flag that made an entire class of multi-line responses parse as empty, a wire command sent with the wrong syntax that "worked" by returning an empty result instead of an error, a struct with no JSON tags that would have silently shipped TotalCount instead of total_count on the wire, a SQLite connection-pool deadlock from calling Query then Exec in the same function without closing rows first, and a fake protocol test server whose oversized buffered Read() silently over-read into subsequent protocol bytes and desynced its own parser (looked exactly like a real product bug until traced).

None of these would have been caught by "the code compiles and looks right." All of them were caught by writing a client that actually spoke the protocol and checking what actually happened downstream.

When you can't test against the real external service, build a fake

one that actually enforces the protocol — not a rubber stamp For OAuth2, ACME, ClamAV, rspamd, and an LLM server, there was no real instance available in the build environment (no real Google/Microsoft app credentials, no public domain to satisfy Let's Encrypt's HTTP-01 validator, no ClamAV/rspamd/llama.cpp installed). The fix was never to skip testing — it was to build a from-scratch fake server that genuinely implements the documented protocol, then prove the real client code against it:

• A fake ACME server that does real ES256 JWS signature verification and real RFC 7638 JWK thumbprint recomputation, and only issues a certificate if a real HTTP callback to the challenge responder actually succeeds — not a server that just returns "OK" to whatever it's sent.
• A fake ClamAV that reassembles the streamed chunks and inspects the bytes for the real EICAR test signature, rather than returning a canned "clean" response regardless of input.
• A fake OAuth2 authorization server implementing the real authorization-code + refresh-token grant flows, including one-time-use codes and CSRF state validation.

This is real verification of protocol correctness — not a substitute for eventually testing against the real service, but a much stronger check than "it compiles" and often catches the exact class of bug that would otherwise only surface against production. Explicitly flag, in both code comments and any handoff docs, that this is fake-server verification and that live verification against the real service is still an open item — don't let a passing fake-server test quietly become "this is proven to work."

When a real published test vector exists, use it

For anything implementing a spec with official test vectors (RFC 6238's TOTP Appendix B, for instance), verify against the actual published numbers rather than only checking internal self-consistency. This is a strictly stronger correctness check: internal self-consistency can pass while the implementation is confidently wrong in a way that happens to be symmetric; matching an external published vector rules that out.

A negative test is not optional polish

Auth-failure paths, permission-boundary checks, tampered-input rejection, and replay-prevention are not "nice to have if there's time" — in this project, testing them found real security-relevant gaps (a privilege escalation attempt that wasn't actually blocked until tested, a CSRF state that could be replayed until a check was added, a signature verification path that would have accepted a forged input). Budget time for these as part of "done," not after it.

Session and state continuity

Checkpoint working state as an artifact, not just as conversation history

In any environment where the working directory or container can reset (sandbox limits, context window limits, session boundaries), the actual recovery mechanism is a packaged artifact saved outside the working directory — a tarball, a committed file, whatever the environment's durable output channel is. Do this after every meaningful increment, not just at the end. Conversation history describing what you did is not a substitute for the file existing somewhere durable.

Before editing a living document, verify its current state — don't

assume your last edit landed Long sessions with many sequential edits to the same planning/handover document are exactly where partial failures compound silently: an aborted tool call, a str_replace that matched the wrong location, or an edit applied against stale context can leave a document with duplicated or contradictory sections (a "done" entry sitting right next to a stale "not started" stub for the same item). Before trusting that a previous edit succeeded, check: grep for the section header count (should be exactly 1), read the actual current content of the section you're about to touch, and only then edit. After editing, verify again — don't chain edits on documents based on what you intended to have written rather than what's actually there.

Keep a living plan document AND a separate onboarding/handover document

The plan document is the source of truth for what's done, what's tested, what's deferred and why — organized by phase/milestone, updated every time scope changes. The handover document is written for a fresh reader with zero context — environment setup that differs between the build environment and a real deployment target, decisions not to re-litigate, and a prioritized "what to do next" list. These serve different readers and get stale in different ways; don't collapse them into one document, but do keep them in sync with each other (a completed phase should be reflected in both).

Scoping and honesty

When a sub-feature is genuinely out of scope for the current pass, say so explicitly — don't half-build it

Real examples from this project: WebAuthn/passkeys were skipped entirely because they require CBOR decoding, which isn't in Go's standard library and is a large sub-project in its own right — attempting a partial implementation would have produced something that looked done but wasn't trustworthy. QR code rendering for TOTP setup was skipped in favor of the plain-text provisioning URI (which every real authenticator app accepts as manual entry) because real QR encoding is a separate, non-trivial piece of work. In both cases, the deferral was written down explicitly — in code comments and in the plan document — with the reason, rather than silently leaving a stub or, worse, a fake implementation that doesn't actually work.

Prioritize by what actually reduces risk, not strictly by original task order

When a later-numbered piece of work would close a security gap that affects everything already built (in this project: real TLS certificate issuance, since every existing self-signed-cert fallback was a flagged weakness), it's worth pulling that forward ahead of strict sequential order — and saying so explicitly, with the reasoning, rather than silently reordering.

A fact stated in a handoff document should be checked, not assumed durable

Version numbers, "as of" dates, and "extract this tarball" instructions written early in a long session go stale as the session continues. Before finalizing a handoff document, re-read it as if you were the person receiving it fresh — do the setup instructions still point at the right artifact? Does the "current phase" claim match what's actually true now?

Technical patterns worth remembering generically

• Raw TCP protocol work (client or test server): never assume one Read() call aligns with the sender's Write() call boundaries — TCP doesn't preserve them. Use a bufio.Reader with exact-length io.ReadFull reads for anything with a defined wire format (a command prefix, a length-prefixed chunk), not a single oversized buffered Read().
• A database connection pool capped to one connection (common for embedded databases like SQLite in a single-process server): never call a second query/exec while an earlier *sql.Rows from the same connection is still open — close it explicitly first, don't rely on a deferred close if you need the connection again before the function returns.
• Wire-format structs: always give them explicit serialization tags. A struct with no tags will round-trip fine in same-language tests (Go-to-Go) while silently breaking the actual wire contract expected by any other client.
• When verifying a hand-rolled crypto/signing implementation, prefer checking against a real published test vector over only checking internal round-trip consistency, wherever the spec provides one.

What this skill is not

This isn't a checklist to run through mechanically per line — it's a set of habits earned from what actually went wrong across a large real project. The common thread: verify against reality (real protocol, real published vectors, real negative paths, real current document state) instead of trusting that code which compiles or looks right is therefore correct.