https://blog.vnykmshr.com/writing/tags/reliability/Recent content in Reliability on vnykmshrHugoenFri, 16 Jan 2026 00:00:00 +0000https://blog.vnykmshr.com/writing/plumbing-and-abstractions/Fri, 16 Jan 2026 00:00:00 +0000https://blog.vnykmshr.com/writing/plumbing-and-abstractions/<p>The diagrams always show the happy path. Request goes in, response comes out, maybe a queue in between. Three boxes and two arrows. The failure modes live in the whitespace.</p> <p>Distributed systems work at demo time. In production, the third service fails after the first two succeeded: the debit went through, the credit went through, the write to the audit log dropped on a socket timeout, and both sides of the transfer had moved without anything recording <em>that</em> they had moved. The same row was about to be replayed the next morning. Someone noticed the mismatch six weeks later, at the end of the month, in a spreadsheet one of the ops people was maintaining by hand.</p>